ICTworks by Inveneo

I am Michael Benedict and while working with SMS-based applications I’ve noticed an air of mystery around the issue of reliability. I hear colleagues say ambiguous things like “carriers consider SMS to be lower priority than voice”, or “SMS delivery isn’t guaranteed”.

My personal experience has been that messages are almost always delivered quickly and correctly, but I’ve heard stories of hours-long delays, corrupt data, and occasionally messages that never arrive. Since I am working on two projects that depend on reliable SMS service — one involves field-based data collection and another employs SMS as a transport layer between computers — I’m curious about how factors such as network, location, and time of day impact message transmission.

I found myself in Mwanza, Tanzania last week with two GPRS modems and a local partner who was unenthusiastic about the work I was actually there to do, so I tried a little experiment. I bought SIM cards and airtime for three of the major TZ networks, put two at a time in the modems, and wrote a simple python script using Adam Mckaig’s excellent pygsm library.

The script sent messages in all permutations at an arbitrarily chosen interval of about 20 minutes. This means that four messages were sent at a time: each SIM to itself and to the other SIM, repeated about every 20 minutes for periods of up to 27 hours. Outgoing messages noted the origin and destination networks, send time, and a sequential message number to make it easier to spot lost messages. Incoming SMS were stamped with the time received, and the delay in seconds between sending and delivery. Messages were sent two seconds apart to reduce errors from the modems trying to send and receive at the same time, and results were recorded in a CSV file.

a) Airtel and Zantel

Zantel is one of the smaller networks in TZ, but I was having technical problems with my Tigo SIM so started with Zantel and Airtel. Airtel was known as Zain until recently when Zain was acquired by Baharti Airtel, based in India. The script ran from about 1AM on January 14 to just before 5AM on January 15. Here are the results:

Delivery times and dropped messages between Airtel and Zantel Tanzania. Click to see scale. Note that the Airtel modem stopped sending and receiving at about 7AM on Jan 15, and the script hung until I woke up shortly after 8 and reset the modem. I do not know what caused the problem.

331 messages were sent and 10 (3%) were dropped. Delivery times fell into three bands. Airtel-Airtel (i.e. the Airtel SIM sending to itself) was easily the fastest, with messages typically delivered in 6 or 7 seconds. Messages in either direction between Airtel and Zantel took roughly twice as long, typically between 13 and 16 seconds. One outlier, an Airtel to Zantel message sent between 2 and 3 AM on Jan 15, took 24 seconds to arrive, and the only dropped Airtel-Airtel message occurred around the same time. Delivery times for Zantel-Zantel messages were split unevenly: one cluster near 6 seconds, another around 15 seconds, and a third around 32 seconds. 9 messages sent from Zantel-Airtel were not received.

Some aspects of this plot seem reasonable. Airtel is a larger company with wider network coverage, so it’s not too far fetched to imagine that messages sent from an Airtel SIM to itself would be the fastest to arrive. By that reasoning I’d expect messages sent between the two networks to take somewhat longer, and they do. The Zantel-Zantel delivery times are less intuitive. I don’t know what would account for the three distinct time ranges for delivery. It is also unexpected to see that 9 Zantel-Airtel messages were never delivered but no Airtel-Zantel messages experienced this problem.

b) Airtel and Tigo

SMS delivery times between Airtel and Tigo SIMs on January 14, 2011. Click image to see scale. *Messages marked unreadable appear to have reached the modems but could not be read, and resulted in an (unfortunately unlogged) exception in pygsm.

192 messages were sent and 3 (1.6%) were not readable by the modem. Four distinct delivery time ranges emerged. I eliminated one outlier from the plot to make it easier to view: on January 15 at 11:47, a Tigo-Airtel message took 129 seconds to arrive. I believe that the 3 unreadable Tigo-Tigo messages reached the modem, but the script raised an exception and the messages were not read. Unfortunately I was not logging pygsm’s debugging output so I am marking the missing messages as “unreadable” instead of dropped. I will have to log output in future runs and see if I can reproduce the problem. I didn’t expect Tigo-Airtel messages to arrive slower than Airtel-Tigo. The difference is small but it would be interesting to know why it’s there.

c) Two Tigo SIMs.

The purpose of having the modems send messages to themselves in the previous two runs was to test delivery times within a network. It occurred to me that sending from a SIM to itself may not be equivalent to sending between two different SIMs on the same network, so I bought a second Tigo SIM:

SMS delivery times between two Tigo SIMs. Click to scale.

164 messages were sent and 1 (0.6%) was not received. The data shown excludes one outlier: the first Tigo A-Tigo B message took 457 seconds to arrive. So: is SIM to SIM within a network equivalent to sending from a SIM to itself? Sometimes. I don’t know why it varies, but note that the number for Tigo A is +255716379091 while Tigo B is +255717435798.

In particular, after the 255 country code the numbers have different prefixes, 716 for A and 717 for B. Typically Tanzanian SIMs on the same network have the same first 3 digits, and the difference here may indicate that Tigo’s servers treat the two numbers differently. The Airtel/Tigo data above used the SIM that is labeled as “Tigo B” here.

Thoughts and next steps

Unexpected patterns emerged, including asymmetric delivery times between networks, that I would like to understand better. The few dropped messages do not seem to follow a clear pattern, which is a concern for both human and computer-focused SMS applications where missing messages can cause confusion and frustration.

It seems possible that having the modems send all 4 messages with only a 2 second delay between sending could be impacting the results. It would be interesting to revise the script to randomly offset the message send times by a few minutes and see if that increases reliability.

This is not a scientific experiment since there are variables such as time of day, day of the week, network traffic, modem location, weather, and likely many others that are not being controlled for. Still I think the data gives a representative snapshot of network performance. Overall SMS delivery was quite good. With the exception of a few of outliers most messages were delivered in under a minute, and Airtel consistently delivered on-network SMS in less than 10 seconds.

Occasional dropped messages show that safeguards need to be in place if SMS is used for critical applications. Some preliminary timing data I took in Uganda suggests that the networks there are not nearly as reliable. I’m looking forward to looking more carefully at Ugandan networks when I’m back.

If you like this sort of thing and also have too many modems and not enough social life, feel free to try out the script yourself. I’d be interested to see what the results look like in other countries.

A special thanks to Henry Corrigan-Gibbs and Matt Berg for interesting discussions about the timing script and SMS reliability.

This was posted first as SMSpecially Reliable? Part 1 and is repulblished here with permission

Kenya's Prime Minister Raila Odinga has convened a 15-member National Communications Strategy Taskforce which has been mandated to come up with overarching communications guidelines that all public sector agencies and departments will be expected to draw from.

This according to Capital FM, has 12 months to formulate a National Communications Strategy and Policy Handbook under the theme "one government, one voice".

Spokesman, Dr Alfred Mutua said that the taskforce was working on ways through which the government would bypass the media and disseminate its plans and policies directly to the people. Towards this end he said his office had set up a new SMS service through which the members of the public will receive 'breaking news from the government'.

"If you send an SMS to 4458, we will be sending you government breaking news so that you can hear it before the media manipulates or takes it or breaks it apart," he said.

The question is: have you signed up for these alerts? And if you have, what are some sample text messages? Are they helpful? Useful? Would you pay for them?

With all the hype around the use of text messaging on mobile phones, like Google Trader or FrontlineSMS, it seems that the future of communication in Africa will be SMS.

Yet, Erik Hersman recently asked a great question: Should we be Building SMS or Internet Services for Africa?

David looking for an IP-based future

What a lot of people don’t realize is that for the first time, last year, mobile phones shipped to Africa with data service capabilities outnumbered the simple SMS-only phones that are so prevalent on the continent (Gartner 2009). Of course, this doesn’t mean that there will be a majority of IP accessible phone users immediately, but it is on its way.

Equally important to understand, and a point that increases the momentum of the mobile services over IP argument, is the fact that where there is mobile penetration, there is also available data services. This stands true in Uganda, where MTN says there is 92% GPRS coverage on their network. It’s even true in countries still trying to catch up, like Liberia, where though there are only islands of coverage, that coverage generally comes with data.

Expanding on Erik's observations, I see several trends that will have the next generation of African ICT solution developers crunching IP code, not developing SMS extensions:

High SMS costs

Currently, text messaging is one of the the most profitable activities for mobile network operators. An SMS costs them next to nothing to transport and deliver, yet those 140 characters are grossly expensive to the end user. Any type of automated SMS traffic (Twitter for example) would quickly bankrupt a company who attempted a large scale activity. This is stifling SMS creativity.

Ability to innovate

Mobile network operators continue to run very closed systems, only allowing very few applications to run directly on their networks. They do allow IP-based services however, with data plans. And with an Internet presence, entrepreneurs can also get international attention and investment, and attract computer users too.

Demand for video

Already, market IT entrepreneurs are selling video compression services so movies can be played on phones. As soon as there is a critical mass of data service phones and plans, expect to have movie services start offering daily downloads. Churches will lead (a sermon a day), and commerce will follow closely behind. Services like this are impossible with SMS, but easy with IP.

Like what you read? Then subscribe to ICTworks updates via RSS, Email, or Twitter

.