Thursday, 5 November 2009

Bug Sending Long Binary SMS Messages to Java Apps on Motorola Devices

One of the more interesting technical support incidents of the past week comes from Mohit Kumar Sethi at 3i Infotech Consumer Services Ltd.


Mohit is using NowSMS Lite to send binary SMS messages to a Java application that his company is developing.

He noticed that when sending SMS binary messages longer than 140 bytes, the messages were received correctly by Nokia and SonyEricsson devices, however many Motorola devices received corrupted messages.

In GSM environments, a single SMS message can be no longer than 140 bytes.

When you send a text message, as long as the text only contains characters that are included in the GSM 7-bit character set , 160 7-bit characters are compressed into 140 8-bit bytes to produce the 160 character limit that we are so familiar with. (Note: 160 * 7 = 140 * 8)

There's more general discussion of SMS message size limits in the article at http://blog.nowsms.com/2007/06/long-sms-text-messages-and-160.html.

But back to Mohit's problem...

When sending a binary SMS message longer than 140 bytes, it has to be segmented into multiple SMS messages. A special header is added to each physical SMS message so that the receiving client knows that it is a multipart SMS message that must be reassembled by the client. These headers are known as segmentation or concatenation headers. 6 bytes (8-bits each) are required for these concatenation headers in each physical SMS message. These headers are placed in the User Data Header (UDH) field of the message, but they do count against the overall size limit of the message.

As a result, a long binary SMS message will be divided into 134 byte segments. Or 153 characters per segment for standard text messages. Or 67 Unicode characters per segment for Unicode text messages.

Mohit observed that with various Motorola RAZR and SLVR models, the received message would lose one byte for each of these segments.

He had his applications on the phones send long binary SMS messages back in to NowSMS.

When his application on a Nokia or SonyEricsson device sent a long binary SMS message, it generated a User Data Header like this: 0B0504232923290003070301

0B is the length of the user data header.

05 indicates that source and destination port information is present in the message.

04 indicates the length of the source and destination port information.

2329 2329 indicates source and destination ports of 2329.

00 indicates that the message is segmented using an 8-bit reference number.

03 indicates the length of the segmentation headers.

07 is the 8-bit reference number for the segmented message.

03 is the number of segments in the overall message.

01 is the number of the current segment (i.e,. 1 of 3).

Long binary messages sent by the Motorola devices were different. A typical user data header looked like this: 0C080400460301050423292329

0C is the length of the user data header.

08 indicates that the message is segmented using a 16-bit reference number.

04 indicates the length of the segmentation headers.

0046 is the 16-bit reference number for the segmented message.

03 is the number of segments in the overall message.

01 is the number of the current segment (i.e,. 1 of 3).

05 indicates that source and destination port information is present in the message.

04 indicates the length of the source and destination port information.

2329 2329 indicates source and destination ports of 2329.

Whether the segmentation headers or port information headers was not significant. But what was significant was that the Motorola device was generating messages that used 16-bit reference numbers.

Mohit determined that when the Motorola device received a long binary message using an 8-bit reference number, it converted this to a 16-bit reference number. If the overall message length including UDH was 140 before this conversion, the conversion would cause the last byte of data in that segment to be lost.

The Motorola device didn't seem to have any problems with standard long text messages being sent to the SMS client on the device, or with long binary WAP push messages. The bug seems to be in the part of the device code that routes messages to Java applications.

To make a long story short, this is an annoying bug. And if you're sending messages to Java applications, you may encounter it.

To deal with this scenario, we added a configuration option to NowSMS and NowSMS Lite, which would cause NowSMS to use 16-bit reference numbers when generating concatenated multipart messages. By default, NowSMS uses 8-bit reference numbers, which allows the most characters to be fit in a single SMS.

When 16-bit reference numbers are used, message length per segment rules change. For long text messages, instead of 153 characters per segment, only 152 characters per segment can be achieved.

For Unicode messages, instead of 67 characters per segment, only 66 characters per segment can be achieved.

For binary messages, 133 bytes of binary data can be included per segment, instead of 134 bytes.

This 16-bit reference number support was added in the 2009.11.04 release of NowSMS. This support will be carried through to future releases. For now v2009.11.04 has been made available as a special release at http://www.nowsms.com/download/nowsms20091104.zip (standard version) and http://www.nowsms.com/download/lite20091104.zip (NowSMS Lite).

To enable 16-bit reference numbers, edit SMSGW.INI, and under the [SMSGW] section header, add ConcatRef16Bit=Yes.

Note that when this setting is applied, 16-bit reference numbers are only used when NowSMS performs the segmentation (message submitted either via HTTP, or via SMPP using a single message_payload submission) ... NowSMS does not resegment messages that have already been segmented by the submitting client.

For more about sending SMS messages to J2ME apps, see http://blog.nowsms.com/2007/06/send-sms-to-java-appmidlet.html.

Monday, 2 November 2009

SMS and MMS with the Novatel Merlin XU870 ExpressCard Modem

If you are using NowSMS in a laptop with an available ExpressCard slot, the Novatel Wireless Merlin XU870 ExpressCard Modem is a good modem to use with NowSMS to send and receive SMS and MMS messages.

This modem supports HSDPA up to 7.2 Mbps (although locating the appropriate firmware driver update seems to be a challenge, without it, it is limited to 3.6 Mbps). That is good for receiving MMS.

However, the modem does not support HSUPA, limiting it to 384 Kbps for uploads and sending MMS.

Although we have yet to test it, the Merlin X950D would likely be a better choice for MMS, as it claims to support HSUPA up to 2.1 Mbps (Note: the X950D requires a firmware update to enable HSUPA and there may be some difficulty locating the firmware update). SMS speeds should not be a difference between the two modems.

Drivers for this modem can be found on the Novatel Wireless web site at http://www.novatelwireless.com/index.php?option=com_content&view=article&id=220&Itemid=272. (If this direct link is broken, navigate from http://www.novatelwireless.com/support/.)

There are no special considerations for using the Novatel XU870 modem with NowSMS. However, it should be noted that this modem does NOT support delivery receipts.

For general information on configuring NowSMS to send and receive SMS and MMS messages with a GSM modem, such as the Novatel Wireless Merlin XU870, please see the NowSMS Quick Start Guide at http://www.nowsms.com/documentation/quickstartguide/.

Frequencies supported by the Novatel Merlin XU870:

3G/UMTS/HSDPA/HSUPA - 850Mhz/1900Mhz/2100Mhz (Good for most of the world, plus AT&T USA)
EDGE/GPRS - 850Mhz/900Mhz/1800Mhz/1900Mhz (Global)

HSDPA Max Performance: Download speed - 7.2 Mbps; Upload speed - 384 Kbps

Thursday, 29 October 2009

Using NowSMS as a WAP Push Proxy Gateway (PPG)

NowSMS supports the Open Mobile Alliance (OMA) Push Access Protocol (PAP), which allows other applications to use NowSMS as a WAP Push Proxy Gateway (PPG).


Even though NowSMS has its own MMSC capability, we do have customers who are using NowSMS as a push proxy gateway for another vendor's MMSC.

In addition to providing PPG support for MMS, NowSMS can be used as a push proxy gateway for other types of push messages, including, but not limited to SyncML DM, SyncML DS, Wireless Village/IMPS, OMA Digital Rights Management, OMA E-Mail Notification (EMN), Service Indication and Service Load.

To post a push message via the PAP interface, perform an HTTP POST to the "web" port configured for the NowSMS web interface (not the MMSC), posting to a URL of "/pap". If user authentication is enabled in NowSMS, user identification can be included as URL parameters (e.g., "/pap?user=username&password=password"), or via an HTTP Basic Authentication "Authorization:" header.

A PAP post is a multipart MIME request, with the first MIME part being the PAP control document that specifies message recipient(s). The second MIME part is the data to be pushed.

An example of a service indication push being sent via PAP is shown below:

-----------------BEGIN EXAMPLE-----------------
POST /pap?user=username&password=password HTTP/1.0
Content-Type: multipart/related; boundary=mime-boundary; type="application/xml"
Content-Length: xxxxx

--mime-boundary
Content-Type: application/xml

--mime-boundary
Content-Type: application/xml

<?xml version="1.0"?>
<!DOCTYPE pap PUBLIC "-//WAPFORUM//DTD PAP 2.0//EN" "http://www.wapforum.org/DTD/pap_2.0.dtd"
[<?wap-pap-ver supported-versions="2.0,1.*"?>]>
<pap>
<push-message push-id="9fjeo39jf084@pi.com">
<address address-value="wappush=xxxxxxxxx/type=user@ppg.operator.com"></address>
</push-message>
</pap>
--mime-boundary
X-Wap-Application-Id: x-wap-application:wml.ua
Content-Type: text/vnd.wap.si

<?xml version="1.0"?>
<!DOCTYPE si PUBLIC "-//WAPFORUM//DTD SI 1.0//EN"
"http://www.wapforum.org/DTD/si.dtd">
<si>
<indication href="http://www.xyz.com/email/123/abc.wml" created="2009-10-25T15:23:15Z"
si-expires="2009-11-03T00:00:00Z">
You have 4 new emails
</indication>
</si>
--mime-boundary--
-----------------END EXAMPLE-----------------

Note that the "/type=user@ppg.operator.com" included in the WAP Push address specification is optional for NowSMS, and will be ignored, as only the relevant destination phone number is parsed from the request.

NowSMS will perform XML to WBXML conversions for WAP push of the following content types:

  • text/vnd.wap.si
  • text/vnd.wap.sl
  • application/vnd.oma.drm.rights+xml
  • text/vnd.wap.emn+xml (or application/vnd.wap.emn+xml)
  • text/vnd.wap.co
  • text/vnd.wap.connectivity-xml

Other types (including non WBXML based encodings such as "application/vnd.wap.mms-message" for MMS or "application/vnd.syncml.ds.notification" for SyncML SAN/DS) must be submitted via the appropriate binary encoding.

NowSMS supports the following text values for "X-Wap-Application-id":

  • x-wap-application:push.sia
  • x-wap-application:wml.ua
  • x-wap-application:wta.ua
  • x-wap-application:mms.ua
  • x-wap-application:push.syncml
  • x-wap-application:loc.ua
  • x-wap-application:syncml.dm
  • x-wap-application:drm.ua
  • x-wap-application:enm.ua
  • x-wap-application:wv.ua

Other "X-Wap-Application-id" header values must be submitted to NowSMS using their assigned decimal or hexadecimal value rather than the text name. (If using a hexadecimal value, preface the value with "0x" to indicate hexadecimal format.)

Further information on the WAP Push Access Protocol can be found from by downloading the WAP-247-PAP specification from http://www.openmobilealliance.org/tech/affiliates/wap/wapindex.html

Note that NowSMS does not support guaranteed delivery or delivery notifications.

NowSMS does not support base64 encoding for PAP content, binary content should be posted as 8-bit data using Content-transport-encoding: binary. We further recommend that the MIME part that includes the binary content have a "Content-Length:" header to avoid the potential of extra bytes (such as blank lines) being interpreted as part of the binary stream.

NowSMS sends WAP Push messages using connection-less WDP (Wireless Datagram Protocol) over SMS. SMSC connection from the NowSMS Push Proxy Gateway can use either SMPP, UCP/EMI or CIMD2. GSM modem connections can also be used as the SMSC connection for smaller test environments.

Special considerations exist for WAP Push in CDMA and CDMA2000 environments, namely the SMSC needs to support WDP Adaptation for SMPP. This is described in more detail at http://www.nowsms.com/support/bulletins/tb-nowsms-010.htm.

Monday, 26 October 2009

Sending SMS from Microsoft Excel

A recent posting on the NowSMS Discussion Board has some interesting information on sending an SMS a link within an Excel spreadsheet.



The gist of the posting is that you cannot use the Excel HYPERLINK command to do this. It would be logical to build a NowSMS URL to send a message, but the HYPERLINK command connects to the URL twice, resulting in duplicate messages being sent.

As an alternative, Des created a VBScript macro instead. This macro initiates an HTTP connection to NowSMS to send a message, dynamically creating a message based upon cell values in the Excel spreadsheet. I've quoted the details from Des below:

The alternative approach is to write a simple VBScript macro instead.

Today was my first attempt at writing one ... but I did manage to create a button in a spreadsheet, where when you click on the button, it reads data from the spreadsheet to send out an SMS.

Here's what I did ...

From the Developer menu in Excel, I added an Active X Command Button.

I then associated the following code with my command button:

Private Sub CommandButton1_Click()
Set objHTTP = CreateObject("MSXML2.ServerXMLHTTP")
URL = "http://192.168.0.222:8800/"
objHTTP.Open "POST", URL, False
objHTTP.send ("&PhoneNumber=" + Range("a9").Text + "&text=" + Range("a10").Text)

End Sub

In this particular case, I am extracting the phone number from cell A9, and the text to send from A10.

Once I exit design mode, I can click on the button, and it triggers this code, which makes the HTTP submission to NowSMS.

I'm using HTTP POST instead of GET to avoid some URL encoding issues.

It's not as easy as using HYPERLINK, but hopefully you can adapt this to your scenario.

Post any questions or commands in the discussion board posting at http://www.nowsms.com/discus/messages/1/41882.html.

Wednesday, 14 October 2009

SMS and MMS with the Alcatel One Touch X200 or X060 USB Modem

The Alcatel One Touch X200 and X060 USB Modems are good (but not great) modems to use with NowSMS to send and receive SMS and MMS messages.

For alternative modems, please see our other reviews: http://blog.nowsms.com/search/label/GSM%20modem

For general information on configuring NowSMS to send and receive SMS and MMS messages with a GSM modem, such as the Alcatel One Touch X200 or X060, please see the NowSMS Quick Start Guide at http://www.nowsms.com/documentation/quickstartguide/.

To point out one initial point of confusion when configuring NowSMS to work with the Alcatel One Touch modem, the modem driver installed for the modems is named "Modem Interface". When adding the modem to NowSMS, you must select the modem named "Modem Interface" (which is a less than intuitive choice of name).

The primary problem when using the Alcatel One Touch X200 or X060 modem with NowSMS is that each time your PC is rebooted, the modem will be inaccessible to NowSMS until the Alcatel "HSPA USB Modem" software that ships with the modem is loaded. Until that software is run, the modem functions only as a USB disk drive, and there is no modem functionality.

If you want to run NowSMS automatically each time your PC is restarted, after installing NowSMS and verifying that it works with the Alcatel One Touch modem, reboot your PC. If NowSMS is unable to access the modem after a reboot, you will need to follow the following steps.

We have been unable to identify a solution for this problem, other than configuring the Alcatel "HSPA USB Modem" software application to run at startup.

For some reason, on Windows Vista and Windows Server 2008 systems, the Alcatel "HSPA USB Modem" software requires that it be run with administrative rights.

Therefore, the best way to configure this software to be started automatically when the PC reboots is to configure the Windows Task Scheduler to run it at startup.

The Windows Task Scheduler can be accessed via the "Schedule Tasks" option under "Administrative Tools" in the Windows Control Panel.

Select "Create Task" to create a new task to load the Alcatel "HSPA USB Modem" software at startup.

On the "General" page, give the task a name, such as "Alcatel HSPA USB Modem", so that you can easily identify the task later. Under "Security Options", select "Run whether user is logged on or not", and "Run with highest priveleges". The other settings on this page can be left at their defaults.

On the "Triggers" page, select "New", and then specify to "Begin the Task" "At Startup". Press "OK" to save the trigger.

On the "Actions" page, select "New", and then specify the "Action" to be "Start a program". In the "Program/Script" field, press "Browse" and locate "HSPA USB MODEM.EXE", which is usually located in the "Program Files\HSPA USB MODEM" or "Program Files (x86)\HSPA USB MODEM" directory. Press "OK" to save the action.

On the "Conditions" page, clear any of the checked conditions, so that the program is always run at startup.

On the "Settings" page, check "Stop the task if it runs longer than", and specify "5 minutes". Also check "If the running task does not end when requested, force it to stop".

Press "OK" to save this task. You will be prompted for your username and password.

Reboot your PC to confirm that NowSMS can now access the modem properly when the PC is restarted.

For general information on configuring NowSMS to send and receive SMS and MMS messages with a GSM modem, such as the Alcatel One Touch X200 or X060, please see the NowSMS Quick Start Guide at http://www.nowsms.com/documentation/quickstartguide/.

For alternative modems, please see our other reviews: http://blog.nowsms.com/search/label/GSM%20modem

Frequencies supported by the Alcatel One Touch X200:

3G/UMTS/HSDPA/HSUPA - 850Mhz/1900Mhz/2100Mhz (Good for most of the world, plus AT&T USA)
EDGE/GPRS - 850Mhz/900Mhz/1800Mhz/1900Mhz (Global)

HSPA Max Performance: Download speed - 7.2 Mbps; Upload speed - 2 Mbps
UMTS Max Performance: Download speed - 384 Kbps; Upload speed - 384 Kbps
EDGE Max Performance: Download speed - 247.4 Kbps; Upload speed - 123.7 Kbps
GPRS Max Performance: Download speed - 85.6 Kbps; Upload speed - 42.8 Kbps

The Alcatel One Touch X060 is similar to the X200, except that its HSDPA download speed peaks at 3.6 Mbps, and there is no HSUPA support, limiting HSPA upload support to 384 Kbps.

NowSMS Support Summary: Supports sending and receiving both SMS and MMS messages. Does not support "Direct to Modem" ("Default" mode is ok).

SMS Delivery Receipts are NOT supported.

The Alcatel One Touch X200 and X060 modems do not support an external antenna (uses internal antenna only).