[qtopia] +CUSD wronlgy decoded

[vnevoa]

Class 0 SMS messages (the kind that show up immediately on screen and don't get saved) are not decoded right in my GTA02.
When I finish a GSM call, my operator sends me a class 0 SMS with the remaining amount in my pre-paid account. This used to work ok until one of those UTF-8 patches some time ago (sorry, don't know when it happened). Now the message shows up like a string of numbers instead of text.
For example, my FR just showed me this:
00340035002E00370033002000
Which I presume means this:
45.73 €

My SW is a OM2008.8-update jffs2 image that has been upgraded everyday via opkg.

[vnevoa]

Actually, it looks more like UTF-16.

[zecke]

Please see ​http://wiki.openmoko.org/wiki/Bug_Filing_Policy on how to bugs and specially GSM issues.

[zecke]

add a report somewhere :)

[vnevoa]

Ok, I've added the logread.
My reproduction of the problem starts at line 566.
Steps taken:
1 - compose and send an SMS message to myself (body has "test");
2 - receive the class 0 SMS with the pre-paid account's current sale;
3 - receive the normal SMS I wrote to myself (say NO to qtopia's question).
Anything else?

[erin_yueh]

Replying to vnevoa:

Ok, I've added the logread.
My reproduction of the problem starts at line 566.
Steps taken:
1 - compose and send an SMS message to myself (body has "test");
2 - receive the class 0 SMS with the pre-paid account's current sale;
3 - receive the normal SMS I wrote to myself (say NO to qtopia's question).
Anything else?

from this log:
you receive a SMS and stored it to SIM card index 16.
Sep 23 23:30:22 om-gta02 user.notice root: AtChat? : N : "+CMTI: "SM",16"
what is this SMS?
Sep 23 23:30:24 om-gta02 user.notice root: AtChat? : F : "+CMGL: 16,0_{24"
621 Sep 23 23:30:24 om-gta02 user.notice root: AtChat? : F : "07915391131213F4040C9153914352823500008090323223154005F4F29C0E02"}

07 91 5391131213F4 04 0C 91 539143528235 0000 80903232231540 05 F4F29C0E02
'0000' those numbers means TP-PID and TP-DCS. The special case of bits 7..0 being 0000 0000 indicates the Default Alphabet as in Phase 2.
the message text are encoded by GSM Alphabet, not UCS2.

you can read SMS format by this site:
​http://www.dreamfabric.com/sms/

[vnevoa]

Erin, I have no idea if you are making a statement or asking a question...
I also don't know how to interpret the log file. If I knew, I would have debugged the problem myself and provided better info.
If you need more info or clarifications, please ask in a clearer way. Thanks!

[erin_yueh]

mmh....you could use this site and it can decode/encode SMS message. Coz I tried to read your log, i cannot find that SMS you said it's UCS2 and class 0 one.
​http://twit88.com/home/utility/sms-pdu-encode-decode

but i found it's not SMS problem and it's a supplement service data from your operator. since we changed the character set of GSM modem, from IRA to UCS2. It seems like this problem. I will cc this bug to Tick. Thanks a lot for reporting this bug!

[vnevoa]

I'm glad you understood it, because I still don't! :)

Anyway, here's a new section of logread, this time with a simpler use case:
1 - make a voice call;
2 - end the call;
3 - wait for reception of "current pre-paid sale" message.
So this time there are no other SMS messages in the log to create confusion. :)

Sep 25 13:56:02 om-gta02 user.notice root: AtChat? : T : "^{Z"
Sep 25 13:56:02 om-gta02 user.notice root: AtChat? : T : "ATD21424XXXX;"
Sep 25 13:56:02 om-gta02 user.notice root: AtChat? : N : "%CPI: 1,9,0,0,0,0,"214242051",129"
Sep 25 13:56:04 om-gta02 user.notice root: AtChat? : N : "%CPI: 1,3,0,0,0,0,"214242051",129"
Sep 25 13:56:04 om-gta02 user.notice root: AtChat? : N : "+CSSI: 1"
Sep 25 13:56:05 om-gta02 user.notice root: AtChat? : N : "%CPI: 1,4,0,1,0,0,"214242051",129"
Sep 25 13:56:06 om-gta02 user.notice root: AtChat? : N : "%CPI: 1,5,0,1,0,0,"214242051",129"
Sep 25 13:56:06 om-gta02 user.notice root: AtChat? : N : "%CPI: 1,2,1,1,0,0,"214242051",129"
Sep 25 13:56:08 om-gta02 user.notice root: AtChat? : N : "%CPI: 1,6,0,1,0,0,"214242051",129"
Sep 25 13:56:08 om-gta02 user.notice root: AtChat? : F : "OK"
Sep 25 13:56:08 om-gta02 user.notice root: Modem : QModemCall::dialRequestDone()
Sep 25 13:56:42 om-gta02 user.notice root: Modem : QModemCall::hangup()
Sep 25 13:56:42 om-gta02 user.notice root: AtChat? : T : "}Z"
Sep 25 13:56:42 om-gta02 user.notice root: Modem : hangup groups
Sep 25 13:56:43 om-gta02 user.notice root: AtChat? : T : "AT+CHLD=1"
Sep 25 13:56:44 om-gta02 user.notice root: AtChat? : N : "%CPI: 1,1,0,0,0,0,"214242051",129"
Sep 25 13:56:44 om-gta02 user.notice root: AtChat? : F : "OK"
Sep 25 13:56:44 om-gta02 user.notice root: AtChat? : N : "%CPI: 1,7,0,0,0,0,"214242051",129"
Sep 25 13:56:48 om-gta02 user.notice root: AtChat? : N : "+CUSD: 0,"00340035002E003200350020004500550052",15"

So, the content of the received message (that is shown to me by qtopia) is the last line: "00340035002E003200350020004500550052" (and is shown "as-is", without any translation - or even line breaks, but that's another problem).
Using the page you linked does not seem to work... I don't think it can translate this.
Thanks!

[vnevoa]

Using this page:
​http://www-atm.physics.ox.ac.uk/user/iwi/charmap.html
I was able to correctly decode my received message "00340035002E003200350020004500550052" = "45.25 EUR"
which is the content I expected. :)
Conclusion: it uses Unicode, but it is not a complete PDU - it has no meta info.

[vnevoa]

Ok, after reading #1832 and #1226, I know that the received message is a Unicode USSD.
So, when is Treviño's patch for "*xxx#" commands and USSD support going to be integrated? :)
Obviously, it would also need to be checked for Unicode compatibility.

[zecke]

Which patch? I have not noticed anything mentioning a patch.

[vnevoa]

From the conversations in #1832 and the Community ML (which you were a part) I can see that Treviño has apparently not submitted a source patch yet, although he did upload a patched binary which people tested and confirmed working. He removed the binary at the community's request but failed to upload the code yet (AFAIK).

So, time to ping Treviño again, I guess... I'll mail him, just in case.

In case you decide to leave this ticket open, maybe you could change my title to something less misleading? like "USSD Service Messages with Unicode not translated"... Thank you. :)

[Treviño]

Well, the problem is in USSD parsing and this is due to the last change to UCS2 format... I'm quite near to fix the issue, however another soulution would be putting back the IRA or GSM charset after that the modem has read the SIM contacts.
This would be easier but I don't know if it will be the best solution.

[Treviño]

Ok, it should be fixed in #2038

[tick]

Sorry, It's my fault, It's not there yet.

[tick]

​http://trolltech.com/developer/task-tracker/index_html?method=entry&id=212600 the patch needs to be merged.

[zecke]

Okay I landed the patch but it might not fix the bug. As we still need to properly decode the string according to the charset as Trevino pointed out.

[tick]

Patch for decode the ussd