RVD Send SMS does not support Unicode characters

[marca56]

@scottbarstow commented on Tue Jul 18 2017

Related to #1903 there is no way to set encoding of SMS message to UCS-2 to support Unicode messages such as Japanese or other languages.

There needs to be a setting added to the SMS Verb to support the setting of encoding for the message.

---

@marca56 commented on Tue Jul 18 2017

@otsakir ... we need to get this issue fixed this week due to customer meetings in Japan and rest of Asia

[otsakir]

@scottbarstow , as far as i understand and having in mind RestComm/Restcomm-Connect#1903 discussion, we need to do the following:

1. Make sure RVD properly handles unicode characters and produces correct "UTF-8" xml/rcml.
2. An additional "Encoding" attribute is included in the UI and added to RCML output.
3. Implement support for "Encoding" attribute at restcomm side.

So, between RVD/Restcomm we have a clean compatible contract of using UTF-8 encoding in the payload. Restcomm will then use the 'Encoding' attribute if present to encode the SMS accordingly.

@scottbarstow , @gvagenas , @leftyb please tell me what you think

@marca56 fyi

[scottbarstow]

@otsakir this sounds right. UTF-8 should be the default if the new attribute is not touched / used. If used, it should set the attribute appropriately on the SmsSessionRequest.

[otsakir]

thanks @scottbarstow. Let me try to clarify what happens in terms of RCML:

1. The RCML response itself generated by RVD will always be UTF-8 encoded no matter what language of SMS the user wants to send. I'm talking about this:

<?xml version="1.0" encoding="UTF-8"?>
<Response>
...
</Response>

The first line of the snippet above will not change. The SMS elements included will be encoded in UTF-8 too. So we always have a proper UTF-8 response.

2. RVD/SMS element will include a new Encoding property where the user be able to select between UTF-8|UCS-2|GMS or whatever literals/constants rcomm or smsc support.

Thus, sending a Japaneese SMS from RVD will involve an RCML response like the following:

<?xml version="1.0" encoding="UTF-8"?>
<Response>
    <Sms Encoding="UCS-2">Hello World in Japaneese encoded in UTF-8</Sms>
</Response>

[scottbarstow]

Makes sense to me.

[gvagenas]

From the discussion we had with @otsakir:
The RCML will be UTF-8 encoded, same for the SMS text.
RC will have to encode the SMS Text body to the charset specified at Encoding parameter of the SMS verb

[otsakir]

@gvagenas , before proceeding to the implementation tell me if you agree with the following changes in RCML spec:

• SMS element will have an additional attributed named "encoding".
• The "encoding" will be set to the values described here:
  Issue with encoding of Unicode characters Restcomm-Connect#1903 (comment), namely:
  UCS-2, UTF-8, GSM . @lefty ?

Also, what is the default encoding value (if any) restcomm will use if no "Encoding" parameter is set in the REST api ? @leftyb , @gvagenas ? I will use that as a placeholder in the RVD UI.

[scottbarstow]

@otsakir pretty sure the default is GSM as it stands now. @leftyb can confirm