RE: Character encoding in xAP messages

["ian.bird@xxxxxxxxx" <ian.bird@xxxxxxxxx>]

I am not expressing a strong view here but moving away from ASCII would
break all my embedded applications as I use the upper range (msb =3D 1) for
my own user defined characters. Since I only use a byte to index the
character array there is no room for user defined characters and full
utf-8=
.

I am not particularly bothered but I will not be updating my embedded
applications that exist now. On the up side I think I am the only one using
the hardware too which is why it is not a big problem.

If this change is made I will probably update my current application but in
the documentation I will state the character set used. User defined
characters will replace the standard UTF-8 set at some points. There should
be enough room for both though assuming people use the 'normal' printed
characters.

Ian



Original Message:
-----------------
From: Malcolm Green groups@xxxxxxx
Date: Sun, 23 Jan 2005 23:18:47 -0000
To: xAP_developer@xxxxxxx
Subject: RE: [xAP_developer] Character encoding in xAP messages


If the data you're sending is 7-bit ASCII, then encoding using ASCII or
UTF-8 will yield identical results.  So an existing application which is
only using 7-bit ASCII should still work OK if you change to UTF-8.  As
with your Exstreamer client, if you try to send data which includes an
extended character set, it currently won't work correctly with ASCII
encoding, but will if you change to UTF-8.  So I don't see any downside to
the proposed modification.  Incidentally, wherever filename strings are
being sent via xAP, I think it really ought to cater for Unicode strings as
these have been supported in Windows filenames since NT4.=20=20
=20
I intend to release another version of TelCanto with further xAP support
this week.  My current choices are:
- stick with the existing ASCII encoding, which fails to handle several
files in my music library correctly.
- change to using binary encoding, which will require a change to the Audio
schema and changes to all other applications which use it.
- use UTF-8 encoding which works correctly with both TelCanto and xAP
Desktop as far as I can tell (I've not tried any other applications).
=20
I don't think the first option is viable in the long term, as it just
doesn't work correctly!  I obviously favour the third option, but am happy
to take advice from those with a deeper knowledge of xAP than myself.
=20
Malcolm


_____=20=20

From: Edward Pearson [mailto:edward.mailgroup@xxxxxxx]=20
Sent: 22 January 2005 13:46
To: xAP_developer@xxxxxxx
Subject: RE: [xAP_developer] Character encoding in xAP messages


The spec says ASCII and, thinking it through, it probably means ASCII and
not UTF8.
=20
I'd previously observed accented characters not transferring correctly in
song titles between my custom Exstreamer app and its xFx controller. The
'fix' to xFx below would allow these characters to be encoded but in a way
that's off spec. The Barix Control Language (tinyBasic) that the Exstreamer
code is written in expects ASCII of course so (not tried it) will barf if
it starts to see double-octet characters in messages.
=20
Sounds like using the binary operator is the only safe way to go for
unicode strings then. But I bet many xAP progs don't deal with this
correctly.

_____=20=20

From: Mark Harrison (Groups) [mailto:mph@xxxxxxx]=20
Sent: 19 January 2005 23:24
To: xAP_developer@xxxxxxx
Subject: Re: [xAP_developer] Character encoding in xAP messages


Edward Pearson wrote:=20

Oh cool - you fixed an old bug in my Extreamer client that'd I'd never had
the time to track down!
=20
To formalise this - the xAP specification says ASCII everywhere. Was this
done for a specific reason or were the specification definers just being a
bit loose here? Maybe the spec should be revised to say UTF8.
=20
Right, I'm off to listen to some German techno with umlauts in the title
and other diacritically challenged corners of my music collection...

I had understood that non-ASCII stuff should all be represented in a binary
format, and therefore use the "keyword!" rather than the
"keyword=3D" form.=
..
and that the UTF-8 encoding should be specified as the encoding method in
notes section of the Schema definition

>From the point of view of my own code, I honestly don't know how, say,
xAPlogger would respond if you tried to log a xAP message containing
"Pot
s=C4=83 m=C4=83n=C3=A2nc sticl=C4=83 =C8=99i ea nu m=C4=83
r=C4=83ne=C8=99t=
e." (which is, in case you were
wondering, the Romanian for "I can eat glass.")=20

Anyone got access to a suitable installation this evening to try?

I have no particular objection to a Specification (xAP 1.2.A ?) revision to
say "UTF-8" in appropriate places.

I seem to remember that ASCII was picked support by LCD / VFD display
devices at a time when it was felt that these would be a common part of
most xAP installations.... I'm not sure how many have been produced, and
whether they all support non-ASCII.

M.


_____=20=20