[Message Prev][Message
Next][Thread Prev][Thread Next][Message
Index][Thread Index]
Re: xAP EOM identifier in xAP v1.3
--00148531b766446b4704739d9148
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable
Hi Edward,
Please see in-line responses...
2009/9/15 Edward Pearson <edward.mailgroup@xxxxxxx>
>
>
> The double 0x0a terminator works for me, it=92s simple to implement
=96
> already done for my stuff.
>
>
>
...but it isn't robust unless the message is sent as a single datagram, and
if it is sent as a single datagram, the os should deliver it as such -- and
if it doesn't, adding an eom marker isn't going to help.
> For reliable streams, such as TCP, I generally frame the messages with
ST=
X
> and ETX.
>
>
>
> I thought all this business about 1500 bytes was somewhat urban myth
(at
> least as it applies to xAP). The data packet size is not what=92s
importa=
nt;
> it=92s how the particular IP stack implementation deals with
>datagrams<
> that=92s the key.
>
I don't understand what you are syaing here? The data packet size is
inextricably linked to the IP stack. What aspect is it that you consider to
be an urban myth?
> I only have experience of the two most recent Windows stacks (XP and
Vist=
a)
> which I agree are likely more capable than old embedded stacks. My
> experiments with Wireshark show that those stacks will happily deal
with
> fragmentation and defragmentation. 64KB in a single datagram? No
problem =
if
> your receive buffer is long enough (even if you force a small MTU).
>
Yes, agreed, OSes perform fragmentation for you. The individual fragments
have a maximum size as determined by the MTU. For pragmatic, practical
reasons, xAP needs to define a maximum overall message size, and for
convenience's sake that was set as equal to the 'standard' MTU size.
Device=
s
which use a smaller MTU *should *fragment and reassemble seamlessly
provide=
d
that the correct socket options have been set to define the maximum UDP
packet size. By electing to use a single MTU's worth of data, we we avoid
the overhead associated with fragmentation and reassembly (principally
memory buffering) which is a good thing. When reassembled, the os should
deliver a datagram as a complete entity in one go (irrespective of the mtu
size, assuming that the datagram falls within the maximum datagram size
defined for the stack). If the sender doesn't send a message as a single
datagram, then the whole thing falls apart because effectively you are then
doing fragmentation and reassembly at the application layer, and that won't
work because the ordering across datagrams is not guaranteed and individual
datagrams may get lost.
> If anything goes wrong (eg, missing fragment) the whole datagram is
> dropped. I can=92t see how packet re-ordering can happen on a single
LAN =
for
> broadcast UDP =96 there are no multiple paths and no retry mechanism.
> Certainly never observed it =96 I assume the stack would again just
drop =
the
> entire datagram.
>
Re-ordering of datagrams can occur for multiple reasons on a single lan. A
sender may send individual UDP packets in any order it chooses. This
commonly occurs with fragmented packets originating from a multi-threaded
sender, which can be interleaved with smaller, non-fragmented datagrams as
required (optimising transient memory use, as soon as they are sent, the
buffer can be released). A switch is not obliged to retransmit packets in
the order in which they are received. And most fundamentally, the specs do
not require UDP packets to be ordered so, even if you don't observe it, it
=
*
can* happen, and sooner or later interoperability issues will arise if the
working assumption is made that they always arrive in order.
>
> A bigger issue for me, and the reason for me experimenting a few times
wi=
th
> TCP stream serving from hubs, is datagram loss over WiFi networks.
This i=
s
> greatly accentuated for UDP when you use broadcast (as we do) from
wired =
to
> wireless (fine the other way as it=92s not really a broadcast packet
till=
it
> gets to the AP) as the access point and NIC can not longer apply their
> ack/nak procedure at the transport level. I commonly see xAP datagram
los=
s
> rates from wired to wireless connections of 20%. So I=92d like us to
agre=
e on
> a standard transport wrapper for TCP streams which a lot of platforms
wou=
ld
> find useful.
>
>
>
I'd suggest using the same framing as the "transport wrapper", as
this then
allows for code to be shared across transports. If xAP was extended to
support TCP, then that should also include a formal discovery mechanism by
which the IP address/characteristics of the hub can be discovered (over UDP
xAP?)
Patrick
> *From:* xAP_developer@xxxxxxx [mailto:
> xAP_developer@xxxxxxx] *On Behalf Of *Patrick Lidstone
> *Sent:* 14 September 2009 14:19
> *To:* xAP_developer@xxxxxxx
> *Subject:* Re: [xAP_developer] xAP EOM identifier in xAP v1.3
>
>
>
>
>
> So the xAP delimiters for serial are defined in the 1.2 xAP spec here:
>
> http://www.xapautomation.org/index.php?title=3DProtocol_definition#Transp=
ort_Wrapper
>
> To the best of my knowledge, the MTU size for an ethernet frame is
1518
> bytes, which leads to a UDP packet MTU of 1500 bytes and this is the
size
> that is adopted by the majority of operating systems. Internet routers
(i=
.e.
> ISPs) sometimes use an MTU of 576 bytes, but this wouldn't be relevant
to
> xAP since the traffic doesn't pass over the wider net, or if it does,
it'=
s
> generally gatewayed via a TCP/IP connection.
>
> If a device is receiving fragmented UDP packets, I think the same
issues
> arise as those related to extending the xAP message size beyond 1500
byte=
s -
> what happens if a fragment gets discarded.
>
> If you take the scenario:
>
> Part 1(a), Part 1(b), Part 2(a), Part 2(b), Part 3(a), Part 3(b)
>
> --- first there is no guarantee that the parts will be delivered in
order=
,
> and second, if part 1(b) was dropped, and you were blindly assembling
> messages based on the proposed double-0a terminator, you'd end up with
a
> message comprising part 1(a), 2(a) and 2(b) which is not only
obviously
> corrupt, but also possibly larger than the maximum xAP message size,
blow=
ing
> away your buffers.
>
> I think the solution is to probably parse incoming messages on the
fly,
> byte-by-byte. You can then at least reset your state when you
encounter t=
he
> xap-header, and if you count open and close curly braces, you can tell
wh=
en
> you have an apparently complete message. This won't solve the issue of
UD=
P
> fragments being potentially received out of order, but so long as we
are
> dealing with a single LAN, and fragmentation occurs at the receiver,
we w=
ill
> be ok I think.
>
> It is absolutely possible for UDP packets to be discarded, and the way
we
> deal with this in xAP is to accept that this can happen to xAP
messages, =
and
> layer application level acknowledgements where knowing that a message
has
> been received is critically important - whether explicitly or
implicitly
> through status messages. There are various schemes that could be
adopted =
to
> allow a receiver to detect lost messages (e.g. sequence numbering),
but t=
hey
> quickly become quite onerous, and assume that the originator keeps a
copy=
of
> the original message or is able to reconstruct it - which is
non-trivial =
for
> the many xAP nodes.
>
> Perhaps you can share more of the specific details of the device(s) in
> question (manufacturer, docs, o/s etc), and their specific behaviour,
whi=
ch
> seems a bit anomalous?
>
> Patrick
>
> 2009/9/13 Kevin Hawkins <yahoogroupskh@xxxxxxx>
>
> One of the issues seems to be that there is conflicting views as to
the
> length of a a UDP data packet payload. Some people cite 500 or 512
> characters and some 1500. Regardless in some low memory/performance
> devices it is being reported, that even with UDP, packets are being
> received from the buffer either truncated or appended back to back.
> The latter I assume is due to the speed of the device in servicing the
> buffer.
>
> We have an opportunity to protect against this with v1.3 and the
double
> '0A' seems the most compatible. I would be loathe to support
anything
> that wasn't backward compatible.
>
> K
>
>
> Patrick Lidstone wrote:
> >
> >
> > I will dig it out - it included an optional checksum I think, and
IIRC
> > was framed by stx and etx (a kind of pseudo industry standard). I
> > certainly used it with the PIC serial stuff and the xAP-serial
bridge.
> > Re.: long message truncation and concatenation: If we need to
support
> > messages that are larger than one UDP packet, then there are
> > additional complexities and the proposed scheme won't work as
intended
> > as the ordering of UDP messages is not guaranteed. I'm happy to
help
> > refine a spec to support these capabilities, but it is moving
away
> > from the basic ethos of xAP somewhat, as devices would have to be
able
> > to buffer received messages, and that raises the bar
considerably.
> > Perhaps there is scope for co-existence of a long and standard
message
> > protocol though?
> >
> > Patrick
> >
> > 2009/9/13 Kevin Hawkins <yahoogroupskh@xxxxxxx
>
> > <mailto:yahoogroupskh@xxxxxxx>>
>
> >
> > ... Oh ... where is that in the spec ? it might be all we
need.
> >
> > This is also tied in with some aspects of long message
truncatio=
n
> > and concatenation of messages received in UDP receive buffers
> > though...
> >
> > K
> >
> > Patrick Lidstone wrote:
> > >
> > >
> > > The original xap spec provides extensions for framing a
message
> over
> > > async serial which also delimit the start of the message
- you
> don't
> > > need this 'hack' if you follow the original spec for
non-UDP
> > transports.
> > >
> > > Patrick
> > >
> > > 2009/9/13 Kevin Hawkins <yahoogroupskh@xxxxxxx
> > <mailto:yahoogroupskh@xxxxxxx>
>
> > > <mailto:yahoogroupskh@xxxxxxx
>
> > <mailto:yahoogroupskh@xxxxxxx>>>
> > >
> > > We have been asked on several occasions how to
detect the
> > end of a
> > > xAP messager as there is no unique EOM character.
Typically
> > in any
> > > reasonable sized packet structured transport eg UDP
then the
> > packet
> > > provides such an indication but on systems with
small packets
> or
> > > non eg
> > > asynchronous serial this is not useable.
> > >
> > > In discussing this with the specification team we
must
> > consider
> > > backwards compatability and so we do not wish to
alter the
> > > specification
> > > to include a unique EOM character. What we do
propose
> > however is that
> > > xAP v1.3 will specify that all messages should end
with two
> > > consecutive
> > > chr(10)'s immediately after the closing '}'
> > >
> > > ie ..... 7D 0A 0A
> > >
> > > Some apps, even v1.2 ones, already do this. We
don't
> > believe this
> > > will cause any backwards compatibility issues and it
will
> > always be
> > > unique within a xAP message.
> > >
> > > So, in developing xAP v1.3 apps could you therefore
append
> > two 0A's
> > > at the end of your messages , and of course handle
incoming
> > messages
> > > containing such.
> > >
> > > K
> > >
> > >
> > > ------------------------------------
> > >
> > > Yahoo! Groups Links
> > >
> > >
> > > mailto:xAP_developer-fullfeatured@xxxxxxx
> > <mailto:xAP_developer-fullfeatured@xxxxxxx>
> > > <mailto:xAP_developer-fullfeatured@xxxxxxx
> > <mailto:xAP_developer-fullfeatured@xxxxxxx>>
> > >
> > >
> > >
> > >
> > >
> > >
> >
> >
> >
> > ------------------------------------
> >
> > Yahoo! Groups Links
> >
> >
> > mailto:xAP_developer-fullfeatured@xxxxxxx
> > <mailto:xAP_developer-fullfeatured@xxxxxxx>
> >
> >
> >
> >
> >
> >
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>
>
>
>
>=20
>
--00148531b766446b4704739d9148
Content-Type: text/html; charset=windows-1252
Content-Transfer-Encoding: quoted-printable
<head>
<style type=3D"text/css">
<!--
/* start of attachment style */
.ygrp-photo-title{
clear: both;
font-size: smaller;
height: 15px;
overflow: hidden;
text-align: center;
width: 75px;
}
div.ygrp-photo{
background-position: center;
background-repeat: no-repeat;
background-color: white;
border: 1px solid black;
height: 62px;
width: 62px;
}
div.photo-title=20
a,
div.photo-title a:active,
div.photo-title a:hover,
div.photo-title a:visited {
text-decoration: none;=20
}
div.attach-table div.attach-row {
clear: both;
}
div.attach-table div.attach-row div {
float: left;
/* margin: 2px;*/
}
p {
clear: both;
padding: 15px 0 3px 0;
overflow: hidden;
}
div.ygrp-file {
width: 30px;
valign: middle;
}
div.attach-table div.attach-row div div a {
text-decoration: none;
}
div.attach-table div.attach-row div div span {
font-weight: normal;
}
div.ygrp-file-title {
font-weight: bold;
}
/* end of attachment style */
-->
</style>
</head>
<html><head>
<style type=3D"text/css">
<!--
#ygrp-mkp{
border: 1px solid #d8d8d8;
font-family: Arial;
margin: 14px 0px;
padding: 0px 14px;
}
#ygrp-mkp hr{
border: 1px solid #d8d8d8;
}
#ygrp-mkp #hd{
color: #628c2a;
font-size: 85%;
font-weight: bold;
line-height: 122%;
margin: 10px 0px;
}
#ygrp-mkp #ads{
margin-bottom: 10px;
}
#ygrp-mkp .ad{
padding: 0 0;
}
#ygrp-mkp .ad a{
color: #0000ff;
text-decoration: none;
}
-->
</style>
</head>
<body>
<!-- **begin egp html banner** -->
<br><br>
<!-- **end egp html banner** -->
Hi Edward,<br>Please see in-line
responses...<br><br><div class=3D"gmail_qu=
ote">2009/9/15 Edward Pearson <span
dir=3D"ltr"><<a href=3D"mailto:edwar=
d.mailgroup@xxxxxxx">edward.mailgroup@xxxxxxx</a>></sp=
an><br>
<blockquote class=3D"gmail_quote" style=3D"border-left:
1px solid rgb(204, =
204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
=20=20=20=20=20=20=20=20
<div bgcolor=3D"white" link=3D"blue"
vlink=3D"purple" lang=3D"EN-GB">
<br><br>
<div>
<p><span style=3D"font-size: 11pt; color: rgb(31, 73,
125);">The double 0x0=
a terminator works for me, it=92s simple to
implement =96 already done for my stuff.</span></p>
<p><span style=3D"font-size: 11pt; color: rgb(31, 73,
125);">=A0</span><br>=
</p></div></div></blockquote><div>...but it
isn't robust unless the mes=
sage is sent as a single datagram, and if it is sent as a single datagram,
=
the os should deliver it as such -- and if it doesn't, adding an
eom ma=
rker isn't going to help. <br>
</div><blockquote class=3D"gmail_quote"
style=3D"border-left: 1px solid rgb=
(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left:
1ex;"><div bgcolo=
r=3D"white" link=3D"blue" vlink=3D"purple"
lang=3D"EN-GB"><div>
<p><span style=3D"font-size: 11pt; color: rgb(31, 73,
125);">For reliable s=
treams, such as TCP, I generally frame the
messages with STX and ETX.</span></p>
<p><span style=3D"font-size: 11pt; color: rgb(31, 73,
125);">=A0</span></p>
<p><span style=3D"font-size: 11pt; color: rgb(31, 73,
125);">I thought all =
this business about 1500 bytes was somewhat urban
myth (at least as it applies to xAP). The data packet size is not what=92s
important; it=92s how the particular IP stack implementation deals with
>=
;datagrams<
that=92s the key.
=A0</span></p></div></div></blockquote><div><br>I
don'=
;t understand what you are syaing here? The data packet size is
inextricabl=
y linked to the IP stack. What aspect is it that you consider to be an
urba=
n myth?<br>
=A0</div><blockquote class=3D"gmail_quote"
style=3D"border-left: 1px solid =
rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left:
1ex;"><div bgc=
olor=3D"white" link=3D"blue" vlink=3D"purple"
lang=3D"EN-GB"><div><p><span =
style=3D"font-size: 11pt; color: rgb(31, 73, 125);">I only
have experience =
of the two most recent
Windows stacks (XP and Vista) which I agree are likely more capable than
ol=
d
embedded stacks. My experiments with Wireshark show that those stacks will
happily deal with fragmentation and defragmentation. 64KB in a single
datag=
ram?
No problem if your receive buffer is long enough (even if you force a small
MTU).</span></p></div></div></blockquote><div>Yes,
agreed, OSes perform fra=
gmentation for you. The individual fragments have a maximum size as
determi=
ned by the MTU. For pragmatic, practical reasons, xAP needs to define a
max=
imum overall message size, and for convenience's sake that was set
as e=
qual to the 'standard' MTU size. Devices which use a
smaller MTU <i=
>should </i>fragment and reassemble seamlessly provided that the
correct so=
cket options have been set to define the maximum UDP packet size. By
electi=
ng to use a single MTU's worth of data, we we avoid the overhead
associ=
ated with fragmentation and reassembly (principally memory buffering)
which=
is a good thing. When reassembled, the os should deliver a datagram as a c=
omplete entity in one go (irrespective of the mtu size, assuming that the
d=
atagram falls within the maximum datagram size defined for the stack). If
t=
he sender doesn't send a message as a single datagram, then the
whole t=
hing falls apart because effectively you are then doing fragmentation and
r=
eassembly at the application layer, and that won't work because the
ord=
ering across datagrams is not guaranteed and individual datagrams may get
l=
ost.<br>
=A0</div><blockquote class=3D"gmail_quote"
style=3D"border-left: 1px solid =
rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left:
1ex;"><div bgc=
olor=3D"white" link=3D"blue" vlink=3D"purple"
lang=3D"EN-GB"><div><p><span =
style=3D"font-size: 11pt; color: rgb(31, 73, 125);"> If
anything goes wrong=
(eg, missing fragment) the whole datagram is
dropped. I can=92t see how packet re-ordering can happen on a single LAN
fo=
r
broadcast UDP =96 there are no multiple paths and no retry mechanism.
Certainly never observed it =96 I assume the stack would again just drop
the entire
datagram.</span></p></div></div></blockquote><div><br>Re-orderin=
g of datagrams can occur for multiple reasons on a single lan. A sender
may=
send individual UDP packets in any order it chooses. This commonly occurs =
with fragmented packets originating from a multi-threaded sender, which
can=
be interleaved with smaller, non-fragmented datagrams as required (optimis=
ing transient memory use, as soon as they are sent, the buffer can be
relea=
sed). A switch is not obliged to retransmit packets in the order in which
t=
hey are received. And most fundamentally, the specs do not require UDP
pack=
ets to be ordered so, even if you don't observe it, it
<i>can</i> happe=
n, and sooner or later interoperability issues will arise if the working
as=
sumption is made that they always arrive in order.<br>
<br></div><blockquote class=3D"gmail_quote"
style=3D"border-left: 1px solid=
rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left:
1ex;"><div bg=
color=3D"white" link=3D"blue"
vlink=3D"purple" lang=3D"EN-GB"><div>
<p><span style=3D"font-size: 11pt; color: rgb(31, 73,
125);">=A0</span></p>
<p><span style=3D"font-size: 11pt; color: rgb(31, 73,
125);">A bigger issue=
for me, and the reason for me experimenting a few
times with TCP stream serving from hubs, is datagram loss over WiFi
network=
s.
This is greatly accentuated for UDP when you use broadcast (as we do) from
wired to wireless (fine the other way as it=92s not really a broadcast
packet till it gets to the AP) as the access point and NIC can not longer
a=
pply
their ack/nak procedure at the transport level. I commonly see xAP datagram
loss rates from wired to wireless connections of 20%. So I=92d like us to
agree on a standard transport wrapper for TCP streams which a lot of
platfo=
rms
would find useful.</span></p>
<p><span style=3D"font-size: 11pt; color: rgb(31, 73,
125);">=A0</span></p>=
</div></div></blockquote><div>I'd suggest
using the same framing as the=
"transport wrapper", as this then allows for code to be
shared a=
cross transports. If xAP was extended to support TCP, then that should
also=
include a formal discovery mechanism by which the IP address/characteristi=
cs of the hub can be discovered (over UDP xAP?)<br>
<br>Patrick<br>=A0</div><blockquote
class=3D"gmail_quote" style=3D"border-l=
eft: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex;
padding-left:=
1ex;"><div bgcolor=3D"white" link=3D"blue"
vlink=3D"purple" lang=3D"EN-GB"=
><div>
<div style=3D"border-style: none none none solid; border-color:
-moz-use-te=
xt-color -moz-use-text-color -moz-use-text-color blue; border-width:
medium=
medium medium 1.5pt; padding: 0cm 0cm 0cm 4pt;">
<div>
<div style=3D"border-style: solid none none; border-color: rgb(181,
196, 22=
3) -moz-use-text-color -moz-use-text-color; border-width: 1pt medium
medium=
; padding: 3pt 0cm 0cm;">
<p><b><span style=3D"font-size: 10pt;"
lang=3D"EN-US">From:</span></b><span=
style=3D"font-size: 10pt;" lang=3D"EN-US"> <a
href=3D"mailto:xAP_developer=
@yahoogroups.com"
target=3D"_blank">xAP_developer@xxxxxxx</a>
[mailto:<a href=3D"mailto:xAP_developer@xxxxxxx";
target=3D"_blank">=
xAP_developer@xxxxxxx</a>] <b>On Behalf Of </b>Patrick
Lidstone<br>
<b>Sent:</b> 14 September 2009 14:19<br>
<b>To:</b> <a href=3D"mailto:xAP_developer@xxxxxxx";
target=3D"_blan=
k">xAP_developer@xxxxxxx</a><br>
<b>Subject:</b> Re: [xAP_developer] xAP EOM identifier in xAP
v1.3</span></=
p>
</div>
</div><div><div></div><div
class=3D"h5">
<p>=A0</p>
<p>=A0 </p>
<div>
<div>
<div>
<p style=3D"margin-bottom: 12pt;">So the xAP delimiters for
serial are defi=
ned in
the 1.2 xAP spec here:<br>
<a href=3D"http://www.xapautomation.org/index.php?title=3DProtocol_definiti=
on#Transport_Wrapper" target=3D"_blank">http://www.xapautomation.org/index.=
php?title=3DProtocol_definition#Transport_Wrapper</a><br>
<br>
To the best of my knowledge, the MTU size for an ethernet frame is 1518
byt=
es,
which leads to a UDP packet MTU of 1500 bytes and this is the size that is
adopted by the majority of operating systems. Internet routers (i.e. ISPs)
sometimes use an MTU of 576 bytes, but this wouldn't be relevant to
xAP=
since
the traffic doesn't pass over the wider net, or if it does,
it's ge=
nerally
gatewayed via a TCP/IP connection.<br>
<br>
If a device is receiving fragmented UDP packets, I think the same issues
ar=
ise
as those related to extending the xAP message size beyond 1500 bytes - what
happens if a fragment gets discarded.<br>
<br>
If you take the scenario:<br>
<br>
Part 1(a), Part 1(b), Part 2(a), Part 2(b), Part 3(a), Part 3(b)<br>
<br>
--- first there is no guarantee that the parts will be delivered in order,
=
and
second, if part 1(b) was dropped, and you were blindly assembling messages
based on the proposed double-0a terminator, you'd end up with a
message
comprising part 1(a), 2(a) and 2(b) which is not only obviously corrupt,
bu=
t
also possibly larger than the maximum xAP message size, blowing away your
buffers.<br>
<br>
I think the solution is to probably parse incoming messages on the fly,
byte-by-byte. You can then at least reset your state when you encounter the
xap-header, and if you count open and close curly braces, you can tell
when=
you
have an apparently complete message. This won't solve the issue of
UDP
fragments being potentially received out of order, but so long as we are
dealing with a single LAN, and fragmentation occurs at the receiver, we
wil=
l be
ok I think.<br>
<br>
It is absolutely possible for UDP packets to be discarded, and the way we
d=
eal
with this in xAP is to accept that this can happen to xAP messages, and
lay=
er
application level acknowledgements where knowing that a message has been
received is critically important - whether explicitly or implicitly through
status messages. There are various schemes that could be adopted to allow a
receiver to detect lost messages (e.g. sequence numbering), but they
quickl=
y
become quite onerous, and assume that the originator keeps a copy of the
original message or is able to reconstruct it - which is non-trivial for
th=
e
many xAP nodes.<br>
<br>
Perhaps you can share more of the specific details of the device(s) in
ques=
tion
(manufacturer, docs, o/s etc), and their specific behaviour, which seems a
=
bit
anomalous?<br>
<br>
Patrick</p>
<div>
<p>2009/9/13 Kevin Hawkins <<a href=3D"mailto:yahoogroupskh@xxxxxxx=
om"
target=3D"_blank">yahoogroupskh@xxxxxxx</a>></p>
<p>One of the issues seems to be that there is conflicting
views as to the<br>
length of a a UDP data packet payload. =A0Some people cite 500 or
512<br>
characters and some 1500. =A0Regardless in some low
memory/performance<br>
devices it is being reported, that even with UDP, =A0packets are
being<br>
received from the buffer either truncated or appended back to
back.<br>
The latter I assume is due to the speed of the device in servicing
the<br>
buffer.<br>
<br>
We have an opportunity to protect against this with v1.3 and the
double<br>
'0A' seems the most compatible. =A0 I would be loathe to
support an=
ything<br>
that wasn't backward compatible.<br>
<br>
=A0 K</p>
<div>
<p><br>
Patrick Lidstone wrote:<br>
><br>
><br>
> I will dig it out - it included an optional checksum I think, and
IIRC=
<br>
> was framed by stx and etx (a kind of pseudo industry standard).
I<br>
> certainly used it with the PIC serial stuff and the xAP-serial
bridge.=
<br>
> Re.: long message truncation and concatenation: If we need to
support<=
br>
> messages that are larger than one UDP packet, then there
are<br>
> additional complexities and the proposed scheme won't work
as inte=
nded<br>
> as the ordering of UDP messages is not guaranteed. I'm
happy to he=
lp<br>
> refine a spec to support these capabilities, but it is moving
away<br>
> from the basic ethos of xAP somewhat, as devices would have to be
able=
<br>
> to buffer received messages, and that raises the bar
considerably.<br>
> Perhaps there is scope for co-existence of a long and standard
message=
<br>
> protocol though?<br>
><br>
> Patrick<br>
><br>
> 2009/9/13 Kevin Hawkins <<a href=3D"mailto:yahoogroupskh@googlemail=
.com"
target=3D"_blank">yahoogroupskh@xxxxxxx</a></p>
</div>
<p>> <mailto:<a href=3D"mailto:yahoogroupskh@xxxxxxx";
target=
=3D"_blank">yahoogroupskh@xxxxxxx</a>>></p>
<div>
<p>><br>
> =A0 =A0 ... Oh ... where is that in the spec ? =A0it might be all
we need.<br>
><br>
> =A0 =A0 =A0 =A0This is also tied in with some aspects of long
message truncation<br>
> =A0 =A0 and concatenation of messages received in UDP receive
buffers<br>
> =A0 =A0 though...<br>
><br>
> =A0 =A0 =A0K<br>
><br>
> =A0 =A0 Patrick Lidstone wrote:<br>
> =A0 =A0 ><br>
> =A0 =A0 ><br>
> =A0 =A0 > The original xap spec provides extensions for
framing a
message over<br>
> =A0 =A0 > async serial which also delimit the start of the
message - you don't<br>
> =A0 =A0 > need this 'hack' if you follow
the original spec =
for
non-UDP<br>
> =A0 =A0 transports.<br>
> =A0 =A0 ><br>
> =A0 =A0 > Patrick<br>
> =A0 =A0 ><br>
> =A0 =A0 > 2009/9/13 Kevin Hawkins <<a
href=3D"mailto:yahoogroups=
kh@xxxxxxx"
target=3D"_blank">yahoogroupskh@xxxxxxx</a><br>
> =A0 =A0 <mailto:<a href=3D"mailto:yahoogroupskh@xxxxxxx";
tar=
get=3D"_blank">yahoogroupskh@xxxxxxx</a>></p>
</div>
<p>> =A0 =A0 > <mailto:<a href=3D"mailto:yahoogroupskh@googlemail.=
com"
target=3D"_blank">yahoogroupskh@xxxxxxx</a></p>
<div>
<div>
<p style=3D"margin-bottom: 12pt;">> =A0 =A0
<mailto:<a href=3D"mailto=
:yahoogroupskh@xxxxxxx"
target=3D"_blank">yahoogroupskh@xxxxxxx=
om</a>>>><br>
> =A0 =A0 ><br>
> =A0 =A0 > =A0 =A0 =A0 We have been asked on several
occasions how to detect the<br>
> =A0 =A0 end of a<br>
> =A0 =A0 > =A0 =A0 xAP messager as there is no unique EOM
character. =A0Typically<br>
> =A0 =A0 in any<br>
> =A0 =A0 > =A0 =A0 reasonable sized packet structured
transport eg UDP then the<br>
> =A0 =A0 packet<br>
> =A0 =A0 > =A0 =A0 provides such an indication but on
systems with small packets or<br>
> =A0 =A0 > =A0 =A0 non eg<br>
> =A0 =A0 > =A0 =A0 asynchronous serial this is not
useable.<br>
> =A0 =A0 ><br>
> =A0 =A0 > =A0 =A0 =A0 In discussing this with the
specification team we must<br>
> =A0 =A0 consider<br>
> =A0 =A0 > =A0 =A0 backwards compatability and so we do not
wish to alter the<br>
> =A0 =A0 > =A0 =A0 specification<br>
> =A0 =A0 > =A0 =A0 to include a unique EOM character.
=A0What we do propose<br>
> =A0 =A0 however is that<br>
> =A0 =A0 > =A0 =A0 xAP v1.3 will specify that all messages
should end with two<br>
> =A0 =A0 > =A0 =A0 consecutive<br>
> =A0 =A0 > =A0 =A0 chr(10)'s immediately after the
closing
'}'<br>
> =A0 =A0 ><br>
> =A0 =A0 > =A0 =A0 ie =A0..... 7D 0A 0A<br>
> =A0 =A0 ><br>
> =A0 =A0 > =A0 =A0 =A0Some apps, even v1.2 ones,
=A0already do this. =A0We don't<br>
> =A0 =A0 believe this<br>
> =A0 =A0 > =A0 =A0 will cause any backwards compatibility
issues and it will<br>
> =A0 =A0 always be<br>
> =A0 =A0 > =A0 =A0 unique within a xAP message.<br>
> =A0 =A0 ><br>
> =A0 =A0 > =A0 =A0 =A0So, in developing xAP v1.3 apps
could you therefore append<br>
> =A0 =A0 =A0two 0A's<br>
> =A0 =A0 > =A0 =A0 at the end of your messages , and of
course handle incoming<br>
> =A0 =A0 messages<br>
> =A0 =A0 > =A0 =A0 containing such.<br>
> =A0 =A0 ><br>
> =A0 =A0 > =A0 =A0 =A0 K<br>
> =A0 =A0 ><br>
> =A0 =A0 ><br>
> =A0 =A0 > =A0 =A0
------------------------------------<br>
> =A0 =A0 ><br>
> =A0 =A0 > =A0 =A0 Yahoo! Groups Links<br>
> =A0 =A0 ><br>
> =A0 =A0 ><br>
> =A0 =A0 > =A0 =A0 =A0 =A0mailto:<a href=3D"mailto:xAP_developer-ful=
lfeatured@xxxxxxx"
target=3D"_blank">xAP_developer-fullfeatured@yah=
oogroups.com</a><br>
> =A0 =A0 <mailto:<a href=3D"mailto:xAP_developer-fullfeatured@yahoog=
roups.com"
target=3D"_blank">xAP_developer-fullfeatured@xxxxxxx</a>=
><br>
> =A0 =A0 > =A0 =A0 <mailto:<a href=3D"mailto:xAP_developer-fullfe=
atured@xxxxxxx"
target=3D"_blank">xAP_developer-fullfeatured@yahoog=
roups.com</a><br>
> =A0 =A0 <mailto:<a href=3D"mailto:xAP_developer-fullfeatured@yahoog=
roups.com"
target=3D"_blank">xAP_developer-fullfeatured@xxxxxxx</a>=
>><br>
> =A0 =A0 ><br>
> =A0 =A0 ><br>
> =A0 =A0 ><br>
> =A0 =A0 ><br>
> =A0 =A0 ><br>
> =A0 =A0 ><br>
><br>
><br>
><br>
> =A0 =A0 ------------------------------------<br>
><br>
> =A0 =A0 Yahoo! Groups Links<br>
><br>
><br>
> =A0 =A0 =A0 =A0mailto:<a href=3D"mailto:xAP_developer-fullfeatured@yah=
oogroups.com"
target=3D"_blank">xAP_developer-fullfeatured@xxxxxxx<=
/a><br>
> =A0 =A0 <mailto:<a href=3D"mailto:xAP_developer-fullfeatured@yahoog=
roups.com"
target=3D"_blank">xAP_developer-fullfeatured@xxxxxxx</a>=
><br>
><br>
><br>
><br>
><br>
><br>
><br>
<br>
<br>
<br>
------------------------------------<br>
<br>
xAP_Development Main Index |
xAP_Development Thread Index |
xAP_Development Home |
Archives Home
|