Standalone xPL Hub Summary, Mark II

[Gerry Duprey <gerry@xxxxxxxxxxx>]

Howdy,

Below is an updated version of the xPL Hub summary, with changes to
represent the dialog of the last few days.  The majority of the changes are
near the end, covering use of hbeat echo for detection of the hub.

I've got to say, I'm feeling really good about how this is progressing.
We've hashed out a lot of details and managed to find a solution that is
(in
my opinion) elegant, simple and can work with any platform/framework.  It's
event compatible with existing xPL apps to boot!


Clarifying the role and location of xPL Hubs
============================================

Overview
--------
For network-centric xPL apps (that is, nearly any xPL program that runs on
a
PC), xPL requires a hub to be running.  A hub acts as a single contact
point
for the network and handles dispatching messages to other xPL programs on a
computer.

Every computer running xPL apps must have a hub on it.  If there is no hub
or the hub is not running, xPL based applications cannot start.

A hub must be a separate program/task from any xPL based application.  An
xPL application, either directly or via it's framework, should not be
responsible for nor try to start a hub if none is detected.  If the app
were
to take on responsibility for starting/monitoring a hub, the hub becomes
susceptible to that applications lifecycle and health.  Given other xPL
apps
are going to also depend on the hub, such embedding would put xPL on that
computer in jeopardy should anything happen to the app (or apps) that are
also trying to manage the hub.

To quote Paul Robinson, "Keep the hub simple and separate".

Hub Author Notes
----------------
Details about the technical aspects of how the hub itself should work can
be
found here <insert link to technical description of a hub>

Hubs should be designed to be as simple and stable as possible.  Failure of
the hub will result in the all xPL apps running on a computer to be unable
to hear each other or the rest of the network,  As such the hub program
needs to be written in as "safe" a manor as possible (do all
possible
exception checking, make sure you are using the right APIs in the right way
and that there are no shortcuts, insure code is written with no assumptions
and as defensively as possible, etc).

Even the best written programs can (and usually do) have bugs, it's
important that the hub has some separate supervisory process that insures
the hub is running and when it detects the hub has stopped, restarts the
hub
immediately (optionally logging the problem so that eventually, the cause
of
the failure can be diagnosed).

Ideally, the supervisory role would be played by a standard system
mechanism.  For example, under windows, the hub could be installed as a
service and be set such that the windows service manager will automatically
restart the hub if it dies.  Under linux, this would have to be done by an
additional program that launched the hub and then watches it or instead of
a
program, with scripts.  Similar solutions exist in other operating systems.

When a supervisory program needs to be used (for systems that do not
provide
their own), it's important to keep that program as short and simple as
possible.  For something like unix-y based systems, this could be a simple
program that just forks off the hub, then waits for the forked process to
die and forks again (probably 10 lines of C code).

Between a short, simple, stable supervisor and a responsibly written hub,
there should be sufficient redundancy and failure traps for all but the
most
disastrous cases.

Hub Deployment
--------------
Hubs must not be deployed as part of an xPL application.  If multiple
applications are installed on a computer and each one installs a hub, chaos
could reign.  And since there are multiple xPL application frameworks,
there
is no good way for them to all agree on a "check and if missing,
install a
hub" method.

The user of the xPL application should be made aware, in notes about the
application and even on the download page for the application, that an xPL
hub is required and if they don't have one, they should be provided a link
to a webpage listing hubs for different platforms (maintained on the xPL
website).

Separate Hub impact on xPL frameworks and applications
------------------------------------------------------
xPL applications and frameworks must not provide any sort of hub service,
at
least not automatically. There can, of course, be code for a hub in a
framework, but it should not be run automatically as any part of normal xPL
application startup without some special/explicit setting to do so.

An xPL application should not come bundled with hub  -- either as part of
the application or as a bundled, separate hub install.  Hubs can evolve and
the user should be directed to a webpage of hubs instead of packaging what
may become an obsolete program.  Additionally, this helps to keep this
whole
issue of separate hubs clear in the mind of the user (one consistent
message).

Application Startup and xPL Hub interaction
------------------------------------
In order for an xPL application to have meaningful use of the xPL protocol
on a computer, there must be a hub running.  If there is none, the
application may be able to send xPL messages, but will not be able to
receive them.  Additionally, when a hub is running, the hub has to hear
from
an application sending messages before it knows the application exists and
registers it to receive messages.

Keep in mind that depending on how applications are started, it is possible
the xPL application will start before the hub is up and running.  Until the
hub is noticed, the application really cannot do a lot, so determining that
the hub is active and working is a critical detail.

Here is an outline of how an xPL based application (or more likely, the
underlying framework) should handle hub detection at startup:

Upon startup, the xPL application (or more likely, it's framework) needs to
start sending standard xPL heartbeat messages.  These messages can be of
the
schema class "hbeat" or "config" -- whichever is
appropriate to the xPL device.

The app should send out the heartbeat message every 3 seconds.  This is
distinctly different from the normal hbeat (usually every 5 minutes or so)
and config (usually every minute) sending rate.  The idea with this rate is
that we are looking to notice a hub being active as quickly as possible to
speed app startup.

The "every 3 second" sending rate terminates on one of two
conditions:

1) The application receives one of it's own hbeat messages.  This is only
possible when the hub sees the message (which only happens if the hub is
running and listening), adds the application to it's list of clients and
echoes the message back.  It's important that you check the remote-ip and
port parameter to confirm the message really is from yourself.  It's
possible, though uncommon, that in the early stages of startup, there may
be
more than one device/app with the same source identifier
(pre-configuration,
for example).

Once the echo is heard, the application is ready to go and the heartbeat
rate can go back to normal (every minute for config, every few minutes for
hbeat).

2) A 2 minute timers expires without receiving an echo.  At this point, the
hub is still not ready and may never be.  Because hope springs eternal,
hbeat/config messages are still sent out at an accelerated rate, but after
that first 2 minutes, the rate should drop to once every 30 seconds and
remain there indefinitely until the hub finally answers or the application
is terminated.  If possible, it would be wise to alert the user (if
appropriate) that there may be a problem after 1 or 2 minutes with no
response so they can be aware and take action (like starting the hub). If
appropriate/possible, it may also be wise to log a message to the system
event log and/or console as well (for future diagnostics).
------------------------


Gerry
--
Gerry Duprey
Ann Arbor, MI 48103
http://www.cdp1802.org