Custom Client/Data Export Interface Description

This section details the interface for writing a custom data retrieval client that will get data from the Data Export application. The programming concepts presented assume a familiarity with programming software applications to connect with a TCP/IP socket.

The Data Export application functions as a server providing a TCP/IP socket connection for one remote client. Once the Data Export has connected to the LoggerNet communication server the TCP/IP socket is established and the Data Export application starts “listening” for a client to attach. As soon as a data retrieval client connection is detected the Data Export application sends the first record out over the socket connection. Upon receiving the record the data retrieval client needs to send back an acknowledgment message consisting of the datalogger name, table name and the record number of the received record.

If the Data Export application loses its connection with the LoggerNet communication server, it will need to be re-connected before any records can be obtained and sent out.

There are two record formats used to send the record data:

• RTMS format – this format is provided for backward compatibility for customers who had the RTMS system and developed data handling procedures that use the format provided by the RTMS socket export.

• Standard format – this format provides an easily interpreted data string containing the data along with format information for each data field.

Details on these formats are provided at the end of this section.

The following illustrations show the state diagrams for the custom client/Data Export interface. (The diagramming notation is by Booch who claims to have adopted it from Harel).

Key concepts from the state diagrams are shown in the following tables with key words from the diagrams. In these definitions the “server” refers to the Data Export data server and the “client” is the custom data retrieval client application.

Client State Diagram:

Key Word	Description
Test For Server Rdy	With Socket APIs, usually there will be a function used to open the socket. In this state, the client program should attempt to open the socket. If Open Failed, the client should wait 5 seconds and try again.
Wait For Record	In this state the client is waiting for the next data record from the server. When a record is received it should be “secured” (saved to disk or database), then an acknowledgment should be sent back to the server. Once the server has processed the acknowledgment it will not send that record again. The client should use a watchdog timer while waiting for a data record. If the client is in the Wait For Record state for longer than expected (RecIntv2) then it should assume that the server has died and close the session. This watchdog operation may be difficult to implement, but it seems that some implementations of sockets do not properly report a broken socket and so the watchdog is necessary for reliability.
Rec Intv 2	This is an amount of time greater than 2 times the expected interval between data records. It is just longer than the longest period between records the client would expect to receive from the server. If the client goes longer than this interval without receiving a new record then it should close and reopen the socket, thus allowing the server to recover if it has broken socket connection.
Secure Rec	The secure record action is taken when a Record Rdy event occurs while the client is in the Wait For Record state. Before the client sends an acknowledgment to the server it should “secure” the data record sufficiently so that if a power failure or crash occurs the data will be safe.
Send Ack	The send acknowledgment action is done in response to the Record Rdy event, after the record is secured. In Send Ack the client forms an acknowledgment record from information taken from the data record and sends it to the server.
Stop	It is important to note that the Stop event could occur at any time. If it occurs while in the Session Open state then the socket should be closed (Close Session) before program termination.

Server State Diagram:

Key Word	Description
Wait For Record Available	In this state the server is waiting for the next record to become available from the server’s data record source.
Wait For Ack	In this state the server is waiting for the client to acknowledge that it has secured the record. If an acknowledgment for the wrong record comes in, the server will just continue to wait. After waiting for a minute, the server will re-issue the data record and wait again.
Advance Rec	The advance record action is executed after the server receives a valid acknowledgment record from the client while in the Wait For Ack Record state. This is the point at which the server recognizes that the client has secured a record and the server relinquishes responsibility for the well being of that record. The server moves on to the next record.
Stop	Note that in the Session Open and Server Registered states there are “exit” actions that need to be executed on the Stop event.

Communications between the client and server are conducted using ASCII records where each record is terminated by a carriage return – line feed (CRLF) pair. Record length varies quite a bit. For each datalogger record there is exactly one ASCII record. Because of the Block Mode Protocol used to communicate with dataloggers, the maximum size datalogger record is limited to something less than 1024 field values. Assuming 6 characters per value, 13 characters per field name, and 6 characters per field type designation, a single ASCII record could come out to be a little longer than 25K characters.

Typical datalogger programming will produce record sizes of about 150 characters. It would not be unusual to see records that contain one or two hundred values which would come out to a length of 2 to 3K characters in ASCII.

To express the format of ASCII records used for communications between the client and server, we will use Extended Backus Naur – Formalism (EBNF), a notation used to express syntax. This notation was adopted from Wirth [3], and extended here by adding a repetition count preceding some brackets. EBNF is summarized in the following table where A, B and C are syntactic entities of the language being described. Where one of these entities is a literal string it is enclosed in quotes.

Expression Means

A = BC	The construct A consists of B followed by C.
A = B | C	A consists of B or C.
A = [B]	A consists of B or nothing.
A = {B}	A consists of any number of B’s including none.
( )	Brackets used to group sections of an expression.