Implementing a client application

Implementing a Client Application

This page describes the ICAR ADE standard from a client perspective; what do you need to know to use the API's as described by the standard. You don't need to be a member of ICAR in order to use the standards (although there are many benefits to membership).

For more general information on ICAR and this standard, see About ICAR and the Animal Data Exchange Working Group.

Introduction

The standard consists of JSON-schema definitions defining the messages and an accompanying (recommended) URL scheme. The messages can be transported via many channels and thus can be used by themselves. We also provide a recommended URL scheme which allows REST-style access to these messages.

Most developers should be familiar with REST and JSON files. To simply view the standard, you can open the exampleUrlScheme.json file in your favorite OpenAPI Specification editor. Or, for convenience, use the following URL to have a preview in the free Redoc tool: view exampleUrlScheme.json

Backward and forward compatibility

When consuming the data, use a tolerant-reader pattern. If there are any fields in the message that you do not recognise, simply ignore them. This allows the standard to add new fields while maintaining backwards compatibility.

Synchronization with a source

There are use cases where a client has a requirement to know for certain that all events of a source have been retrieved. For example, for a milk recording agency it is essential to make sure all milking events have been recorded. This synchronization between a client and a source is not trivial and there are many ways to solve it. The ADE standard prescribes the following pattern to allow for synchronization.

Each event message has modified and source fields. The modified field is a datetime indicating the latest change to this messages record. The source field indicates the system of record for the message These can be used for synchronization. All data sources are required to fill these fields. This field is guaranteed to be monotonically increasing, e.g. it never goes back in time. This holds for each event related to the same source. A client thus may keep track of the latest received modified datetime per source and query all events from that time on. This guarantees that all changes will be retrieved.

There is a scenario that we do not cover in this specification: in case a device with a local clock has drifted too much, an operator may decide to reset its clock either forward or backward. If it is reset backward in time, clients may miss data recorded in that correction period. In case of such a hard reset, clients should be prepared to recapture a larger period in time. How to detect this is out of the scope of the spec and assumed a manual proces, similar to reparations needed with hardware failure or other kinds of data loss. In this scenario, the client should be able to rely on the ‘eventId’ being unique for the ’source’.

The url scheme should thus allow to query for ‘modified >= x and source = y’. Alternatively, a client could query with ‘modified in last z hours and source = y’. The latter does not require the client to keep the modified date but has to take into account a possible drift between the clock of the source and the clock of the client.

The WG also discussed the possibility of allowing for a tolerance period (scenario 1b). No default could be found that is both reasonably efficient as well as allowing all vendors to guarantee it, the perceived benefit of defining a period is low.

The WG also discussed future extensions: if messages (or rather, devices) are defined for which relying on a datetime ('modified') cannot be implemented, the standard could define a tolerance period for those messages (e.g. it cannot go back in time more than 24hrs), or the standard could be extended with specific synchronization fields (for example, a specialized Sync group with sourceID and syncOffset allowing other synchronization methods). This will be considered only when these messages present itself with a use case.