Implementation Notes

Notes on suggestions for

• Caching - see https://github.com/hapi-server/cache-tools
• Durations, cadence, and timeStampLocation
• Serving public and private datasets

Durations

The HAPI specification allows metadata for cadence and timeStampLocation for a dataset that applies to all parameters in the dataset. The cadence is to be interpreted as the nominal time difference between records.

Consider a parameter composed of measurements in 10 channels with time stamps typically every hour:

1999-01-01T01Z,ch1,ch2,ch3,...,ch10
1999-01-01T02Z,ch1,ch2,ch3,...,ch10
1999-01-01T03Z,ch1,ch2,ch3,...,ch10
1999-01-01T05Z,ch1,ch2,ch3,...,ch10

Suppose these data are to be represented as a grid of colored tiles with the color indicating the value. There is an ambiguity in how wide the tiles should be in the time direction (in the channel direction, the ambiguity can be resolved by specifying bin ranges). If the cadence of the records returned is uniform or given in the metadata, a typical guess is that the time width of the tiles is the cadence. However, there are circumstances in which this is not correct - for example, the hourly measurements could have been made by an average over +/- 10 minutes. Or, a decrease in the cadence could be a result of the instrument requiring more time to record an accumulation - for example, the "missing" value at hour 04 in the above was not caused by a bad record.

To resolve this ambiguity, we recommend the use of a x_duration metadata keyword. Metadata key names prefixed by x_ are not a part of the standard. If it is found that this is an often used key, it will be added to the HAPI specification. This key can be used at the parameter or dataset level. If it appears at the dataset level, it applies to all parameters. If it appears at the parameter level, it applies to all parameters and the dataset-level value is ignored.

Note that if duration is time-varying, the same method for specifying time-varying bins can be used (i.e., the x_duration key refers to the name of a parameter that provides the time-varying duration values).

{
    ...
    "timeStampLocation": "center",
    "cadence": "PT1H",
    "x_duration": "PT1S",
    "parameters": [
        ...
        {
            "name": "Npart1", // duration is PT1S
            "description": "number of particles measured in channel 1",
            "type": "double"
        }
        {
            "name": "Npart2", // duration is PT10M
            "description": "number of particles measured in channel 2",
            "type": "double",
            "x_duration": "PT10M",
        }
}