Binary Data Storage and Logging

Logging is primarily done behind the scenes, by using DataReader and DataWriter objects within a component.

Both data reads and data writes can be logged, although in the current implementation, only data writes are logged. During a computation cycle, data writes are buffered (as LogEntry instances) in a LogQueue. At the end of the cycle, all the accumulated LogEntry instances are serialized to the log and are then released. There can be zero or more LogWriter instances for any LogEntry – a set of LogRule instances can be used to define what LogWriter is used for each LogEntry.

There are three "flavors" of LogEntry: EventEntry, DataEntry, and SyslogEntry

EventEntry

An EventEntry is serialized as:

• a 16-bit header (serialized using OutStream::writeUShortCompact) including:
  • 12 high bits: Actor code (which can be used to look up what code unit is causing the event), or zero if this is the start of a cycle
  • 2 bits: EventEntry::Action code, indicating the type of action being logged; e.g., the start of a cycle, the setting of accuracy, the setting of the state of a variable, or the setting of a variable to the "best" for its associated Universal variable
  • 2 bits: the value 0, indicating this is a EventEntry
• Unless the action is EventEntry::START_CYCLE, 16 bits (serialized using OutStream::writeUShortCompact) indicating the DataAccessor code of the DataElement undergoing the event.
• If the action is EventEntry::START_CYCLE, a 64-bit Timestamp as integer microseconds since the start of the epoch. Otherwise, a 64-bit Timespan as integer microseconds since the previous LogEntry. (In both cases, serialized using OutStream::writeLongLongCompact).
• If the action is EventEntry::SET_DATA_ACCURACY, a 16-bit floating point number (serialized using OutStream::writeFloat2) indicating the new accuracy of the DataElement. Or if the action is EventEntry::SET_DATA_STATE, a 8-bit ORed result of the EventEntry::StateChange flags for the DataElement.

DataEntry

A DataEntry is serialized as:

• a 16-bit header (serialized using OutStream::writeUShortCompact) including:
  • 13 high bits: DataAccessor code (which can be used to look up metadata about the entry that follows)
  • 1 bit: "Best" flag, set if this is a member of a "Universal" set of variables, and this one is the "best" one (see UniversalDataElement)
  • 2 bits: the value 1, indicating this is a DataEntry
• 64-bit Timespan as integer microseconds since the previous LogEntry, serialized using OutStream::writeLongLongCompact.
• the data, length determined by the DataAccessor

SyslogEntry

A SyslogEntry is serialized as:

• a 16-bit header (serialized using OutStream::writeUShortCompact) including:
  • 12 high bits: Actor code (which can be used to look up what code unit is writing to the log)
  • 2 bits: Syslog::Severity code, indicating the severity of the message being logged: from debug messages to messages that must be relayed to shore via satellite.
  • 2 bits: the value 2, indicating this is a SyslogEntry
• 64-bit Timespan as integer microseconds since the previous LogEntry, serialized using OutStream::writeLongLongCompact.
• 16-bit integer length of the character sequence that follows, serialized using OutStream::writeUShortCompact
• the characters of the message, with no termination character.