MMC-SIAM Users Guide
May 27, 2004
Kent Headley, Bob Herlien, Tom
O'Reilly, Michael Risi, Karen
Salamy
Logging onto the MMC nodeRunning the config script
Resetting and rebooting the MMC node
The siamPort.cfg fileConfiguring instrument pucks
The nodeID property
Port properties
Rarely modified properties
Starting the node application
Listing instruments on the node
Removing an instrument from the node
Installing a "pucked" instrument onto the node
Installing a "non-pucked" instrument onto the node
Shutting down the node application
Primary wireless linkSIAM power management
Secondary wireless link
Keeping the cpu awakeFrequently used SIAM command summary
Obtaining a lease for the wireless link
Introduction
The
MMC-SIAM Users Guide describes how to configure, use, and trouble-shoot a
MOOS mooring controller (MMC) running SIAM software, in the lab and while
deployed at sea. This document is intended primarily for the operations user,
and so mostly describes "how to" rather than "why". Hardware and software
architecture are described elsewhere; links to these documents may be found at
the SIAM
website
A MOOS mooring consists of one or more MMCs or "nodes"; a node on the surface buoy, and optionally, additional nodes in the water column and on the seafloor. Each node may host multiple science instruments through the node’s serial ports. Most of the instrument ports on a node utilize a dual port adapter (DPA); each DPA hosts two serial ports and power channels, and provides overcurrent protection and isolation capability.
The surface node is equipped with solar panels, GPS, and RF transceivers for communications with shore. Optionally, one or more MMC benthic instrument nodes or BINs may be connected to the surface node via an EOM cable; in this case the surface node and BINs form a moored network. The EOM cable provides power and data pathways between surface and benthic nodes. Commands to instruments and other devices, as well as instrument data flow through this network.
A portal machine residing on shore is periodically linked to the moored observatory’s surface node via RF link, and provides a "gateway" between the moored network and other shore-based components, such as the Shore Side Data System (SSDS).
SIAM provides the software infrastructure and applications for MOOS; SIAM components run on the deployed nodes as well as on shore.
SIAM enables "self-identifying" instruments that are automatically recognized and processed when plugged into the observing system at sea. Software and other information needed by the system to operate the instrument and process its data can actually be stored with the instrument itself in a small attached device known as an MBARI "instrument puck". The observing system retrieves and utilizes the instrument information when the instrument and its puck are plugged in. Some of this information uniquely identifies the instrument serial number, its calibration coefficients, and other characteristics of the science data; SIAM automatically associates this metadata with the instrument data stream which is eventually processed on shore. This "plug-and-work" functionality provided by pucks and SIAM is very powerful, enabling automatic integration of instruments into the observing system, even when the system has no a priori knowledge of the instrument, its protocols, or data characteristics.
User accounts and
environment
Each MMC node
has two user accounts: ops and root. The ops login script
automatically sets up PATH and other environment variables for SIAM. Most
operations described in this manual can be executed as ops. Some
operations require root privileges; for these, you should first login as
ops, in order to set the SIAM environment, then invoke the su
command to become root. Default shell for both accounts is bash.
Several environment variables are defined for the ops account,
including:
SIAM_HOME – Specifies the top directory of the SIAM software package, set to /mnt/hda (on the compact flash). Do not modify.
LOG4J_THRESHOLD – specifies the "verbosity" of debugging, error, and informational messages written to stdout by SIAM applications. You can set this with the export command to "DEBUG", "INFO", "WARN", or "ERROR", where "DEBUG" is most verbose and "ERROR" is least verbose.
SIAM_AUTOSTART – If true, then the SIAM node application is started automatically when the MMC is rebooted or reset. This variable is usually set automatically by the config script (see below).
SIAM_PORTAL – specifies the IP address of the shore "portal" machine. This variable is usually set automatically by the config script (see below).
You can log into the node over the network from any host which has access to the node’s ppp or ethernet interface, by invoking the ssh command from that host:Running the config script% ssh mooring
The argument to ssh, in this case "mooring", must resolve to the node’s ppp or ethernet interface. Obviously the specified interface must "enabled" in order to connect via ssh. You can determine which interfaces are up by executing /sbin/ifconfig on the node itself (e.g. if you are logged into the console). Note that the node’s ppp interface can generally be accessed only from the portal host.
To run config, first log into the
MMC as ‘ops’, then run the su command to get root permissions. Then
simply type config; the script will prompt for the options listed above.
After all options have been entered by the user, the script prints out a review
of user inputs, and gives the option to continue or abort the script. If the
user elects to continue, config then automatically modifies the relevant
Linux and SIAM scripts. Note that the MMC must be rebooted before the changes
take effect.
Resetting and
rebooting the MMC node
The
MMC can be restarted either via the reboot command, or by a hardware
reset.
NOTE: You should terminate the SIAM node application if
possible before rebooting or resetting the MMC.
NOTE: It is very important
that you issue the sync command immediately prior to rebooting or resetting the
MMC
Rebooting the node
As user root, type the following commands:
sync; rebootThe system will synchronize the file system and then reboot; after the reboot process completes, the command prompt will appear on the console.
Resetting the node
As user ‘root’, type the following commands:sync; halt
When the ‘halt’ process completes, the message "Power down" appears on the console; only then is it safe to reset the node. You can then reset either by pressing the reset button on the Sidearm board, or by calling the reset pager phone number.The SIAM node applicationTo access the reset button, the MMC can must be open and the Sidearm board must be accessible.
After dialing the reset pager number, enter any digits (e.g. "111") when prompted by the beep tone. Hang up when you hear the "thank you" response. The onboard reset pager should be notified within a few minutes. NOTE: occasional pager delays of many minutes, perhaps even an hour or more, have been observed.
platformSerialPorts =
/dev/ttySX0:\
/dev/ttySX1:\
/dev/ttySX2
A "#" character at the beginning of a line denotes a comment.
NOTE: in-line comments on the same line as the name=value pair are not allowed. E.g. the following is an error:
nodeID = 4912 # Comment placed here is not allowed!
NOTE: the
properties are read from siamPort.cfg only when node is started. Any
user edits to siamPort.cfg will not take effect until the node
application is restarted.
nodeID = 4912
Port
properties
The
operations user might need to modify these properties if a port is to be
"preconfigured" with a particular instrument service, instead of using a
puck.
Instrument ports and
the source of the instrument service on each port are specified with the
serialPort, powerPort, and serviceJar properties. Ports
are numbered consecutively, starting at 0. For each port, you must specify
three properties as follows:
serialPortn =
serialName
powerPortn = powerName
serviceJarn =
jarName
where n is the port number.
serialName is the name of the
serial device, including the leading Linux /dev directory, e.g.
"/dev/ttySX0".
powerName must be either "DPA" or left blank; "DPA" indicates
that the port is associated with a dual port adapter; blank means that the
serial port is not associated with a DPA.
Finally, jarName is "PUCK", the name of a Java jar file in $SIAM_HOME/ports, or "EMPTY". A value of "PUCK" indicates that the serial port should be "scanned" for a puck. If jarName is the name of a jar file in $SIAM_HOME/ports, it instructs SIAM to load the instrument service from that jar file; this allows you to bypass the puck mechanism and "preconfigure" the service. "EMPTY" means there is definitely no instrument plugged in, and SIAM does not attempt to scan for a puck. Note that "EMPTY" is never strictly necessary; if a port is empty and SIAM tries to scan for a puck, a message indicating "no puck present" will be written to the stdout. Likewise, if a jar file in the ports directory cannot be loaded for some reason, an error message is generated and SIAM continues with the next port. "EMPTY" simply skips the port, thus saving several seconds.
Following are examples of these
properties:
serialPort1 =
/dev/ttySX1
powerPort1 = DPA
serviceJar1 = EMPTY
serialPort2 =
/dev/ttySX2
powerPort2 = DPA
serviceJar2 = PUCK
serialPort3 =
/dev/ttySX3
powerPort3 = DPA
serviceJar3 = PowerCan-1322.jar
NOTE: These
properties must be specified with consecutively increasing port numbers; if
a number in the sequence is missing, SIAM will stop searching for
ports.
By convention, /dev/ttySX15 is reserved
for the MSP430 environmental processor, which has no associated
DPA:
Rarely modified properties
The following properties are rarely if ever
modified by the operations user.
The platformSerialPorts property is a colon-separated list of all of the available serial ports on the platform. SIAM will search the system for serial ports if this entry is left blank or not present; however, SIAM may not find them all. If this entry is set SIAM will only know about these serial ports and will not search for others. For sideARM systems the entry should be set as follows:
platformSerialPorts = /dev/ttySX0:/dev/ttySX1:/dev/ttySX2:/dev/ttySX3:/dev/ttySX4:/dev/ttySX5:/dev/ttySX6:/dev/ttySX7:/dev/ttySX8:/dev/ttySX9:/dev/ttySX10:/dev/ttySX11:/dev/ttySX15:/dev/ttySA2
The following properties specify the directory paths for log files, instrument service jar files, and the node’s codebase; these properties should not be modified:
# Instrument service jar files go
here (path relative to $SIAM_HOME)
jarLocation = ports
# Instrument logs go here (path
relative to $SIAM_HOME)
logLocation = logs
# Codebase location (absolute
path)
codebaseLocation = /mnt/hda/codebase
The following properties control the behavior of sleep manager, which, if enabled, puts the Sidearm processor into low-power "sleep" mode when no tasks need to be executed, and causes the processor to "wake up" as tasks become imminent.
# enabled: true to enable, false
means you'll never go to sleep
SleepManager.enabled = true
# sleepString: command to invoke
to put us to sleep
SleepManager.sleepString = /root/suspend.sh
# pollSeconds: How often we poll
to see if we can go to sleep
SleepManager.pollSeconds = 2
# sleepDelay: How long to delay
before going to sleep, to allow logs
# to get out to the flash file system. This
one is in milliseconds.
SleepManager.sleepDelay = 0
# wakeupSeconds: How many seconds
we wake up early, to allow things to
# stabilize before sampling.
SleepManager.wakeupSeconds =
3
# minSleepSeconds: Minimum time
we'll go to sleep. If less than this,
# we'll just stay awake.
SleepManager.minSleepSeconds =
5
CommsSchedulerTask is responsible for ensuring that a comms link is up for certain number of minutes per hour. Every <NodeService.leaseRenewalInterval> milliseconds, CommsSchedulerTask establishes a lease (which ensures that comms are alive) for <NodeService.leaseInterval> milliseconds. CommsManager.enabled enables CommsLeaseListener. This should normally remain true. If false, not only will CommsLeaseListener never set up the comms link, but it will also never notify the portal side that the link is up.
CommsManager.onString and CommsManager.offString are set to the names of Linux executables that set up and tear down the comms link, respectively. If these properties are not defined, CommsLeaseListener does not try to set up the link, but will directly send the portal notifications. These properties very useful for testing, e.g. on ethernet. Just comment out the properties, and it will work on ethernet or any other permanently connected device.
Conversely, CommsManager.protocolWaitTime is the wait time before the link is torn down. That is, when the last lease either expires or is terminated CommsLeaseListener will send a "disconnecting" message, then wait for protocolWaitTime, and then tear down the link by invoking the program specified by CommsManager.offString
CommsManager.processWaitTime is the time, in milliseconds, that we'll wait for the onString or offString scripts to complete, before we timeout and kill the respective processes. If this property is missing or non-positive, we wait forever.
Following are example settings for
these properties:
# For deployment
NodeService.leaseRenewalInterval = 600000
# Added by config script: user
specified link 'freewave'
CommsManager.onString = pon longhaul
CommsManager.offString = commsOff
longhaul
CommsManager.protocolWaitTime = 5000
CommsManager.processWaitTime =
60000
% node shore
The argument "shore" refers to the name of an IP interface on the portal’s host machine, and must correspond to an entry in the MMC's /etc/hosts file. The portal is described in more detail below.
NOTE: Only one instance of node may be executed on the node at any given time. Before starting node manually, insure that it is not already running, using the ps command to search for any instances of the j9 Java virtual machine:
ps –eaf | grep j9
If the above command results in multiple output lines of the form
ops 2039 2037 3 21:12 /mnt/hda/siam/ 00:00:08 j9 - mca:32 -mco:64 -mx
then the application is likely already running, and you must use invoke exitNodebefore restarting.
Listing instruments on
the node
The listPorts
command displays a list of currently running SIAM instrument services
associated with each instrument port on the node.
Removing an instrument
from the node
An instrument
can be removed from the MMC while the node application is running.
First, execute the shutdownPort
utility, which causes the instrument service on the specified port
to shut down "gracefully", and removes power from the port’s DPA channel. When
shutdownPort has completed, it is safe to physically unplug the
instrument from the port. NOTE:
shutdownPort may in some cases take several minutes to complete. If
an instrument is sampling, at the time shutdownPort is invoked, shutdownPort
will block until the sampling is complete.
Installing a "pucked" instrument onto the node
A pucked instrument can be plugged into the
MMC and its service started while the node application is running, if
the port has been configured for a puck in the siamPort.cfg properties file. First, physically plug the pucked
instrument into the port. Then execute the scanPort
utility, which causes SIAM to search for and communicate with the puck on the
specified port, retrieve the instrument service jar file from the puck, and
execute the service code. NOTE: In some cases,
scanPort may take several minutes to complete, depending on the time
required to initialize the particular instrument.
Installing a
"non-pucked" instrument onto the node
A non-pucked instrument can be plugged into an MMC
port and its service started while the node application is running, if
the appropriate jar file has been specified beforehand in the
siamPort.cfg properties file. Suppose that you want to
pre-configure /ttySX3 for a Seabird CTD. Suppose the jar file for this service
is called "Seabird-1289.jar"; this service must be placed in the
$SIAM_HOME/ports directory. The entry in siamPort.cfg then must
be modified as follows:
This output of ps will include multiple
lines of the form:
ops 2039 2037 3 21:12 /mnt/hda/siam/ 00:00:08 j9 - mca:32
Each line displays the pid in the second
field, e.g. 2039 in the case above. Find the "j9" line which has the lowest
pid value, then issue the kill command specifying that pid,
e.g.:
Then issue ps –eaf again and verify that all
j9 processes have been killed. Remember to specify the j9 process with the lowest pid to the
kill command, which will ensure that all j9 instances are
terminated.
NOTE: kill
does not "gracefully" terminate instrument services, and may leave
instruments in a powered state. Instead, use exitNode where
possible.
org ID : 1The ‘service name’ identifies the instrument type and the ‘ISI ID’ identifies the instrument uniquely and can be used in conjunction with the shore side data system to access the archived data from this instrument.
type ID : 1
unique ID : bf0a2eb9-a5ec-11d8-8a78-000c6a000402
payload size : 33204
checksum : 0x91ecc72d
service name : Mooring power can
ISI ID : 1322
current limit : -1
NOTE: power must be removed from pucks for no less then 5 seconds before reapplying power. Failure to remove power from a puck for a period of less than 5 seconds may result in an unpredictable state on power up.
The
shore portal
The SIAM portal
server application runs on a shore-based machine, and periodically retrieves
data from one or more MMC nodes, using the network provided by the RF link;
portal logs the retrieved data locally on its host and also optionally
"publishes" data to the Shore Side Data System (SSDS). See the
portal manual page for details.
System
clocks
The MOOS mooring
controller, running Linux, has 3 concepts of time:
Whenever the controller returns from sleep, the Ricoh RTC is used to set system time. This is used to eliminate systemic time errors that would otherwise occur when returning from sleep. At the end of every communication session with shore the Linux ‘ntpdate’ command is used to set system time. If the ‘ntpdate’ command is successful the Ricoh RTC is updated with the system time.
System time can be set from the Linux command line using the ‘date’ command. Help for the date command is available at the Linux command line using the –help flag as follows,
At least two versions of this link have been implemented – a version that uses Globalstar satellite radio and a version that uses the Freewave radio modem. The difference is simply in the script that is passed to PPP to initialize the link.
When SIAM needs to use the link, it calls the script indicated by the system property "CommsManager.onString". This is typically either "pon longhaul" or "pon globalstar". When it’s done with the link, it calls the "CommsManager.offString" script, typically commsOff, which is a script that, among other things, calls "poff longhaul" or "poff globalstar".
The pon and poff scripts
(located in /usr/sbin) simply call and destroy pppd. Invoking "pon
longhaul" is equivalent to a command of "pppd call longhaul". The object of
the call, in this case, "longhaul", is expected to be a file containing pppd
options, located in /etc/ppp/peers. These files (/etc/ppp/peers/longhaul and
/etc/ppp/peers/globalstar) contain command-line options that are passed to
pppd. If you do a "man pppd" on any standard Linux system, you’ll find a rich
set of options that pppd understands. For example, the
/etc/ppp/peers/longhaul file contains the following (as of this
writing):
Among other things, it tells pppd
to use the IP address associated with "mooring:", to use serial port
/dev/ttySX12 at 57600 baud, don’t use the carrier detect line (local), but do
use RTS/CTS (crtscts). It also tells it to turn on radio power when trying to
connect (connect rfpower.ops 1 on) and turn it off when
disconnecting.
/etc/ppp/peers/globalstar has a similar but slightly different set of options. You can configure the way that pppd uses the link by editing the appropriate file in /etc/ppp/peers.
The commsOff script is located in /mnt/hda/siam/utils/commsOff, and primarily calls "poff <linkname>". It also calls ntpdate for time management, and makes sure radio power is off (in case pppd fails).
Secondary Link
The secondary link, when present, is used simply as a Linux login
console. It’s enabled by inserting the following line into file
/etc/inittab:
T13:23:respawn:/etc/auxTelem.sh -b 38400 -t 30 -s 300 &>/dev/null
This causes the system to run the script /etc/auxTelem.sh, and to rerun it continuously when it exits. This script, in turn, calls the Linux command "getty", which prints the login prompt to the port.
The –b option tells it the baud rate. The –t option is the timeout for "getty" – after that number of seconds, getty times out and the radio is turned off. The –s option tells it how long to sleep before running again.
SleepManager.enabled can be set to false to disable sleep management. If you do this, the CPU will never go into sleep mode. For normal (power saving) operation, set this property to true.
SleepManager.sleepString is normally set to "/root/suspend.sh". This is a shell script in the /root directory (caution – it must be owned by root and have the setuid bit on – ‘chmod 4755 suspend.sh). The line in this file that actually puts the CPU to sleep is the call to suspend (which also must be owned by root with setuid bit on). The rest of the script does things like sync the file system, manage the RTC, etc. The call to ‘setPWER’ can be changed to allow other conditions to wake up the CPU from sleep – see the SA1100 Microprocessor manual section on the PWER register. Most of the bits in PWER allow bits in the GPIO register to wake us up; see the Sidearm4 hardware documentation for the use of GPIO.
Keeping
the cpu awake
The cpuAwake
utility can be used to indicate that you need the CPU awake for a period of
time. If you’re logged onto the SIAM node in a Linux shell, you can simply
type the command
cpuAwake seconds
to stay awake the indicated number of seconds. When done, type
cpuAwake 0
to indicate you no longer need the system awake. See the HTML documentation on cpuAwake for the full command syntax. Note that cpuAwake does not guarantee availablility of the wireless link.
Obtaining a
lease for the wireless link
Similarly, you can indicate that you need to use the radio
communications link by using the lease
command (get a lease for the comms link). Again, if you’re on the node, just
type
lease seconds
to keep the radio on for the indicated number of seconds. This has the side effect of also keeping the CPU awake, since it’s useless without the CPU running pppd. Thus, many people have used this facility to keep the CPU awake, rather than ‘cpuAwake’. But note that lease uses RMI, and can take several seconds to run, during which time the CPU could decide to go to sleep. CpuAwake is really better for keeping the CPU awake.
If you’re not on the node, but are logged on elsewhere (e.g. the portal), and want a comms link to the node, then just add the node name to the lease command. For example, if the node’s name is "mooring" and you want a 10 minute lease, use the command
lease 600 mooring
When getting a lease, the lease command returns a "lease ID" (an integer). To give up the lease and allow the comms link to turn off (assuming no one else has a lease), type
lease –terminate leaseID
For the full syntax of the lease command,
see the HTML doc for
lease.
annotatePort – annotate specified instrument data stream
annotatePort mooring1 3 "Removed barnacles from shutter"
auxTelem.sh - Shell script passed to /etc/inittab to enable
catJar - list the contents of a jar file
catJar /dev/ttySX2
catPuck - quickly identify the cotents of a puck
catPuck /dev/ttySX7
conPort - acquire a console to a specific port
conPort mooring2 10
cpuAwake - Keep the CPU awake (veto sleep mode), or schedule future wakeup
cpuAwake mooring1 600
dpaView – Manipulate DPA ports
exitNode - shutdown each instrument service and exit the SIAM application
exitNode localhost
getLastSample - retrieve and display latest data sample from specified instrument
getLastSample mooring3 8
getMetadata - retrieve and display metadata from specified instrument
getMetadata localhost 12
lease - get a lease on the node's communications link
lease 3600
listPorts - list ports and services associated with a specified node
listPorts mooring5 -stats
listSwitches - list DPA power switches associated with a specified node
listSwitches localhost
logView - Display packets from specified SIAM log
logView 1295 .
node - start a MOOS node service
node shore
nodeStatus - list diagnostic information for specified MOOS node
nodeStatus localhost
portal - launch a portal server
portal -publish
portStatus - display diagnostic information for specified port
portStatus mooring8 0
powerDownPort - power down and isolate an instrument port
powerDownPort localhost 4
powerUpPort - power up and enable comms on an instrument port
powerUpPort localhost 4
readPuck - extract the contents of the puck and store it in a local file
readPuck /dev/ttySX5
ricohRTC - get/set time maintained by the Ricoh RTC; or interactively set its registers
samplePort - trigger acquisition and display data sample from specified instrument
samplePort mooring2 7
scanPort - load instrument service for specified port
scanPort mooring8 3
schedulerUtils – add, remove, modify sampling schedules
shutdownPort - shutdown the service on specified port
shutdownPort mooring8 3
suspend - put the CPU into suspend (sleep) mode
switchHiVoltage - enable the high voltage converters on the MOOS Power System
switchHiVoltage mooring1 4 ON
writePuck - write service code and metadata to the puck's non-volatile memory
writePuck
jars/Seabird-2843.jar /dev/ttySX3