900612 ESP -- SURF Center : Design
This page last changed on Aug 13, 2010 by kgomes.

As a first step in designing the application to act as the Graphical User Interface (GUI) for the ESP instrument(s), a diagram was created to capture the existing structure of the digital information that will need to be tracked for each ESP instrument and the various places it will live. The ESP directory structures contains the following:

  1. /var/log on the linux host is the root directory for the FTP site (i.e. = ftp://espName).
  2. /var/log/messages contains linux kernel messages
  3. /var/log/vsftpd.log records all FTP site traffic
  4. /var/log/$USER is top level output directory for each user's ESP data (can be overridden with $ESPlog variable, but this is rare)
  5. Content in /var/log depends on configuration, but default is:
    1. hires: high resolution camera images (each approx 3.5 MB)
    2. lores: low resolution camera images
    3. midres: medium resolution camera images (typically autoexposures)
    4. Top level directory contains default resolution camera images (typically fixed exposures)
  6. Only files in top level directory are uploaded to shore
  7. File types are:
    1. *.tif = TIFF camera images
    2. *.pcr = Comma Separated Value PCR Data
    3. *.out = Console output capture
      1. Text normally output to the esp interactive console
      2. Redirected here when running non-interactive mission script
      3. Can be viewed with showlog command
      4. File is named after operating mode (real, sim, simreal, quick)
    4. *.log = detailed, binary esp engineering logs. The file is named after operating mode (real, sim, simreal, quick). Can be viewed with dumplog.
    5. *.tube = list of carousel tubes processed
      1. One tube per line, last line is the currently active source tube
    6. ISUS*.err = ISUS error messages (direct from the ISUS)
    7. ISUS*.dat = text records of the ISUS data (complex and incorporates CTD data)
    8. CTD*.hex = CTD data in hexidecimal formatted text records
      1. Seabird specific format

NOTE: Brent wrote up some instructions on parsing the log file here

The initial thought about how a complete data management system might look is:

Things to note
  1. The ESP instrument creates and appends content to some directory on the main CPU.
  2. An FTP server makes that content available to the outside world.
  3. The shore station takes the FTP content from the deployed ESP and creates a mirror of the content on the shore station (this includes the ruby scripts used to run the ESP).
  4. On the shore station, it can be managing content from more than one ESP and by a manual convention, old deployments are named YYMMM.espname.
  5. The .log files are files that have to be read in by a Ruby application as they are serialized Ruby objects and so need Ruby to interpret them
  6. The ESP process is basically started with a 'nohup' command and any console messages are then routed to the .out file.
  7. The link is not always strong over the Freewave so communication can sometimes be very slow.
  8. If a command GUI wishes to send commands directly to the ruby interpreter running on the ESP itself, a slow link would cause major issues.
  9. The data is then manually copied to the \\tornado\esp directory under a different layout than that of the shore station. The list below shows the directory structure on
    tornado which means that the 'esp' directory on tornado may actually be the YYMMM.espname directory on the shore station.
    1. YYMMM
      1. espname
        1. esp
  10. The 'messages' file is simply a symbolic link to the /var/messages so that the FTP mirror will pick them up for debugging.

So, in order to manage the data through some application, it is good to realize that the data will mostly have three locations through time:

  1. The ESP instrument itself. This data is created during the deployment, is transmitted back to shore but is NOT kept permanently on the ESP instrument itself.
  2. The shore station. This is an FTP synch of the data on the ESP itself and, assuming connectivity is there, will be a duplicate copy of what is deployed on the ESP itself. This will outlive the data stored on the instrument itself, but it's parent directory may be renamed when a new deployment comes up.
  3. 'Permanent' storage location. This is a copy of the data that will be stored somewhere that will be a final resting place for all intents and purposes. There is a directory structure that will be adhered to and should provide a common location for all the data from the ESPs. This is sort of a post-deployment activity and should not be relied upon for storage during the deployment.

So, in order to manage this synchronization and storage of data, it was decided that the GUI would be reponsible for that synchronization. Initially it would be a manual process as it will be time consuming and the user should be responsible for firing it off. Then, once it is solidified a bit, it would be nice to push it back to a background thread that happens automatically. The GUI for this synchronization, might look something like this:


edit this mockup

Next, the design group sat down to create a model that would allow the application to capture all the concepts that need to be managed in the GUI application. That data model is available at the link below.

  1. Data Model

Once this model was developed to capture the concepts that will need to be stored persistently, we went to work on some mockups of what the GUI might look like. From an end user's point of view, the application will basically serve two needs: Operation of the instrument(s) and analysis of the data from the instrument(s). To meet the requirements, various GUI components will be needed (terms are from data model):

  1. Create Instrument
  2. Edit Instrument
  3. Delete Instrument
  4. Create ProtocolType
  5. Edit ProtocolType
  6. Delete ProtocolType
  7. Create PhaseType
  8. Edit PhaseType
  9. Delete PhaseType
  10. Add ProtocolTypes to PhaseTypes
  11. Remove ProtocolTypes from PhaseTypes
  12. Create ConsumableType
  13. Edit ConsumableType
  14. Delete ConsumableType
  15. Create/Add a ConsumableAmount by creating a connection between ProtocolType and ConsumableType using an amount to characterize the link.
  16. Edit the ConsumableAmount link
  17. Delete a ConsumableAmount from a ProtocolType
  18. Create EventType (Default events should be DEPLOYMENT, PHASE, PROTOCOL_RUN)
  19. Edit EventType
  20. Delete EventType
  21. Create Deployment for Instrument
  22. Edit Deployment
  23. Delete Deployment (This will cascade delete events attached to it)
  24. Create Bag
  25. Edit Bag
  26. Delete Bag
  27. Add Bag to Deployment (for reagents, waste and power)
  28. Create and Add a Phase (instance of PhaseType) to Deployment
  29. Edit Phase
  30. Delete Phase (and all child events)
  31. Create Resource (HTTP accessible content)
  32. Attach Resource to Deployment or Phase
  33. Delete Resource (not the Resource itself, just the information about it).
  34. Create an Image (HTTP accessible image)
  35. Attach Image to Deployment or Phase
  36. Delete Image (not the actual image, just the information about it).
  37. View the Tide web service and add the specific location to the list of interesting locations
  38. Remove tide location
  39. Create new Comment
  40. Attach comment to any event, image, or resource
  41. Delete comment
Log file interface

In order to get this information from the ESP system, the GUI needs and interface to the ESP which ended up being the log file. The documentation for that interface can be found at ESP Log Interface

Mockups

Now, if we think about sitting down at the GUI and starting it up, I would think the first view we would want is sort of an operational view of things. The things I would like to see are:

  1. ESPs available
  2. Overall calendar showing all planned deployments and logged events for past and current deployments. (should be able to flip this on/off quickly when I want to dive into one instrument).
Mockup Notes
A mockup of what the user might see upon first starting the application.
Logically, the next step would be for the user to explore the information presented:
  1. Information about each ESP
    1. Serial Number
    2. Picture?
      Here is the mockup showing what might happen if the user selected one of the instruments.
After learning about the instrument, I might want to know more about the deployments (past, present and future). The user could then select a deployment from a list or on the calendar and find the following information:
  1. Name, geospatial location
  2. List of scheduled phases with start times

In this view, the individual events (including phases would show up on the calendar and the user could then mouse over events to get more details. These events would include comments, logged events, etc. If the user then clicks on an event in either the calendar or in the list of scheduled phases, a dialog box with more details would open.

There will also be a tab to let the user see a table list of all events for the selected deployment (both scheduled and actual). The same dialog box will show if the events are double clicked in the event list.


edit this mockup

edit this mockup
Instrument Dialog Box

edit this mockup
Deployment Dialog

edit this mockup
Menus

edit this mockup

edit this mockup
Define/Edit Phases and Protocol Templates

edit this mockup

edit this mockup
Instrument State Visualization

To examine the state of the instrument for various deployments, the user would then select View->Instrument State from the pull down menu. This would put the user into a new view that shows the instrument state for the selected deployment. At the bottom is a timeline slider the user can control to change what point in time the state is to be visualized. There are also play controls to let the user control a playback of instrument states from the past


edit this mockup

edit this mockup

Attachments:
Logical Deployment Diagram.jpg (image/jpeg)
Logical Deployment Diagram.jpg (image/jpeg)
mockup_Top Level.xml (text/xml)
mockup_Top Level.xml (text/xml)
mockup_Top Level.xml (text/xml)
mockup_Top Level.xml (text/xml)
mockup_Top Level.xml (text/xml)
mockup_Top Level.xml (text/xml)
mockup_Top Level.xml (text/xml)
mockup_Top Level.xml (text/xml)
mockup_Top Level.xml (text/xml)
mockup_Top Level.xml (text/xml)
mockup_Top Level.xml (text/xml)
mockup_Top Level.xml (text/xml)
mockup_Top Level.png (application/png)
mockup_Top Level.png (application/png)
mockup_Top Level.png (application/png)
mockup_Top Level.png (application/png)
mockup_Top Level.png (application/png)
mockup_Top Level.png (application/png)
mockup_Top Level.png (application/png)
mockup_Top Level.png (application/png)
mockup_Top Level.png (application/png)
mockup_Top Level.png (application/png)
mockup_Top Level.png (application/png)
mockup_Top Level.png (application/png)
mockup_InstrumentSelect.xml (text/xml)
mockup_InstrumentSelect.xml (text/xml)
mockup_InstrumentSelect.xml (text/xml)
mockup_InstrumentSelect.png (application/png)
mockup_InstrumentSelect.png (application/png)
mockup_InstrumentSelect.png (application/png)
mockup_DeploymentSelect.xml (text/xml)
mockup_DeploymentSelect.xml (text/xml)
mockup_DeploymentSelect.xml (text/xml)
mockup_DeploymentSelect.png (application/png)
mockup_DeploymentSelect.png (application/png)
mockup_DeploymentSelect.png (application/png)
mockup_MouseOverEvent.xml (text/xml)
mockup_MouseOverEvent.xml (text/xml)
mockup_MouseOverEvent.png (application/png)
mockup_MouseOverEvent.png (application/png)
mockup_EventDetails.xml (text/xml)
mockup_EventDetails.xml (text/xml)
mockup_EventDetails.png (application/png)
mockup_EventDetails.png (application/png)
mockup_DeploymentPlannedResources.xml (text/xml)
mockup_DeploymentPlannedResources.xml (text/xml)
mockup_DeploymentPlannedResources.png (application/png)
mockup_DeploymentPlannedResources.png (application/png)
mockup_ConsumedResources.xml (text/xml)
mockup_ConsumedResources.xml (text/xml)
mockup_ConsumedResources.png (application/png)
mockup_ConsumedResources.png (application/png)
mockup_InstrumentStateSelect.xml (text/xml)
mockup_InstrumentStateSelect.xml (text/xml)
mockup_InstrumentStateSelect.png (application/png)
mockup_InstrumentStateSelect.png (application/png)
mockup_InstrumentState.xml (text/xml)
mockup_InstrumentState.png (application/png)
mockup_InstrumentDialog.xml (text/xml)
mockup_InstrumentDialog.png (application/png)
mockup_DeploymentDialog.xml (text/xml)
mockup_DeploymentDialog.png (application/png)
mockup_ESP_Menu.xml (text/xml)
mockup_ESP_Menu.png (application/png)
mockup_Edit_Menu.xml (text/xml)
mockup_Edit_Menu.png (application/png)
mockup_Phase_Type_Dialog.xml (text/xml)
mockup_Phase_Type_Dialog.xml (text/xml)
mockup_Phase_Type_Dialog.xml (text/xml)
mockup_Phase_Type_Dialog.xml (text/xml)
mockup_Phase_Type_Dialog.png (application/png)
mockup_Phase_Type_Dialog.png (application/png)
mockup_Phase_Type_Dialog.png (application/png)
mockup_Phase_Type_Dialog.png (application/png)
mockup_Protocol_Type_Dialog.xml (text/xml)
mockup_Protocol_Type_Dialog.png (application/png)
mockup_Data_Import.xml (text/xml)
mockup_Data_Import.png (application/png)
ESP Data Overview (text/xml)
ESP Data Overview (text/xml)
ESP Data Overview (text/xml)
ESP Data Overview (text/xml)
ESP Data Overview (text/xml)
ESP Data Overview (text/xml)
ESP Data Overview (text/xml)
ESP Data Overview (text/xml)
ESP Data Overview (text/xml)
ESP Data Overview (text/xml)
Document generated by Confluence on Feb 03, 2026 14:16