Shore Side Data System (SSDS)

This is the documentation for the Shore Side Data System software developed at MBARI.

Source Code

Over the years, the development of the software got pretty fragmented and several false starts at refactoring and upgrading left it in a fairly confusing state. In January of 2018, I took a fresh look at the source to figure out where things were at. Here is the situation as of that review:

  1. Repositories:

    1. CVS on moonjelly.shore.mbari.org. I actually found that this was the source code that was the closest (if the THE source) for the current production SSDS. I was able to checkout and build the source with some tweaks to get it to work on modern infrastructure. The main thing is that this all needed to be done using JDK 1.6 to get it to run under JBoss 4.0.3SP1 (which is the current production JBoss). This means the JBoss instance needed to be run under the 1.6 version of the JRE/SDK because the JSP pages needed to be compiled with that version so the JBoss container could run them. This also means the ant build needed to be done using JDK 1.6 so there was class compatibility between the runtime and the compiled classes. Another thing to note is that on the production SSDS_Data, there is a table called DeviceParent that is created by a job (not sure where or how) that is created outside of the confines of the SSDS builds system. It was done as a one-off. It is basically used to drive the pick list in the siamRawDataStep1.jsp page. I actually don't think that table has been updated in years. While there are branches and tags, I believe the 'architecture-change-branch' is the one that is working in production.
    2. SVN on kahuna.shore.mbari.org. This was where the source code was moved when we decided to move away from SVN. The source here is very different from the architecture-change-branch that is in CVS and my guess is that I started some refactoring work here, but never got far. There are no other tags or branches here.
    3. SVN on Google Code. To make this source available publicly, we moved it to an SVN server on Google code. I started the refactoring work there and created a branch called jboss-5.1 as I was trying to upgrade it to JBoss 5 which is version 5 of Java EE. I never deployed anything, but did start some good documentation here. A good deal of the refactoring work that I did on the ingest and raw data access was done here. Google decided to shut down Google Code, so we moved the source to Git on BitBucket.
    4. Git on Bitbucket: This was a direct migration from the SVN repo on Google Code, so along with this came both the main and the jboss-5.1 lines of code. I started a new branch on BitBucket named wildfly-8 and started a migration to Wildfly 8 which is certified for Java EE 7. While I should not have named it with a version, it should be good to run in Wildfly 8-10 as they all are targeting EE 7. This is the branch that University of Hawaii is attempting to migrate to.

  2. Deployments:

    1. Production at MBARI. The current production deployment of SSDS at MBARI is broken out across several machines:
      1. bob.shore.mbari.org: This machine used to be an old Mac XServe that finally gave up the ghost and I migrated the code to a Centos machine still running JBoss 3 (Wow!). These services are deployed on bob.shore.mbari.org:
        1. ssds-data-ds.xml: This is the DataSource connection to the SSDS_Data database running on dione.mbari.org
        2. ssds-sql-ingest-service.xml: Creates a JMS topic called: SSDSSQLIngestTopic
        3. ssds-ruminate-services.xml: Creates a JMS topic called: SSDSRuminateTopic
        4. ssds-ingest-service.xml: Creates a JMS topic called: SSDSIngestTopic
        5. ssds-arrival-service.xml: Creates a JMS topic called: SSDSPacketArrivalTopic
        6. ssds-transmogrify-service.xml: Creates a JMS topic called: SSDSTransmogrifyTopic
        7. In the deploy.last directory:
          1. ssds-transmogrify.jar: This is the transmogrify MDB that takes in SIAM packets and converts them to SSDS style packets and then sends them on to SSDSIngestTopic.
          2. ssds-ingest.jar: This is the Ingest MDB that writes data to the SSDS_Data database and forwards the messages on to SSDSRuminateTopic.
      2. new-ssds.mbari.org: This is the main ssds server that contains the web application and the servlets that folks use to access data from SSDS. These services are deployed on new-ssds:
        1. ssds-data-ds.xml: data source connection to the SSDS_Data database on dione.
        2. ssds-metadata-ds.xml: data source connection to the SSDS_Metadata database on dione.
        3. jmx-invoker-service.xml: Not sure, but looks like I was trying to do something with security, probably related to ssds-login-config-services.xml
        4. ssds-login-config-service.xml: Not sure but looks to create a service using the LDAP connection information stored in conf/ssds-login-config.xml. Pretty sure this was never used and can be removed.
        5. conf/ssds-login-config.xml: configuration properties for connecting to MBARI's LDAP server. Don't think I ever used this.
        6. ssds-ruminate-remote-jms-ds.xml: This connects up to the SSDSRuminateTopic on bob.shore.mbari.org and receives the messages on the ruminate topic. I can't remember how this exactly works, but it enables clients to subscribe to the ruminage topic on new-ssds and get the messages that are actually being sent to the topic on bob.
        7. access.war: Not sure
        8. cimt.war: The web pages for the CIMT stuff, not used anymore
        9. foce.war: The web pages for the FOCE data, not used anymore
        10. mars.war: The web pages for the MARS data, not used anymore
        11. muce.war: The web pages used for MUCE data, not used anymore
        12. ssds-ruminate-republish-service.xml: Creates a topic named SSDSRuminateRepublishTopic which, I believe is what the SSDS UpdateBot listens to for messages. It basically gets message from the Ruminate MDB which is deployed in ssds-rumiante.jar
        13. mail-service.xml: used for allowing software to send emails. I used to do this, but turned it all off.
        14. ssds-new-ssds.ear: This is the main application that deploys all the EJBs that connect the metadata objects with the persistence in the SSDS_Metadata database.
        15. axis.war: I can't remember, but seems like I was using this for some sort of web service stuff.
        16. ssds-ruminate.jar: This is the ruminate MDB which takes in metadata messages from ingest and then decides if the metadata is new or not. If it's new, it records the metadata in the SSDS_Metadata database and then forwards the updated metadata message to SSDSRuminateRepublishTopic where it's picked up by the updatebot.