OASIS Operation Documentation

OASIS M1 and M1Slave1 Mooring Turn

Note

Everything below assumes the deployment just ending is 202008 and the deployment coming up is 202106...

Note

Below describes in detail the setup for M1. The setup for M1Slave1 is roughly the same but many steps are not relevant. (eg. there is no tstring on the slave etc.)

Pre-deployment Prep

Log in to coredata a yourself

ssh coredata (as yourself)

sudo -u coredata -i

OASIS M1 Mooring Turn

Pre-deployment Prep

Log in to coredata a yourself

ssh coredata (as yourself)

sudo -u coredata -i

Set up deployment folders

echo This sets up the main deployment directory
cd /mbari/oasis_coredata/deployments/m1
mkdir 202106
cd 202106

echo These pull over the prior deployment configs and header
mkdir -p ./cfg/headers
cp -R ../202008/cfg/* ./cfg
cp ../202008/cfg/o3gf0.cfg  ./cfg
cp ../202008/cfg/tstring.2017m1  ./cfg/tstring.2021m1

echo Now creating the folders to hold raw downloads
mkdir -p ./raw/tar
mkdir -p ./raw/erase

echo Now creating folders for instruments and data
mkdir -p ./m1
mkdir -p ./m1/adcp
mkdir -p ./m1/emeter
mkdir -p ./m1/gps
mkdir -p ./m1/hrh
mkdir -p ./m1/hyperOCR
mkdir -p ./m1/impod02
mkdir -p ./m1/impod03
mkdir -p ./m1/impod04
mkdir -p ./m1/impod05
mkdir -p ./m1/impod06
mkdir -p ./m1/impod07
mkdir -p ./m1/impod08
mkdir -p ./m1/impod09
mkdir -p ./m1/impod10
mkdir -p ./m1/impod11
mkdir -p ./m1/impod12
mkdir -p ./m1/lwr
mkdir -p ./m1/microcat
mkdir -p ./m1/microcat3
mkdir -p ./m1/o3gf
mkdir -p ./m1/swr
mkdir -p ./m1/data
mkdir -p ./m1/error
mkdir -p ./m1/gpsclock
mkdir -p ./m1/hs2
mkdir -p ./m1/imhs2
mkdir -p ./m1/isus
mkdir -p ./m1/metsys
mkdir -p ./m1/microcat2
mkdir -p ./m1/netcdf 
mkdir -p ./m1/oasis
mkdir -p ./m1/tstring

Set up the config file

cd /mbari/oasis_coredata/deployments/m1/202106/cfg

vi m1.cfg

Change the deployment name at line 1.
Correct name in the ssds confg file/location. (~ line 54)
Change the 'data' path (~line 56) for the new deployment.
Change the 'errors' path (~line 57) for the new deployment.
Change the 'header' path (~line 58) for the new deployment.
Change BOTH the tstring path and the tstring filename (~line 59)
Change the 'gndflt' path (~line 60) for the new deployment.
Note above are the ONLY changes. Dont mess with calibrations for ctd, specprr amd hr/hs instruments.

Edit the tstring configuration file

Mooring techs (Jared) will supply a spreadsheet with tstring instriment serial numbers and depths

cd /mbari/oasis_coredata/deployments/m1/202106/cfg
vi tstring.2021m1

Make it correct for the info provided by Jared

Something like below:

# sn  depth  hasP pod# hasO2 ctdtype
0823   40    N    2    N      0
0819   60    Y    3    N      0
0385   80    N    4    N      0
0416  100    Y    5    N      0
0825  150    N    6    N      0
1245  200    Y    7    N      0
0475  250    N    8    N      0
1251  300    Y    9    N      0
0476  999    Y   10    N      0
9309  225    Y   11    Y      1
03712120  50    Y   12    Y      1
#  spaces only, no tabs or commas please
#  ctdtype=1 for IMCTD's like SBE16IM or SBE37w/O2
#  ctdtype=0 for old (pre-9/2010) pods w/o O2

Edit the header files

If you cd to the headers directory and grep for the 'Deployment Date' you will see many (>30) files with the prior deployment date - these will need correcting, now - tentatively - and again on actual deployment day.

cd /mbari/oasis_coredata/deployments/m1/202106/cfg/headers
grep Deployment *.hdr

aanderaaO2.hdr:# Deployment Date:  22 June 2020
adcp.hdr:# Deployment Date:  22 June 2020
ctd.hdr:# Deployment Date:  22 June 2020
durafetph.hdr:# Deployment Date:  22 June 2020
emeter.hdr:# Deployment Date:  22 June 2020
error.hdr:# Deployment Date:  22 June 2020
fluor.hdr:# Deployment Date:  22 June 2020
gpsclock.hdr:# Deployment Date:  22 June 2020

(etc)

You can fix these by hand, or edit and use the "updatehdrfiles" script provided there.

cd /mbari/oasis_coredata/deployments/m1/202106/cfg/headers
vi updatehdrfiles
./updatehdrfiles

Here is the magical updatehdrfiles script (make sure its executable!)

for i in *.hdr; do
  sed -i 's/22 June 2020/14 June 2021/g' $i
done

The getM1_XXXXX script

Next we go set up a new "getM1" script and add it to the "download" script.

We start by copying the 'current' m1 script to the new target script...

Warning

Make sure not to clobber the 'current' script because it's mooring is likely still out there!

cd ~/moorings/downloads/bin
cp getM1_202008 getM1_202106
vi getM1_202106

Change the deployment name from 202008 to 202106 (see line: set DEPLOYMENT = 202106)
Change the TNC Configuration (see line: set TNC_CALL = 2021M1). This is important for the FailMail subject line.
Comment out the IP and radio port that we inherited when we copied (That mooring is still out there so we don't want a conflict!)
Now set the IP and radio port number to what Jared provides. INITIALLY it will likely be the terminal server in the buoy assembly room. Closer to deployment we will change it again to the radio that will be used.
Set the initial 'FAILMAIL' recipients
Near lines 154-163 we set the parent device id's for SSDS, they toggle even/odd years.
Near line 209 Set SEND_SSDS = $FALSE until Kevin is ready to start getting data into SSDS
Save and exit

Test Test Test

Note

Do a double check! - you dont want to dump a bunch of garbage data into the 'current' m1 deployment!

all of the paths in the m1.cfg
all of the deployment paths etc getM1_xxxx are correct
the new deployment radio in your getM1_xxxxx is correct.
During early testing, we disable the push to ssds around line 208 by setting the SEND_SSDS = $FALSE. At least until Kevin is ready for us to start publishing to him. We then 'turn it on' when things are closer to actual deployment.
Check that the new deployment radio is NOT the same as any other current deployment radio. The grep command below can be useful for this.

Note

Actually - since M1-Slave1 is a slave to M1 - they use the share the same radio/port. What we want to check for is that different **deployment:1 ** aren't crossed up on radios...

grep TNC get* | grep -v # | grep 134.89

getM1_202008:set TNC_PORT = 134.89.45.29:20000
getM1_202106:set TNC_PORT = 134.89.12.116:5001
getM1slave1_202008:set TNC_PORT = 134.89.45.29:20000
getM1slave1_202106:set TNC_PORT = 134.89.12.116:5001
getOA1_2020:set TNC_PORT = 134.89.45.24:30000
getOA2_2019:set TNC_PORT = 134.89.45.24:10000

getM1_202106 can now be run from the command line

Add your script to the 'download' that cron runs

Now edit the script ~/moorings/downloads/bin/download and add a section to the run the new getM1_202106 script.

This is usually an add of an additional "get" section because your setting this up pre-deployment and the prior mooring is still out there running.

On cut-over day

Fixing the "Current Deployment" symbolic link

We support a directory of deployments called "current" so users can easily find data from currently deployed platforms. It simply contains symbolic links to whatever the 'real' deployment directory is.

So on cutover day, we need to get rid of the 'old' link and make a new one.

cd /mbari/oasis_coredata/deployments/current

ls -la

drwxrwxrwx.  3 coredata games 111 Sep 10 13:15 .
drwxrwxrwx. 17 coredata games 400 Aug 13 16:05 ..
drwxrwxrwx.  2 coredata games 160 May  9  2013 logs
lrwxrwxrwx.  1 coredata games  15 Sep  2 09:21 m1 -> ../m1/202008/m1
lrwxrwxrwx.  1 coredata games  29 Sep  2 09:23 m1-slave1 -> ../m1-slave1/202008/m1-slave1
lrwxrwxrwx.  1 coredata games  17 Oct 31  2018 oa1 -> ../oa1/201810/oa1
lrwxrwxrwx.  1 coredata games  17 Sep 10 13:15 oa2 -> ../oa2/201912/oa2

you will see symbolic links to current "active" directories.. we will delete the oa1 and make a new one...

rm m1

ln -s ../m1/202106/m1 m1

The same should also be done for the slave can

rm m1-slave1

ln -s ../m1/202106/m1-slave1 m1-slave1

Fixing oasis.rcd

Operations people 'should' reset the on-board CF (compact flash) to get rid of old data collected dockside (but sometimes they forget...) So after a successful download from the newly deployed platform you need to verify that oasis.rcd is pointing correctly:

Look at the most recent download for the 'logs' command output.

/mbari/oasis_coredata/deployments/m1/YYYYMM/raw
cat m1.2021061.11

and look for output like:

OASIS> Connected...
logs
Blocks 0 to 57 logged
Log records in block 57:  1877
Next download is block 57 record 1760
OASIS>

As shown above, there is a lot of data so they likely forgot to reset. Thats OK

If it shows only a few blocks logged it indicates they did a reset - as shown below.

OASIS> Connected...
logs
Blocks 0 to 2 logged
Log records in block 2:  1877
Next download is block 2 record 1760
OASIS>

Now look at the oasis.rcd file of the same directory (shown below).

compare the 'Blocks logged' number above to the last entry in oasis.rcd. If the 'Blocks logged' number from above is less than the last entry in oasis.rcd it needs to be fixed.

cat oasis.rcd

 m1.48: 0-16383
 m1.49: 0-16383
 m1.50: 0-16383
 m1.51: 0-16383
 m1.52: 0-16383
 m1.53: 0-16383
 m1.54: 0-16383
 m1.55: 0-16383
 m1.56: 0-16383
 m1.57: 0-1877

Determine if you are 'in sync' or not. In the 'out of sync' case, you will simple edit the oasis.rcd file as shown below.

m1.1: 0-16383
m1.2: 0-1877

Revisit the update header files

Revisit the update header files sectin above to set the correct actual deployment date in hdr files if needed.

MORE Cutover TODOs

Edit ~/moorings/downloads/bin/process. Fix the m1 deployment number to 202106. Create a new set doM1_202106 = $FALSE and a set doM1slave1_202106 = $FALSE at the top of the file. Next, copy and paste the the old conditional statements to doM1_202106 and doM1slave1_202106 and edit the 'foreach year' list to just be 2021 (in this case).
Also need to edit 'years' in both scripts process_special1_daily and process_special1_hourly to 2021 (in this case).
make new nightly tarraw script in moorings/downloads/bin and then add it to the coredata crontab. (cp the old and then edit)
In the getM1_XXXXX script, make sure the push to ssds is enabled by setting the SEND_SSDS = $TRUE (near line 208).
Once the mooring turn is complete, the downloads can be disabled for the recovered mooring. This is done by editing the ~/moorings/downloads/bin/download script and commenting out the lines for recovered mooring. For example, you would comment out:

echo Downloading M1 202008
$bins/getM1_202008 >> $logs/getM1_202008.out

echo Downloading M1-Slave1 for 202008
$bins/getM1slave1_202008 >> $logs/getM1slave1_202008.out

OTHER RESOURCES

M1 QuickLook Data Products

Francisco's group provides these data products for M1 using the processed data. I dont know which data path... (Legacy or SSDS) I suspect legacy and maintained by Reiko.

see: https://www3.mbari.org/oasis/qc/index.html

And here is from CENCOOS which probably comes theu the SSDS pathway.

https://data.cencoos.org/#metadata/20716/station

OASIS 3 'Some What Outdated"

The Oasis3 docs from "Bob Herlein" and "Zorba" days. Reference old hpux linus system tsunami and the datapaths and download scripts etc have changed. But its good historically to see how the system has evolved.

https://mww.mbari.org/oasis/InternalDoc/index.html

OASIS to SSDS

The original OASIS to SSDS docs - circa 2005 https://mww.mbari.org/oasis/InternalDoc/OASIStoSSDS/doc/html/

OASIS 5 Documents that may be helpful to understanding OASIS-3/4

Kent maintains the OASIS 5 documentation at: http://fangtooth2.shore.mbari.org/oasis5/

Much of it is helpful to understanding Oasis 3 and Oasis 4 controllers as used on M1 and OA moorings

See in particular:

http://fangtooth2.shore.mbari.org/oasis5/content/man/man-telemetry.html#

http://fangtooth2.shore.mbari.org/oasis5/content/ref/software/Oasis5Applications_Rev8.pdf