Servers¶

This doc describes where and how various M3 applications are run. They are organized by server.

m3-u.shore.mbari.org¶

This is the proxy server for the M3 rest services as well as the web server for videos. You can access it at https://m3.shore.mbari.org/.

Server Name

The server is m3-u.shore.mbari.org and has a CNAME pointing at it of m3.shore.mbari.org.

galadriel.shore.mbari.org¶

Runs M3 rest services

Docker on Galadriel¶

mbari/annosaurus on port 8100
mbari/vampire-squid on port 8200
mbari/oni on port 7070
registry:3 on port 5000. This an internal docker repository

elrond.shore.mbari.org¶

Runs annosaurus instances.

Docker on Elrond¶

mbari/annosaurus on port 8100
mbari/raziel on port 8400. This is the VARS configuration server.

Crontab on Elrond¶

30 18 * * 6 "$HOME/Applications/vars-cli/bin/merge_housekeeping.sh" > $HOME/Library/Logs/elrond_merge_housekeeping.log 2>&1

celebrimbor.shore.mbari.org¶

Runs supporting rest services and midwater transect web app

Docker on Celebrimbor¶

mbari/annosaurus on port 8100
mbari/charybdis on port 8300
portal.shore.mbari.org:5000/mbari/midwater-transect-webapp on port 8080

Cron on Celebrimbor¶

Running under brian. Note that the latest Java LTS should be installed on Quasar so non-docker apps can be run to. This simplfies networks share access for some apps.

# --- FRAMEGRABS
0 0 * * 2 $HOME/Applications/vars-cli/bin/images_find_deleted /mbari/framegrabs > $HOME/Library/Logs/celebrinbor_find_deleted_images.log 2>&1

# --- EXTERNAL SUPPORT
# Replication runs on 1st Monday of month. Update images on first tuesday
0 0 8 * *  /u/brian/Applications/vars-cli/vars kb watermark /mbari/VideoLab/VARS_Images/external /u/brian/Applications/vars-cli/etc/overlay.png > /u/brian/Library/Logs/celebrinbor_watermark_kb.log 2>&1
#0 12 8 * * /u/brian/Applications/vars-cli/vars anno watermark /mbari/framegrabs_watermarked /u/brian/Applications/vars-cli/etc/overlay.png > /u/brian/Library/Logs/celebrinbor_watermark_anno.log 2>&1

# --- FATHOMNET
0 * * * *  BASH_ENV=~/.bashrc bash -l -c "conda activate;/u/brian/Applications/fathomnet/write_homemap.py /mbari/FathomNet/web/assets/images/homemap.png > /u/brian/Library/Logs/quasar-write_homemap.log 2>&1"

#0 0 * * * docker rm fv0 > /dev/null 2>&1; docker run --name fv0 -e FATHOMNET_APIKEY=RGNXXgFDCrQ2AVTNuifFUbXzVFu3x7 mbari/fathomnet-support validate-images -d=2 > /u/brian/Library/Logs/celebrinbor-validate-images-daily.log 2>&1
#0 0 1 * * docker rm fv1 > /dev/null 2>&1; docker run --name fv1 -e FATHOMNET_APIKEY=RGNXXgFDCrQ2AVTNuifFUbXzVFu3x7 mbari/fathomnet-support validate-images -f > /u/brian/Library/Logs/celebrinbor-validate-images-monthly.log 2>&1
0 0 * * * docker rm fv5 > /dev/null 2>&1; docker run --name fv5 -e FATHOMNET_APIKEY=RGNXXgFDCrQ2AVTNuifFUbXzVFu3x7 mbari/fathomnet-support fathomnet images add-missing-width-and-height > /u/brian/Library/Logs/celebrinbor-add-missing-width-and-height-daily.log 
0 0 * * 0 docker rm fv2 > /dev/null 2>&1; docker run --name fv2 -e FATHOMNET_APIKEY=RGNXXgFDCrQ2AVTNuifFUbXzVFu3x7 mbari/fathomnet-support fathomnet notify invalid-images > /u/brian/Library/Logs/celebrinbor-invalid-image-notify.log 2>&1

0 * * * * docker rm fv3 > /dev/null 2>&1; docker run --name fv3 mbari/fathomnet-support fathomnet ncei archive-scanner -a RGNXXgFDCrQ2AVTNuifFUbXzVFu3x7 "https://oer.hpc.msstate.edu/FathomNet/staging/" > /u/brian/Library/Logs/celebrinbor-archive-scanner.log     2>&1

# every 30 minutes scan for new image packages and notify users
*/30 * * * * docker rm fv4 > /dev/null 2>&1; docker run --name fv4 mbari/fathomnet-support fathomnet notify new-uploads -a RGNXXgFDCrQ2AVTNuifFUbXzVFu3x7 > /u/brian/Library/Logs/celebrinbor-upload-scanner.log 2>&1

# The following runs fine on dev machines but each update timesout when run from quasar. Whay???
#0 6 * * * /u/brian/Applications/vars-cli/bin/vars i2map merge-all -t 30 /mbari/AUVCTD/surveys /u/brian/Library/Logs/celebrinbor-i2map-merge 2>&1

arwen.shore.mbari.org¶

Arewn has docker_user configured with r/w access to atlas:/framegrabs and read access to titan:/M3. Its purpose is to archive framegrabs and register new videos.

Docker on Arwen¶

mbari/beholder on port 8088
`mbari/skimmer1 on port 8089
galadriel.shore.mbari.org:5000/arwen/panoptes on port 8080

Cron on Arwen¶

Running under brian

0 19 * * 2   $HOME/bin/arwen/registrar.sh /mbari/M3/master "https://m3.shore.mbari.org/videos/M3/master" 300 > $HOME/Library/Logs/arwen_registrar_master.log 2>&1
0 1 * * 1,4  $HOME/bin/arwen/registrar.sh /mbari/M3/mezzanine "https://m3.shore.mbari.org/videos/M3/mezzanine" 300 > $HOME/Library/Logs/arwen_registrar_mezzanine.log 2>&1
0 18 * * 2,5 $HOME/bin/arwen/registrar.sh /mbari/M3/proxy "https://m3.shore.mbari.org/videos/M3/proxy" > $HOME/Library/Logs/arwen_registrar_proxy.log 2>&1
0 1 * * 5    $HOME/bin/arwen/registrar.sh /mbari/M3_Proj_Bioinspiration/StereoCameras/ "https://m3.shore.mbari.org/videos/Projects/Bioinspiration/StereoCameras" > $HOME/Library/Logs/arwen_registrar_stereocameras.log 2>&1
0 12 * * 6   $HOME/bin/arwen/registrar.sh /mbari/M3_Proj_Bioinspiration/Optim/LRAUV/ "https://m3.shore.mbari.org/videos/Projects/Bioinspiration/Optim/LRAUV" > $HOME/Library/Logs/arwen_registrar_optim.log 2>&1
#0 12 * * 6   $HOME/bin/arwen/registrar.sh /mbari/M3_Proj_Bioinspiration/Triton/ "https://m3.shore.mbari.org/videos/Projects/Bioinspiration/Optim/Triton" > $HOME/Library/Logs/arwen_registrar_triton.log 2>&1
0 2 * * 7    $HOME/bin/arwen/registrar.sh /mbari/M3/Projects/Piscivore "https://m3.shore.mbari.org/videos/M3/Projects/Piscivore" > $HOME/Library/Logs/arwen_registrar_piscivore.log 2>&1
0 1 * * 6    $HOME/bin/arwen/registrar.sh /mbari/M3/Projects/Midwater/TopoCam "https://m3.shore.mbari.org/videos/M3/Projects/Midwater/TopoCam" > $HOME/Library/Logs/arwen_registrar_topocam.log 2>&1

registrar.sh¶

#!/usr/bin/env bash

# Brian Schlining
# 2025-10-09
#
# USAGE: registrar.sh <PATH> <URL_OF_PATH> <MAX_VIDEOS (optional)>
#
# EXAMPLE: 
#    registrar.sh /mbari/M3/master "https://m3.shore.mbari.org/videos/M3/master" 300 
#    registrar.sh /mbari/M3/mezzanine "https://m3.shore.mbari.org/videos/M3/mezzanine"

export PATH="$PATH:/u/brian/Applications/vars-cli/bin"

if [ $# -lt 2 ]; then
    vars media register-all --help 
    exit 1
fi

MAX_PARAM=""
if [ $# -eq 3 ]; then
    MAX_PARAM="--max $3"
fi

vars media register-all "$1" "$2" $MAX_PARAM

expanse.shore.mbari.org¶

Expanse replaces the older server seaspray. It runs the Deep-sea guide as well as various support applications.

Podman on Expanse¶

portal.shore.mbari.org:5000/mbari/deepseaguide on port 8080

Cron on Expanse¶

Running as local user (non LDAP) brian

#0 18 * * FRI  podman rm housekeeping > /dev/null 2>&1; podman run --name=housekeeping portal.shore.mbari.org:5000/mbari/m3-merge-rov housekeeping.sh > /home/brian/.m3/logs/merge.log 2>&1
30 7 * * SAT  podman rm midwater > /dev/null 2>&1; podman run --name=midwater portal.shore.mbari.org:5000/mbari/midwater-transects update-database-app > /home/brian/.m3/logs/midwater-db-update.log 2>&1
51 7 * * *    podman rm samples1 > /dev/null 2>&1; podman run --name=samples1 portal.shore.mbari.org:5000/mbari/m3-merge-rov samples_update_rovctd > /home/brian/.m3/logs/samples_update_rovctd.log 2>&1
59 7 * * *    podman rm samples2 > /dev/null 2>&1; podman run --name=samples2 portal.shore.mbari.org:5000/mbari/m3-merge-rov samples_update_keywords > /home/brian/.m3/logs/samples_update_keywords.log 2>&1
#0 9 * * SAT   podman rm sync > /dev/null 2>&1; podman run --name=sync mbari/m3-support  sync-video-metadata -k MjAxNy0wMS0yM1QxODoxNzoyMy41MjZa > /home/brian/.m3/logs/sync_video_metadata.log 2>&1