Overview¶
We’ve put together a set of AI/ML tools and workflows to help with everything from finding and labeling data to training models and running them at scale. These tools supporting processing huge amounts of oceanographic data—including images, video, and audio. Our goal is to make it easier to get your data labeled, train your models, and then use those models on new data. You’ll find these tools in use across many MBARI projects, with model accuracies typically ranging from 76% to 98%.
As AI and ML tools are becoming more accessible and models are rapidly evolving, we’re always looking for ways to improve our workflows and tools, including most recently the Voxel51 tool which allows you to visualize and edit data in an intuitive, interactive way. Check-out the demo below.
Voxel Community edition demo with i2MAP training data¶
Need more details or quick answers? Check out our FAQ or feel free to reach out to anyone on the team.
End-to-End Workflow¶
Key Stages¶
🔍 Finding your data with detection and clustering¶
If your data isn't already localized, you can use detection to help find objects of interest.
🏷️ Labeling data with Tator¶
| Project View | Bulk Editor View |
|---|---|
 |
 |
🔍 Finding similar data with our vector database¶
Training models and checking results¶
Tools¶
🖼️ Images¶
-
SDCAT – Sliced Detection and Clustering Analysis Toolkit
A tool for detecting and clustering objects in images. It helps jump-start labeling for model training on image data. Used across multiple MBARI projects: UAV, BioDiversity, SINKER, Midwater TimeSeries, Planktivore, and CFE lab.
Typically the first step for getting your training data ready. -
🧭 VSS – Vector Similarity Search
We use models like DINOv2 or CLIP to generate these vectors, which are great all-around options. If you want even better results, you can train a model specifically on your own data. Vector databases like Redis keep information in memory, making them much faster than traditional databases for searching through large sets of vectors.
-
🏷️ Tator – Advanced annotation database
Tator is a web-based tool designed specifically for AI workflows. You can use it to label images, videos, and audio, and then export everything to CSV. It handles everything from simple points and lines to polygons and bounding boxes, and it's flexible enough to track whatever metadata you need—like depth, time, or location.
To see projects currently using Tator in Chrome, Brave, or Microsoft Edge, log in with username guest and password mbariguest at:
http://mantis.shore.mbari.org/accounts/login
🎥 Video / Time-lapse¶
-
🏷️ Tator – Advanced annotation database
Great for labeling video and managing your machine learning datasets. It has flexible tools designed specifically for time-lapse and video media. -
RF-DETR – State-of-the-art object detection model
Use this for super-accurate object detection in images or videos. It actually beats YOLOv11 by about 30% on our MBARI datasets.
Example processed video:
-
☁️
DeepSea AI – Cloud-based detection and tracking
Train and run object detection and tracking at scale in the AWS Cloud for as little as $1.50 per video. You can train models using FathomNet or your own custom datasets with up to 8 GPUs per job. It’s perfect for heavy-duty video processing. Check out the DeepSea AI docs for more info.
🐋 🎵 Audio Analysis¶
AWS Pacific Sound Data¶
Examples of using AI to classify sounds in MARS audio data:
API Services¶
- Tator REST API - Use this for operations on the Tator database. You will need an account to use this.
- FastAPI Tator – Use this for bulk operations (like updating large clusters) on the Tator database.
- FastAPI VSS – Our API for vector similarity searches.
- FastAPI YOLOv5 – Run YOLOv5 object detection locally or scale it in the cloud (AWS) on demand. (Currently not in active use, but available if you need it).
- FastAPI kclassify – An API for image classification using Keras models. (Currently not in active use, but available if you need it).
🗓️ Updated: 2026-02=08