Redis vss

🚀 REDIS is used to support fast vector search. This ⚡️accelerates machine learning workflows that require vector similarity search using foundational or fine-tuned models. Each project or project system has its own Redis instance, which supports isolation of the data.

To add a new Redis instance:

1. Add support for a new volume, Redis service, and port to the compose.yml file in the deploy-ai repository: playbooks/files/fastapi/vss.

For example, with a new name and port, e.g. redis-sinker on port 6383:

services:
    fastapi:
    image: mbari/fastapi-vss:${GIT_VERSION}
    container_name: fastapi-vss
    healthcheck:
      test: [ "CMD", "curl", "-f", "http://127.0.0.1:8000/health" ]
    env_file:
      - .env.prod
    ports:
      - "8000:80"
    volumes:
      - /opt/deployment:/opt/deployment
    user: ${MLDEVOPS_GID}:${MLDEVOPS_UID}
    restart: always
    depends_on:
      - redis-uav
      - redis-cfe
      - redis-bio
      - redis-i2map
      - redis-sinker , 👈 add this line
    redis-planktivore: ,  👈 add this block
        env_file:
          - .env.prod
        command:
          - /bin/sh
          - -c
          - redis-stack-server  --port 6383 --appendonly yes --appendfsync everysec --requirepass "$${REDIS_PASSWD:?REDIS_PASSWD variable is not set}"
        image: redis/redis-stack-server
        container_name: redis-sinker
        ports:
          - "6383:6383"
        volumes:
          - redis-sinker-data:/var/lib/redis-stack
        restart: always
        networks:
          - vss
volumes:
  redis-uav-data:
  redis-cfe-data:
  redis-bio-data:
  redis-i2map-data:
  redis-sinker-data:  👈 add this line
networks:
  vss:

1. Create a new config file in the config directory in the aidata repository, e.g. config_planktivore.yml

2. Initialize the database with the aidata command db resest with that config file, e.g.

   aidata db reset --config aidata/config/config_planktivore_lm.yml --redis-password <password>
   

Tip

Logs can be seen on the devops mount at smb://thalassa.shore.mbari.org/DevOps This can be useful for debugging issues with services in docker images.

🗓️ Updated: 2025-07-07