Skip to content

Database/GraphQL API/Worker Development Setup

For software development, it is recommended that you start a postgres database in a docker image and modify other services outside of the docker stack using that database.

Install external dependencies FFMPEG and Psql

You can skip this step if you are not working on anything related to thumbnail generation/crop workers or the database views

  • FFMPEG is used to generate thumbnails of job output for quick preview in the deepsea-ai-frontend webapp
  • Psql is used to insert the views in the database

For Mac OSX

brew install libpq ffmpeg
export PATH=$PATH:/opt/homebrew//Cellar/libpq/14.3/bin/:/opt/homebrew/bin/

1 - Check-out the code

git clone https://github.com/mbari-org/deepsea-ai-backend.git

2 - Install the required node packages

Install node packages for initializing the database and running the API with:

cd deepsea-ai-backend
yarn install

3 - Configure passwords and database location

Copy the example environment configuration files and change the passwords. Note that the password in the .pgpass must match the DEEPSEA_AI_DB_PASSWORD environment variable in .env

cp example.env .env
cp example.pgpass $HOME/.pgpass
chmod 0600 $HOME/.pgpass

4 - Run postgres docker container

./bin/docker_start.sh postgres database

Stopping can be done with 

./bin/docker_stop.sh postgres database

The output of this may be long the first time, but you should see a line near the end when successfully created similar to:

Image link

Now postgres should be running in a docker image called deepsea_ai_pg

5 - Initialize the database and start the database API

Now that you have postgres docker image ready, run the postgres_init.sh script to push the schema tables to the database, seed it and start-up the API. This only needs to be done once, unless changes are made to the schema.

 export DEEPSEA_AI_DB_URL=postgres://postgres:fangtooth@localhost:5433/deepsea-ai-dev 
 ./bin/postgres_init.sh

There is now a node_modules/@prisma/client* which can be imported and used in the code.

Let's run the app in development mode now and open it at http://localhost:4000/graphql

6 - Test loading

To test loading into the database, first get an authentication token through the graphql api http://localhost:4000/graphql, e.g.

mutation loginadmin {
  login(email: "admin@deepsea-ai.org", password: "gulpereel") {
    is_admin
    token
  }
}

This should return the long token in the field "token", e.g. 

{"data":{"login":{"token":"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VySWQiOjEsImlhdCI6MTY1NjI4MzAzMSwiZXhwIjoxNjU2ODg3ODMxfQ.7rlILdHjOQMn5WqJsjnE4WssfNu582rQEn1r9M26j7U"}}}

Use that token to load test data 

./bin/run_load.sh eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VySWQiOjEsImlhdCI6MTY1NTA5NTM4NCwiZXhwIjoxNjU1NzAwMTg0fQ.-EQ6kPBjwXSijyRINNlQ5E_GciZYL2lZHcDnpMZZlYU"

note

If you see Error: P1001: Can't reach database server at localhost:5433 this may be because the database is not up yet. Wait a few seconds then try again.

(optional) Browse the database using Prisma Studio

Prisma provides a nice user interface to see and modify the data called Prisma Studio. Launch with:

yarn studio

then open Prisma Studio at http://localhost:5555

This is useful to manually browse the tables and delete data.

That's it!

Where is the database data stored?

All database data is in the container volume database_deepsea_ai_db

If you want to override this, add POSTGRES_DATA in the docker-compose.yml.postgres and .env files to mount the data to a local directory.

Warning

Be sure the POSTGRES_DATA environment variable set in your .env points to a directory that can be mounted by bind mounted into Docker, and it must be an absolute path.

Check-out the example graphql queries and mutations to explore in the playground.

References