Installation

Setup AWS CLI version 2

Installation depends on your OS, e.g. for Mac OS

$ curl "https://awscli.amazonaws.com/AWSCLIV2.pkg" -o "AWSCLIV2.pkg"
$ sudo installer -pkg AWSCLIV2.pkg -target /

Setup your AWS keys

The fastest way to setup AWS to use the AWS CLI is to use the configure command, e.g.

$ aws configure
AWS Access Key ID: MYACCESSKEY
AWS Secret Access Key: MYSECRETKEY
Default region name [us-west-2]: us-west-2
Default output format [None]: json

Build the AWS Elastic Container Registry (ECR) image

A docker image must reside the ECR before using this with your BLED batch files. To do this run the build_and_push_aws.sh script. This is only needed once, unless a revision has been made the code. This will build and push the generated docker image to the ECR. In production, this is automatically run when a new release is made so there is no need to run this manually, unless you are testing it.

The AWS URI is 872338704006.dkr.ecr.us-west-2.amazonaws.com/soundscape-bled-$version . This is the unique identifier for the docker image that should be reference correctly in the config.yml and job definition files: blue-d-job.json, and blue-d-job.json files.

You can manually build and push the docker image to the ECR with

$ git clone https://bitbucket.org/mbari/soundscape-BLED-preprocess.git
$ cd soundscape-BLED-preprocess/docker
$ build_and_push_aws.sh <version>

e.g.

$ build_and_push_aws.sh 1.0.0

Deploy the Cloud Stack

cd cdk && cdk deploy

Should see something like the following

[Warning at /SoundscapeBLEDBatchFargateStack/Soundscape-blue-a-job-def-pub] Proper policies need to be attached before pulling from ECR repository, or use 'fromEcrRepository'.
[Warning at /SoundscapeBLEDBatchFargateStack/Soundscape-blue-a-job-def-pub/Resource-Batch-Job-Container-Definition] Proper policies need to be attached before pulling from ECR repository, or use 'fromEcrRepository'.
[Warning at /SoundscapeBLEDBatchFargateStack/Soundscape-blue-d-job-def-pub] Proper policies need to be attached before pulling from ECR repository, or use 'fromEcrRepository'.
[Warning at /SoundscapeBLEDBatchFargateStack/Soundscape-blue-d-job-def-pub/Resource-Batch-Job-Container-Definition] Proper policies need to be attached before pulling from ECR repository, or use 'fromEcrRepository'.

✨  Synthesis time: 4.05s

This deployment will make potentially sensitive changes according to your current security approval level (--require-approval broadening).
Please confirm you intend to make the following modifications:

IAM Statement Changes
┌───┬──────────────────────────────────────────────────────────────────┬────────┬────────────────┬─────────────────────────────────┬───────────┐
│   │ Resource                                                         │ Effect │ Action         │ Principal                       │ Condition │
├───┼──────────────────────────────────────────────────────────────────┼────────┼────────────────┼─────────────────────────────────┼───────────┤
│ + │ ${SoundscapeFargateBLEDBatch/Resource-Service-Instance-Role.Arn} │ Allow  │ sts:AssumeRole │ Service:batch.amazonaws.com     │           │
├───┼──────────────────────────────────────────────────────────────────┼────────┼────────────────┼─────────────────────────────────┼───────────┤
│ + │ ${Soundscape-ecsTaskExecutionRole.Arn}                           │ Allow  │ sts:AssumeRole │ Service:ecs-tasks.amazonaws.com │           │
└───┴──────────────────────────────────────────────────────────────────┴────────┴────────────────┴─────────────────────────────────┴───────────┘
IAM Policy Changes
┌───┬──────────────────────────────────────────────────────────────┬─────────────────────────────────────────────────────────────────────────────────────┐
│   │ Resource                                                     │ Managed Policy ARN                                                                  │
├───┼──────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────┤
│ + │ ${SoundscapeFargateBLEDBatch/Resource-Service-Instance-Role} │ arn:${AWS::Partition}:iam::aws:policy/service-role/AWSBatchServiceRole              │
├───┼──────────────────────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────┤
│ + │ ${Soundscape-ecsTaskExecutionRole}                           │ arn:${AWS::Partition}:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy │
└───┴──────────────────────────────────────────────────────────────┴─────────────────────────────────────────────────────────────────────────────────────┘
Security Group Changes
┌───┬───────────────────────────────────────────────────────────────┬─────┬────────────┬─────────────────┐
│   │ Group                                                         │ Dir │ Protocol   │ Peer            │
├───┼───────────────────────────────────────────────────────────────┼─────┼────────────┼─────────────────┤
│ + │ ${SoundscapeFargateBLEDBatch/Resource-Security-Group.GroupId} │ Out │ Everything │ Everyone (IPv4) │
└───┴───────────────────────────────────────────────────────────────┴─────┴────────────┴─────────────────┘

When the stack is complete

 ✅  SoundscapeBLEDBatchFargateStack

✨  Deployment time: 71.58s

Outputs:
SoundscapeBLEDBatchFargateStack.SoundscapeblueaBatchJobQueue = arn:aws:batch:us-west-2:872338704006:job-queue/blue-a-preprocess-job-queue
SoundscapeBLEDBatchFargateStack.SoundscapeblueaJobDefinition = arn:aws:batch:us-west-2:872338704006:job-definition/blue-a-preprocess-job-definition-public:7
SoundscapeBLEDBatchFargateStack.SoundscapebluedBatchJobQueue = arn:aws:batch:us-west-2:872338704006:job-queue/blue-d-preprocess-job-queue
SoundscapeBLEDBatchFargateStack.SoundscapebluedJobDefinition = arn:aws:batch:us-west-2:872338704006:job-definition/blue-d-preprocess-job-definition-public:5
Stack ARN:
arn:aws:cloudformation:us-west-2:872338704006:stack/SoundscapeBLEDBatchFargateStack/eeed2230-8ed1-11ec-950d-029da3f321af