Skip to content

Building the SD Card Image

When changes are made to the codebase that affect the software that runs on the RockPi, you will need to generate a new SD Card image that will be stored on an S3 bucket in the cloud. These images are available for end users to install in their RockPis. Here is an example of how to do this.

Example Build Process

  1. For this example, I made changes to the code when the sampler serial protocol changed from esp.* to fido.*. I changed all the files in the web application code to issue the fido commands. Running a git status gives me the following

    $ git status
On branch main
Your branch is up to date with 'origin/main'.

Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git restore <file>..." to discard changes in working directory)
modified:   web/readinet/api.py
modified:   web/readinet/background.py
modified:   web/readinet/scripts/scheduled_events.py

no changes added to commit (use "git add" and/or "git commit -a")

  2. Once you feel confident about the changes you have made, you need to push those changes into the GitHub repo before running the AWS SD image build because the build checks out the latest repo during the build.

Warning

Before you run the AWS build, you need to make sure that you have the AWS CLI set up and that you have a profile that has permissions to run everything that the build script will run. This is not documented here, but make sure that after you configure the CLI, you update the build-aws.sh file and set the following variables to match your AWS configuration:

declare AWS_PROFILE=xxxxxx
declare AWS_REGION=xxxxxxxxxxxx
declare BUILD_SECURITY_GROUP=sg-xxxxxxxxxxxxxxxxxxx
declare SUBNET=subnet-xxxxxxxxxxxxxxxxxxxx
declare INSTANCE_PROFILE=xxxxxxxxxxxxxxx

  1. While this isn't necessary, you probably want to be notified when the SD card image build is complete. It does take quite some time so it's helpful to be notified of it's completion. There is an SNS (Simple Notification Service) just for this purpose. If you want be notified, go to the topic page and click on the Create subscription button. SNS Step 1

  2. Then choose 'Email' and enter your email address, then click create subscription. SNS Step 2

  3. You will need to confirm an email that gets sent to your inbox before you will start getting notifications (check spam if you don't get it).

  4. Before building the SD card image on AWS, we need to find the next version number. If you go to the S3 Bucket that contains the SD card images, you can find the most recent version number in the file name of the image (reverse sort by last modified usually works). In the example below, the most recent version number is 28. SD Image Listing

  5. This means you will want to generate the next version number which is 29 in this example.

Note

Note that whatever you put as the version number will be the last part of the filename. For this example, we included a -dev to make it clear that it's a development version.

  1. Now you can run the AWS build, but running build-aws.sh 29-dev

Note

If you are building on a Mac, you need to make sure you have gsed installed which I did using homebrew

  1. The AWS build fires off an EC2 instance and then runs the commands that are in the aws-userdata.sh file which builds an image and deploys it to S3. The output of the command should look something like this:

    + declare VERSION_NUMBER=29-dev
+ declare AWS_PROFILE=mbari
+ declare AWS_REGION=us-west-2
+ declare BUILD_SECURITY_GROUP=sg-04dd09170d092d1a0
+ declare SUBNET=subnet-056123e8157180bc3
+ declare INSTANCE_PROFILE=build_server
+ declare 'AWS_CALL=aws --region us-west-2 --profile mbari --no-cli-pager'
++ aws --region us-west-2 --profile mbari --no-cli-pager ec2 describe-images --owners amazon --filters 'Name=name,Values=debian-12-amd64-*' --query 'sort_by(Images, &CreationDate)[-1].ImageId'
++ tr -d '"'
+ declare AMI_ID=ami-08a1cc220ff6e579e
++ mktemp
+ declare USERDATA=/var/folders/bn/d339bs_n75gbwnprm807_9gr0000gn/T/tmp.lzYc1EQLry
+ cp aws-userdata.sh /var/folders/bn/d339bs_n75gbwnprm807_9gr0000gn/T/tmp.lzYc1EQLry
+ declare sedcmd=sed
+ [[ darwin23 == \d\a\r\w\i\n* ]]
+ sedcmd=gsed
+ gsed -i s/__VERSION_PLACEHOLDER__/29-dev/g /var/folders/bn/d339bs_n75gbwnprm807_9gr0000gn/T/tmp.lzYc1EQLry
+ aws --region us-west-2 --profile mbari --no-cli-pager ec2 run-instances --image-id ami-08a1cc220ff6e579e --count 1 --instance-type t3.xlarge --security-group-ids sg-04dd09170d092d1a0 --iam-instance-profile Name=build_server --instance-market-options MarketType=spot --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=build-29-dev}]' --block-device-mapping '[ { "DeviceName": "/dev/xvda", "Ebs": { "VolumeSize": 25 } } ]' --instance-initiated-shutdown-behavior terminate --user-data file:///var/folders/bn/d339bs_n75gbwnprm807_9gr0000gn/T/tmp.lzYc1EQLry
{
"Groups": [],
"Instances": [
        {
        "AmiLaunchIndex": 0,
        "ImageId": "ami-08a1cc220ff6e579e",
        "InstanceId": "i-066c36b248d2426aa",
        "InstanceType": "t3.xlarge",
        "LaunchTime": "2024-09-25T23:28:40+00:00",
        "Monitoring": {
                "State": "disabled"
        },
        "Placement": {
                "AvailabilityZone": "us-west-2a",
                "GroupName": "",
                "Tenancy": "default"
        },
        "PrivateDnsName": "ip-172-31-24-230.us-west-2.compute.internal",
        "PrivateIpAddress": "172.31.24.230",
        "ProductCodes": [],
        "PublicDnsName": "",
        "State": {
                "Code": 0,
                "Name": "pending"
        },
        "StateTransitionReason": "",
        "SubnetId": "subnet-0de09a4754b1dbd1d",
        "VpcId": "vpc-02ad470bcf2b29a3c",
        "Architecture": "x86_64",
        "BlockDeviceMappings": [],
        "ClientToken": "b228d672-48c4-403e-85a0-faf999232cec",
        "EbsOptimized": false,
        "EnaSupport": true,
        "Hypervisor": "xen",
        "IamInstanceProfile": {
                "Arn": "arn:aws:iam::414844587818:instance-profile/build_server",
                "Id": "AIPAWBFVKB4VDCFY6W7MH"
        },
        "InstanceLifecycle": "spot",
        "NetworkInterfaces": [
                {
                "Attachment": {
                        "AttachTime": "2024-09-25T23:28:40+00:00",
                        "AttachmentId": "eni-attach-09407dab288496295",
                        "DeleteOnTermination": true,
                        "DeviceIndex": 0,
                        "Status": "attaching",
                        "NetworkCardIndex": 0
                },
                "Description": "",
                "Groups": [
                        {
                        "GroupName": "build-server",
                        "GroupId": "sg-04dd09170d092d1a0"
                        }
                ],
                "Ipv6Addresses": [],
                "MacAddress": "02:c4:ee:56:76:37",
                "NetworkInterfaceId": "eni-0431c1b6b1398bbd8",
                "OwnerId": "414844587818",
                "PrivateDnsName": "ip-172-31-24-230.us-west-2.compute.internal",
                "PrivateIpAddress": "172.31.24.230",
                "PrivateIpAddresses": [
                        {
                        "Primary": true,
                        "PrivateDnsName": "ip-172-31-24-230.us-west-2.compute.internal",
                        "PrivateIpAddress": "172.31.24.230"
                        }
                ],
                "SourceDestCheck": true,
                "Status": "in-use",
                "SubnetId": "subnet-0de09a4754b1dbd1d",
                "VpcId": "vpc-02ad470bcf2b29a3c",
                "InterfaceType": "interface"
                }
        ],
        "RootDeviceName": "/dev/xvda",
        "RootDeviceType": "ebs",
        "SecurityGroups": [
                {
                "GroupName": "build-server",
                "GroupId": "sg-04dd09170d092d1a0"
                }
        ],
        "SourceDestCheck": true,
        "SpotInstanceRequestId": "sir-was6yc5q",
        "StateReason": {
                "Code": "pending",
                "Message": "pending"
        },
        "Tags": [
                {
                "Key": "Name",
                "Value": "build-29-dev"
                }
        ],
        "VirtualizationType": "hvm",
        "CpuOptions": {
                "CoreCount": 2,
                "ThreadsPerCore": 2
        },
        "CapacityReservationSpecification": {
                "CapacityReservationPreference": "open"
        },
        "MetadataOptions": {
                "State": "pending",
                "HttpTokens": "optional",
                "HttpPutResponseHopLimit": 1,
                "HttpEndpoint": "enabled",
                "HttpProtocolIpv6": "disabled",
                "InstanceMetadataTags": "disabled"
        },
        "EnclaveOptions": {
                "Enabled": false
        },
        "PrivateDnsNameOptions": {
                "HostnameType": "ip-name",
                "EnableResourceNameDnsARecord": false,
                "EnableResourceNameDnsAAAARecord": false
        },
        "MaintenanceOptions": {
                "AutoRecovery": "default"
        },
        "CurrentInstanceBootMode": "legacy-bios"
        }
],
"OwnerId": "414844587818",
"ReservationId": "r-0ec10f768ccd791ce"
}

  2. If you go the AWS console and look at the list of EC2 instances, you should see one starting up that is named after the version you are building EC2 Build Instance

  3. If you registered for the SNS notification in the earlier steps, you will get an email once the build is complete. You can then go to the S3 bucket and you will see the new image built with your new version number New SD Image

To Change the Root SSH Password on the RockPi

Before building the image, you would edit line 14 of the os/userpatches/customize-image.sh file to change it before building a new SD card image using the instructions above.