Updating FIDO Firmware and Parameter Files¶
FIDO's parameter files (configurations.py, calibrations.py and protocols.py on its /flash drive) are checked at boot by a validator that is built into the firmware. When a firmware release adds, removes or changes parameters, the files on an instrument stop passing that check, and FIDO refuses to run until they are fixed.
The fido-update tool does both jobs in one sitting on the RockPi: it flashes the new firmware and rewrites the three parameter files so they pass the new firmware's validator, keeping each instrument's own values. It lives in the esp-sampler repository under tools/param-upgrade/ and is delivered as a single bundle per firmware release.
Warning
Firmware and parameter files must change together. Old firmware rejects the new files and new firmware rejects the old ones, so FIDO will not run between the two steps. Always use the bundle, which does both, rather than flashing firmware on its own.
What is in a bundle¶
A bundle is a file named fido-update-v<version>.tar.gz, built from one tagged esp-sampler commit. It unpacks to a folder of the same name containing:
| File | Purpose |
|---|---|
run |
The program the operator runs on the RockPi |
fido-v<version>.elf |
The firmware |
base/ |
The parameter base files and validator.py frozen into that firmware |
param_upgrade.py |
Rewrites the parameter files against base/ |
VERSION |
The release version, e.g. v1.0.0 |
*.hex |
Optional. A pump controller program, included only if the bundle was built with one |
The version is the esp-sampler git tag. The firmware build stamps it into FIDO's banner as Project: v1.0.0 (<hash>), which fido.version() prints, and run checks it on the instrument after flashing.
Building a bundle¶
Done once per release by a developer, in the WSL copy of esp-sampler. See the esp-sampler README.md for the firmware build itself and tools/param-upgrade/README.md for more detail.
Release bundle¶
- Make sure
mainis up to date and is what you want to release:git checkout mainandgit pull. - Tag the commit before building, since the tag is read at build time:
git tag v1.0.0 - Build the firmware as usual (
make ... BOARD=FIDO). The tree, including themicropythonsubmodule, must have no uncommitted changes, or the banner will readv1.0.0-dirty. - Check the banner:
strings micropython/ports/stm32/build-FIDO/firmware.elf | grep "Project:"should showProject: v1.0.0 (...). - Run
tools/param-upgrade/make-bundle.sh. It refuses to build unless HEAD is taggedvX.Y.Z, the tree is clean and the firmware banner matches. The bundle is written totools/param-upgrade/dist/fido-update-v1.0.0.tar.gz. - Push the tag (
git push origin v1.0.0) and attach the bundle to a GitHub release for that tag inmbari-org/esp-sampler.
Test bundle¶
For trying a build on instruments before a release, use a local release-candidate tag that is never pushed:
git tag v1.0.0-rc.1(userc.2,rc.3, ... for later rounds, each on the commit being built).- Build the firmware.
- Run
tools/param-upgrade/make-bundle.sh --dev. This accepts rc tags, untagged and uncommitted trees, and names the bundlefido-update-v1.0.0-rc.1-dev.tar.gz. - When testing is finished, delete the rc tags:
git tag -d v1.0.0-rc.1.
Note
A plain git push never sends tags, so rc tags stay local unless someone runs git push --tags. Deleting them afterwards avoids that and keeps them out of later version banners.
Including a pump controller program¶
Add --pump-hex <file.hex> to either make-bundle.sh command to include a RoboRun+ compiled pump controller program. It is optional, and the pump controller is versioned separately from the firmware.
Before you start on an instrument¶
- STLINK programmer, connected to the RockPi's USB-C port when asked. The STLINK and the FIDO serial cable cannot both be connected, so you will swap them twice during the run.
- SSH access to the RockPi as root, either over the instrument's WiFi hotspot (
root@10.42.0.1) or through the cloud server (ssh -J admin@prod.readinet.org root@fido-00X). - No sample running.
runchecks that FIDO is idle and stops if it is not, but do not start an update while an event is scheduled to run.
1. Copy the bundle to the RockPi¶
Note
If the SD card image was customized with set-device-vars.sh --bundle (see Add a New Device), the bundle is already on the RockPi in /root. Skip this step, and in step 2 use cd /root instead of cd /tmp. Everything after that is the same, with /root/fido-update-v1.0.0/run in place of /tmp/fido-update-v1.0.0/run.
Download the bundle from the GitHub release, then copy it into /tmp on the RockPi:
scp fido-update-v1.0.0.tar.gz root@10.42.0.1:/tmp/
Through the cloud server, add the jump host:
scp -J admin@prod.readinet.org fido-update-v1.0.0.tar.gz root@fido-00X:/tmp/
2. Unpack it¶
ssh -t root@10.42.0.1
cd /tmp
tar xzf fido-update-v1.0.0.tar.gz
This creates /tmp/fido-update-v1.0.0/. Nothing is installed on the RockPi; the folder and the .tar.gz can be deleted once the update is done.
Note
Use ssh -t (or an interactive ssh session) so run can ask questions.
3. Dry run (recommended)¶
/tmp/fido-update-v1.0.0/run --dry-run
The dry run does everything up to the point of changing FIDO, then stops:
- Stops
readinet-web(if it is running) to free the serial port, sets the port to 115200 baud, checks FIDO is idle at its>>>prompt and runsimport utils. - Pulls the three parameter files from FIDO with
getfromfido. - Upgrades them and prints a report for each file.
- Restarts
readinet-webif it was running.
Read the report. For each file it lists:
- added from base defaults: parameters the new firmware needs that were missing, filled in with the release's default values.
- removed: parameters the new firmware no longer uses, dropped from the file.
- reset to base defaults: vacuum settings that were still in ADC counts, replaced because the firmware now uses hPa.
- evaluated: values written as arithmetic, such as
(41.0/4096.0), replaced by a single number (computed decimals are rounded to 5 significant figures). - reformatted: values rewritten in the form the validator accepts, e.g.
500_000to500000, or a dictionary spread over several lines put on one line. - set by operator / placeholders left in place: see the prompts below.
- validation: pass, or the errors that would stop it.
The upgraded files are rebuilt in the order and with the comments of the release's base files; comments the instrument's files had are not kept. Values are the instrument's own unless listed above.
Prompts¶
The upgrader stops and asks when it cannot decide safely:
- A calibration value is missing. Most calibrations are unique to each instrument, so there is no default. Enter the value; it is checked against the allowed range and asked again if invalid. The
HPA_*vacuum sensor calibrations are the same on every instrument, so those are filled in without asking. - A value is out of range or the wrong type. Enter a valid value.
- A placeholder is still set, such as
FORCESN = "00"(force sensor serial number). These defaults are deliberately invalid so FIDO will not run until they are set. Enter the real value, or press Enter to keep the placeholder and set it later.
Each prompt shows the current value and what is allowed, for example allowed: int from -500 to 500.
4. Full update¶
/tmp/fido-update-v1.0.0/run
The full run repeats the dry run's steps, then continues:
- After the report, it asks
Flash firmware fido-v1.0.0.elf and push the upgraded files? [y/N]. Answering anything butystops here with FIDO unchanged. - Swap to the STLINK. When asked, unplug the FIDO serial cable from the RockPi, plug the STLINK into the RockPi's USB-C port, and press Enter.
runflashes and verifies the firmware with openocd. If flashing fails you can retry or abort. - Swap back to serial. When asked, unplug the STLINK, plug the FIDO serial cable back in, and press Enter.
runwaits for the port, checks FIDO is idle and runsimport utilsagain. - Pushes each upgraded file with
sendtofido, reads it back withgetfromfidoand checks it matches what was sent. If the bundle includes a pump controller program, it is copied to/flash/roboteq.hexthe same way. - Soft-reboots FIDO so it validates the new files at boot, then reports the result.
- Restarts
readinet-webif it was running whenrunstarted.
Reading the result¶
A successful update ends with:
### Result
firmware: v1.0.0 (as expected)
parameter files: accepted
Other outcomes:
firmware: ... - EXPECTED v1.0.0: the flash did not take. Run the update again.parameter files: REJECTED by the firmware validator: followed by the validator'sCONFIGlines. Lines marked(placeholder kept by operator)are expected if you pressed Enter at a placeholder prompt; set those values on FIDO and reset it. Any other line means the files need attention; the files are in the work folder (below).
If something goes wrong¶
FIDO looks busy (sample running?): FIDO was sending output when run started. Nothing was changed. Let the operation finish and run again.
Stopped before flashing (validation failure, a n at the confirmation, Ctrl-C): FIDO has not been changed. Fix the cause and run again.
Stopped after flashing started (flash failure, cable not reconnected, push or read-back failure): FIDO may have new firmware with old parameter files and will not run. run prints the command to finish:
/tmp/fido-update-v1.0.0/run --resume /root/fido-update/<timestamp>
--resume skips pulling and upgrading and pushes the files already prepared in that work folder. It first asks whether to flash again; answer y if flashing had failed or was interrupted.
Rolling back: the files FIDO had before the update are in the work folder's original/. They only pass the validator of the firmware they came from, so rolling back means flashing the previous firmware (flash <previous .elf> on the RockPi, with the STLINK connected) and then pushing the originals with sendtofido after import utils at FIDO's REPL.
Files kept on the RockPi¶
Every run creates /root/fido-update/<timestamp>/ on the RockPi:
| Path | Contents |
|---|---|
original/ |
The three files as pulled from FIDO before the update |
upgraded/ |
The files run generated (and pushed, for a full run) |
readback/ |
The files read back from FIDO after the push |
run.log |
Everything run printed, including operator answers |
To review a run on your own machine, copy the folder down:
scp -r root@10.42.0.1:fido-update/<timestamp> .
After the update¶
run restarts readinet-web only if it was running when the update began. If it was not running, it is left off.
Warning
Stopping readinet-web also stops readinet-network and readinet-install-schedule, which depend on it, and run does not start those again. If readinet-web was running before the update, reboot the RockPi afterwards (or start those services with systemctl start readinet-network readinet-install-schedule) to return it to its normal state.
On its next start, readinet-web runs fido.init() as usual. With the new firmware, fido.version() prints Project: v1.0.0 in its banner and shows frozen for each built-in module.
If the bundle included a pump controller program, run only copied it to FIDO and says so at the end. Programming the controller is a separate step: on an initialized FIDO, run samplePump.uploadRoboteqHex(), which loads /flash/roboteq.hex.