Metadata-Version: 2.4 Name: microbasic-compiler Version: 0.1.0 Summary: Compiler toolchain for Roboteq MicroBasic scripts, targeting Intel HEX bytecode for SDC2160-family drives License: MIT Requires-Python: >=3.10 Description-Content-Type: text/markdown Provides-Extra: dev Requires-Dist: pytest>=7.0; extra == "dev" # microbasic-compiler A from-scratch Python compiler for Roboteq's MicroBasic scripting language, targeting Intel HEX bytecode files loadable onto SDC2160-family drives. ## Pipeline ``` source.mb -> lexer -> #define expansion -> parser -> AST -> codegen -> bytecode -> hexfile -> script.hex ``` | Stage | File | Status | |---|---|---| | Lexer | `src/microbasic_compiler/lexer.py` | Implemented | | Preprocessor | `src/microbasic_compiler/preprocessor.py` | Implemented -- `#define` macro expansion | | AST | `src/microbasic_compiler/ast_nodes.py` | Implemented | | Parser | `src/microbasic_compiler/parser.py` | Implemented -- full grammar, precedence confirmed against real compiled output | | Codegen | `src/microbasic_compiler/codegen.py` | Implemented -- see caveats below | | Command codes | `src/microbasic_compiler/commands.py` | 317 constants -- 10 bytecode-confirmed, the rest sourced from the official Roboteq Controllers manual (cross-checked against all 10 confirmed values, zero mismatches); anything not in the table raises a clear error | | Intel HEX I/O | `src/microbasic_compiler/hexfile.py` | Implemented -- standard format, read + write | | CLI | `src/microbasic_compiler/cli.py` | Implemented (`mbc build script.mb -o script.hex`) | **The compiler works, and now passes the real end-to-end test: all 8 actual production `PumpController*.mbs` scripts compile byte-for-byte identical to their real RoborunPlus `.hex` output** (`tests/test_real_scripts.py`). `tests/test_codegen.py` additionally compiles 40 fixture programs (a representative slice of the full reverse-engineering corpus, `tests/fixtures/`) against the exact `.hex` RoborunPlus produced for each -- not hand-written expectations, ground truth captured from the real compiler. 73/73 tests pass total. ```bash pip install -e ".[dev]" pytest mbc build examples/blink.mb -o examples/blink.hex ``` ## The bytecode format is not publicly documented Roboteq publishes the MicroBasic *language* grammar in full, but not the bytecode it compiles to -- no opcode table exists anywhere in their materials. Everything in `codegen.py` was reverse-engineered by diffing real `.hex` output from a purpose-built corpus of ~130 test programs (6 batches, plus 8 real production scripts), decoding every byte with zero leftovers. Full evidence trail: [`docs/reverse_engineering.md`](docs/reverse_engineering.md). Consequences of that: - Every opcode used in `codegen.py` is backed by a specific decoded example in that document. - A handful of things are reasonable **extrapolations beyond what was directly confirmed** -- e.g. `Continue`'s exact jump target inside a `For` loop, or `GetValue`/`GetConfig`'s 1-argument form defaulting Index to 0 (confirmed for `SetCommand`'s equivalent case, assumed symmetric for these by analogy -- see batch 6 in `reverse_engineering.md`). These are marked `# ASSUMPTION` in `codegen.py` and listed in `reverse_engineering.md`'s "still open" sections. None of them are things a normal script is likely to hit, but if compiled output ever fails on real hardware in a way the test suite doesn't catch, start there. - `commands.py`'s command-code table now covers essentially every command, runtime query, and config parameter documented in the official *Roboteq Controllers User Manual* (V2.1) -- 317 constants, up from the 10 that had been individually reverse-engineered. All 10 bytecode-confirmed values match the manual's `HexCode:` field exactly, which is why the rest of the manual's codes are trusted even though they haven't each been diffed against a compiled `.hex` file. Extracting the table also caught 3 copy-paste typos in the manual itself and found that a couple of aliases (`_BRUN`, `_CSS`) mean different things depending on whether they're used with `SetCommand` vs `GetValue`/`SetConfig` -- see `commands.py`'s docstring and the "Full command-code table" section in `reverse_engineering.md` for the full story. Using an unrecognized `_XXX` constant still raises a clear error rather than guessing. ## Language reference Source: *MicroBasic Scripting Language User & Reference Manual*, V2.0, July 8 2019 (Roboteq). Key facts baked into the lexer/parser: - Comments start with `'` and run to end of line. - Numbers: decimal (`120`), hex (`0xA1`), binary (`0b101`); range is signed 32-bit. - Strings: double-quoted, with `\n \t \r \a \b \f \v \' \" \\` and `\xHH` escapes. No string variables -- strings are only usable with `Print`. - Variables: `Integer` or `Boolean` only (no floats), plus 1-D arrays of either, in **separate slot-numbering spaces** (confirmed). `Dim` required for `Boolean`/arrays; `Integer` falls back to implicit declaration unless `Option Explicit` is used (that directive is currently accepted but not enforced -- see TODO in `codegen.py`). - Labels: identifier at start of line + `:`, target of `GoTo`/`GoSub`/`Return`. - Operator precedence (confirmed by decoding real output, not just assumed): `Or`/`XOr` > `And` > `Not` > relational (`= <> < > <= >=`) > shift (`<< >>`) > additive (`+ -`) > multiplicative (`* / Mod`) > unary `-` > postfix `++`/`--`. Unary `+` is **not supported** -- RoborunPlus itself rejects it. - Compound assignment (`+= -= *= /= <<= >>=`) desugars to plain `=` at parse time -- confirmed to compile identically to the expanded form. - Device I/O: `GetConfig`, `SetConfig`, `GetValue`, `SetCommand`, four timer functions, five math functions (`Abs`/`Atan`/`Cos`/`Sin`/`Sqrt`), `ToBool`. - All four loop shapes (`While`/`End While`, `Do While`/`Loop`, `Do Until`/`Loop`, `Do`/`Loop While`, `Do`/`Loop Until`), both `For` styles (traditional and C-style `AndWhile`), `Exit`/`Continue`. ## Getting started ```bash pip install -e ".[dev]" pytest mbc build examples/blink.mb -o examples/blink.hex ``` ## Status: all 8 real production scripts compile byte-for-byte Batch 6 closed the last known real-world gap (the 8 command codes needed by the real scripts), which also caught and fixed a real bug in `SetCommand`'s 2-argument defaulting behavior, and diffing against `PumpcontrollerR0-3-LowCurrentPressure.mbs` caught a second bug: a synthesized `Wait(1)` between split `Print` string chunks that wasn't previously implemented. With both fixed, all 8 real `PumpController*.mbs` scripts compile identically to their actual `.hex` files (`tests/test_real_scripts.py`). On top of that, the official Roboteq Controllers manual turned out to document the full command-code table, expanding `commands.py` from 10 to 317 confirmed constants (see above) -- 73/73 tests pass overall. ## Next steps 1. The remaining `# ASSUMPTION` items in `codegen.py` could each be closed the same way -- a small test script, diffed against real output. None of them block the real scripts from compiling; they're edge cases not yet exercised (e.g. `GetValue`'s 1-arg defaulting, array base-slot allocation when an array isn't the last declaration, `Continue For`'s exact jump target). 2. `Option Explicit` enforcement (declare-before-use checking) is a TODO, not currently implemented.