Migrating XML scripts to TethysL

The conversion of a mission script to TethysL involves three main steps:

1. Automated conversion of the XML script to TethysL
2. Refinement of the generated TethysL file
3. File renaming and pushing the new TethysL script to the lrauv-mission repository

Note

Strictly speaking, you can also manually write the TethysL version from "scratch", a perfectly fine approach especially for small missions.

To perform these steps we recommend using VS Code with the TethysL Extension, but you can also use the TethysL command-line program directly along with some text editor and a git client.

The following steps assume that the lrauv-mission repository (or a fork) has been cloned and that the lrauv-mission directory has been loaded as a workspace in VS Code.

Here's a screencast of a VS Code session covering some initial steps with an XML file that happens to expose a couple of easy-to-fix structural issues.

In more detail, with Maintenance/DUSBL.xml as an example, the steps are as follows:

1. Automated conversion from XML to TethysL

With Maintenance/DUSBL.xml open in VS Code, run the command:

TethysL: Generate TethysL (.xml.tl) from this XML

which will generate Maintenance/DUSBL.xml.tl.

Note

Unless a given foo.xml file is pretty badly constructed, the command should generate foo.xml.tl.

The .xml.tl file extension at this point simply indicates the TethysL version has been automatically generated from the .xml. We will be renaming this file to a simple .tl extension in a later step.

Important

The XML parser in TethysL is pretty lenient especially in terms of the original structure: it basically performs a direct transliteration of the given XML elements into corresponding TethysL constructs.

Two key aspects to keep in mind:

1. Depending on how compliant the given XML file is to the Tethys XML Schema, our subsequent compile step below on the generated TethysL script may expose issues with the original XML, for example, involving missing or out-of-order constructs, and other structural or syntax errors.

2. Pay special attention to the translation of expressions involving binary operations (<Or>, <And>, <Ge>, <Add>, <Mult>, etc.) to verify the intended execution semantics is preserved. As of TethysL 2.9.0, this should be taken care of by the xml-to-tethysl translator but we kindly ask you to double check. As a reminder, the LRAUV performs a flat-precedence, left-to-right evaluation of expressions involving binary operators, while TethysL follows the standard in most common programming languages (C/C++, Python, Java, etc.) regarding operator precedence and associativity. As needed or convenient, you can of course adjust any parenthesization on the resulting TethysL source toward a legible and less error-prone expression.

2. Refine generated .xml.tl

2.1. Compile / fix errors

The VS Code Extension automatically compiles any open .tl file. Fix any reported errors in the TethysL version until successful.

Note

By design, the TethysL compiler is very strict and more comprehensive.
If errors persist, don't hesitate to ask for help if you are not sure how to fix them.

A successful compile of a *.tl file not only validates the script, but also generates a corresponding XML file, which is named by setting .tx as the file extension. In our example, the generated XML file is Maintenance/DUSBL.tx.

What's this .tx file extension?

• The .tx file extension was designated to denote "TethysL Compiled XML" files, so one can easily tell them apart from the regular files originally authored in XML.
• .tx files are not to be pushed to the repository.

2.2. Compare

Compare the generated Maintenance/DUSBL.tx with the original Maintenance/DUSBL.xml to verify they both represent the same logical mission or aggregate as intended, that is, keeping in mind the files may be formatted differently.

You can proceed as follows:

1. Do a direct comparison using the compare command in VS Code, with two options:
   • Using regular VS Code mechanism:
     • select the two files in the Explorer view
     • right-click and select Compare Selected
   • Or, with the same effect, but more straightforwardly, run the TethysL extension command TethysL: Compare XML & TX, which will compare the file.xml and file.tx files corresponding to the currently opened document (any of file.tx, file.xml, file.xml.tl, file.tl).
2. If the different formatting of the two files makes the visual comparison challenging, you can use the TethysL: Reformat XML & TX and compare command, which will first reformat the two files prior to opening the diff view.

A typical diff view in VS Code looks like this:

Any difference in the namespace declarations?

Please disregard any differences in the namespace declarations because:

• In general, they are not well maintained in the original XML files
• TethysL automatically adds the correct namespace declarations (and only the ones actually needed)

When you are happy with the comparison you can complete this step by running the Format Document command on the .xml.tl file.

3. Update lrauv-mission

At this point, for the target mission that has been converted, you will have these three files, with only the original XML currently committed in the git repository:

File	Description
Maintenance/DUSBL.xml	Original XML script
Maintenance/DUSBL.xml.tl	TethysL conversion, including your adjustments (if any)
Maintenance/DUSBL.tx	"Compiled XML," as by-product of compiling the TethysL

Once you are happy with the resulting Maintenance/DUSBL.xml.tl, the final steps are to rename it, add it to the git repository, and remove the original XML file.

More concretely, assuming a pull-request workflow and a new branch in the origin remote for purposes of the PR:

• Rename the TethysL script to remove the .xml part:

  mv Maintenance/DUSBL.xml.tl Maintenance/DUSBL.tl
  

• Create a branch and check it out, e.g.:

  git checkout -b xml2tl/Maintenance/DUSBL
  

• Add the TethysL script to the git repository:

  git add Maintenance/DUSBL.tl
  

• Remove the XML script from the git repository:

  git rm Maintenance/DUSBL.xml
  

• You can remove the compiled XML file (no longer needed):

  rm Maintenance/DUSBL.tx
  

• Commit and push:

  git commit -m "Convert Maintenance/DUSBL to TethysL"
  git push origin xml2tl/Maintenance/DUSBL
  

• Create pull request in the remote repository.

That's it! 🎉

Additional notes

• While the TethysL compiler is pretty comprehensive in terms of syntax and semantic validation, unless you can verify exact logical equivalence of the generated .tx and the original .xml in the procedure above, there might be migration cases where the resulting TethysL script should also be tested against an LRAUV simulator or even a real vehicle. Please check with an LRAUV expert about such potential cases, probably as part of the description of your pull request.

• If your original XML contains <Insert>'s referring to .xml files, that's fine; both XML and TethysL insert's are allowed, so there is no need to first convert those.

• TethysL offers enhanced capabilities at the syntax level, including arrays and macros, which make it possible to express some mission scripts in a more concise way (in some cases, very significantly). Although the automated conversion from XML to TethysL (first step above) will not make use of these features, you can consider using them in some future TethysL version of the script if they are applicable.

  Tip

  You can use the lint command.

• Ideally, we would like the original authors or recent committers of the XML files to be directly involved in the respective migrations, such that authorship can at least be preserved to some extent.¹ However, recognizing this may not be practical in many cases, we will be proceeding as needed.

---

1. Note that the git history of a converted script will be lost due to the file renaming involved. There are possible mechanisms (including git-filter-repo) that we could consider to better preserve history, but this is not currently planned. ↩