What’s under the bonnet in CanalPlanAC: Part 2 – data formats

Over the years CanalPlanAC has accumulated a host of data formats for various purposes. At least two – a custom C-like syntax (surprisingly similar to another I’ve adopted!) and “quick load” place and link files – are entirely obsolete but support for them still lurks in the interpreter.

Here are the main ones that anyone playing with the system might come across:

  • SQL – since summer 2008 the main database for CanalPlanAC, the users, the information about photographs, some of the program flow control and state information during a session have been stored in SQL databases. I’m using SQLite – this is small and can be directly integrated into the interpreter making setup very easy.
  • XML – until then XML was used for the master data file, and quick-read varieties in a sort of unstructured text file were generated. The unstructured files are also obsolete (although some support may still exist in route.c until I rip it out) but XML is still used although much more round the edges these days. In particular, it’s used for:
    • Sending data to some Javascript utilities.
    • Storing the configuration file /cgi-bin/config.xml which tells the program where all the data should be stored and similar things
    • To store the default user options in a file, and the user options (in a blob) in the user SQL database
    • To define options for gazetteer plugins (to be described some other time!)
  • JSON – not used statically, but used for a number of inter process and code communications (particularly those involving Javascript)
  • HTML – static pages are written in HTML
  • CST – ClearSilver Templates. Since summer 2007 pages are generated from template files. This is an extended form of HTML with ClearSilver directives built into it – see the website for more.
  • HDF – this is the configuration file language for ClearSilver. I didn’t have to use this, I’ve integrated ClearSilver so tightly I can easily pass it any format I like, but it seemed sensible. The most important HDF file is /templates/default/config.hdf which controls all the menus and buttons on all the pages.

So there you are. A pile of formats, but many of them widely known, and many of them only used in a few places. In the next few articles I’ll show how to add a sophisticated new feature (the Virtual Cruise) to CanalPlanAC and you’ll see how these various languages and formats are used.