Table of Contents
The LUA Make Environment
This page describes a build environment which is not released to the public yet. Although it is in daily use in a OEM already, we need to make it first more more generic for common use before releasing it. So this is more an internal paper yet.
How is the Make Environment set up?
The environment was designed to suit the following requirements:
- Handle multiple car lines, each with dozens of module specs and scripts
- batch translation of XML module specs, handwritten scripts and other templates in just one step
- automatic encryption of the generated files, if wanted
To support this, the same kind of environment is used as what is widely known in software devopment: a command line based interface, where are the different tasks are started and controlled by makefiles, scripts and embedded other tools.
That means all step are started by typing the appropriate command and its parameter into a command line.
Setup the Cygwin Runtime Environment
To use the much more powerful unix environment instead of just a stupid DOS command interface, you need to install at first a CygWin Runtime Environment. To do so, please follow the CygWin Installation instructions
Basics about the Cygwin Environment
After installing and starting Cygwin, you'll be presented to a DOS like command line, but with some differences:
- Unix takes care about uppercase and lowercase writing, so everything needs to be correctly written
- the basic commands are different to DOS/Windows. A good introduction can be found here
- Unix does not have backslashes \ in paths, it uses a normal slash / instead
- Unix does not know drive letters like C:\. Instead each drive and directory is a subfolder under the global root directory /cygdrive. So drive C:\ becomes /cygdrive/c/, your Desktop folder would be /cygrive/c/Users/yourusername/Desktop (take care of upper/lower letters!)
- with the cursor keys you can move back in the command history. Very helpful to not typing in the same commands all the time
- with the TAB key you can auto-complete commands and file names.
- Very important: Unix text files have a different End-of-Line-Character as Windows. So please always make sure when creating or editing files, that all files are saved in “Unix Line feed UTF8 without BOM” format
Connect yourself to the central Script Repository
As soon as several scripts need to be maintained or more as one person is working on the scripts, it's highly recommended to use a central script repository, as we do here in our example with a central Subversion Server. The usage of subversion as such is out of the scope of this document, but you'll find tons of documentation in the Internet. Please read at last some base tutorials to get a basic understanding about subversion works.
To become connected to the so called repository, please get in contact with your repository admin. From him you'll get the so called repository-URL, your username and your password.
As next generate an empty directory, wherever you want to store all the script data, e.g. on your desktop
cd /cygdrive/c/Users/yourusername/Desktop mkdir OOBD-Script-Repository cd OOBD-Script-Repository
as next create the so called “local working copy” of the repository with the command
svn checkout repository-URL
You'll be asked for username and password, and then svn (which is the short form for subversion) will transfer the whole content of the central repository into your working directory.
You might notice later that a lot of .svn directories have been created. In there subversion stores the metadata for the maintenance. Do not delete or even edit these directories!
The Directory Tree
To use the available build tools for OOBD, it is essential that you keep the file structure as shown, as the tool settings rely on that.
After finishing the initial checkout, you'll find a directory and file structure like this:
. +-- ... +-- lua +-- mdx_pool +-- oobd_groups.pub +-- oobd_groups.pub.sig +-- tools
So for what all this good for?
The mdx_pool is the directory, where all the XML module can be stored in, named by carline_modellyear_moduleshortname
The tools directory contains all the necessary oobd related programs and scripts. The experienced user might need to know that some of the subdirectories are linked as so called “svn:external” to other repositories.
The oobd_groups.pub is a pgp keyring which contains the public keys of the workgroups, by where the execution rights of the different scripts are controlled.
Finally there's the lua directory, which contains the lua script files itself and a lot of other stuff:
. +-- car1_my99_r ¦ +-- mdx_pool_reference ¦ +-- car1_my99_carcheck.carcheck ¦ +-- car1_my99_sw_versions.lua ¦ +-- lua_reference ¦ +-- files.inc ¦ +-- makefile ¦ +-- presets.rtd ¦ +-- ABC.seccode +-- car1_my99_s ¦ +-- ... +-- car1_my99_t ¦ +-- ... +-- car1_my99_w ¦ +-- ... +-- carcheck.footer +-- carcheck.header +-- carcheck.xslt +-- db ¦ +-- oem.oodb ¦ +-- VehInfo_Stylesheet.xsl +-- generic.make +-- Healthfiles ¦ +-- OOBD-xmlfile.xml +-- lib_lua ¦ +-- ... +-- lib_protocol ¦ +-- ... +-- local.make +-- local_template.make +-- makefile
Inside the lua directory it becomes finally complicated. All these files and directory have their meanings:
The Meanings of the Files and Directories
car1_my99_r /_t / _s / _w
These directories contain the car-line and model-year related stuff. But also each directory represents a so called workgroup, which is used if you going to encrypt the scripts. When doing so, you grant user access to each workgroup seperately, which means you allow the user to use that what you compile inside the related directory.
To make a clear separation between the different level of risks, what a user can make wrong by using a script on a vehicle, we use the directory name extensions to define, in which directory a script must be stored to avoid misuse. The extensions are their meanings are:
|_r||Only read access - no risk of any damage (shall be used for all dummy users)|
|_t||Temporary modifications - temporary changes like activating outputs, but all changes must disappear without trace when finishing diagnostics, minor risk because of unexpected movement of e.g. actuators (for adviced users)|
|_s||Service - starting tests, changing factory modes etc (for service and repair people)|
|_w||Write Access: Full control over the system (only for experienced specialists)|
Whenever you want to compile some module specs out of the module pool for this particular carline and access level, you list the affected module files name in this mdx_pool_reference- file. By this reference the module spec only needs to be stored only in the pool directory and be referenced from many different places.
Whenever the build process finds a *.carcheck file, it takes the list of modules of this carline out of the mdx_pool_reference and builds a kind of summary script, which collects all modules in it and allows a quick overview through the whole vehicle. By adding some content into these *.carcheck file, some carline specific value outputs can be added to the generic content of the vehicle summary.
In opposite to all the other automatic generated output this is now a pure handmade script. Whenever the build process finds a *.lua script, it just compiles it.
In conjunction with the seccode files (see further below) you could compile the same script for different security levels. By the lua_reference file you can store the original script in a different directory, reference to it and compile it in the actual directory with a different security level.
By convention the original script has to be stored in the workgroup directory with its lowest security level, and the higher level reference to it.
*.seccode files contains a piece of lua code, which defines the security access codes of modules. In the lua scripts of the module itself you then can use the pseudo command
– optinclude(“ABC.seccode”), which include the seccodes only if that file exist, but does not generate an error message if the file does not exist.
For confidential reasons, these *.seccode files are not included in the software repository, and it is forbitten to upload this or any other format of security codes into the repository, so each script developer has to maintain his personal knowledge about security codes locally and must not publish this into the repository.
Any other file, which is needed by the script later and which is not already part of the lua compilation process, like e.g. DTC databases can be listed in the files.inc. By that it will be encrypted and copied to the output dir too.
This makefile don't need to be edited. It's just needed for the build process.
for futher extensions.
carcheck.footer, carcheck.header and carcheck.xslt
These are the templates for the carcheck scripts
In here database files (*.oodb) are stored to reference to by files.inc, but also the stylesheet (VehInfo_Stylesheet.xsl) for a Vehicle Health report is stored here.
This file controls the whole build process and must not be modified without very good reasons…
When generating a vehicle report when doing a carcheck, the resulting XML files should saved here. When then opening the file in a web browser, the file automatically points to the VehInfo_Stylesheet.xsl in the db folder and present the data with the right style sheet.
In here you can find some lua library scripts for generic functions
In here you can find some lua library scripts for diagnostic related functions
local.make and local_template.make
local.make will contain your personal settings. It is not part of the repository content, as it's content is different for each user. To generate your own local settings, copy the template file local_template.make and make your settings in local.make
cp local_template.make local.make
Also here in the lua base directory the makefile is needed for the build process. Just don't change it.
Running the make process
In opposite to all the complicate setup and explanations above, the build process is just simple:
Edit or add the files as you need. For a good start, choose one of the existing *.lua files, copy it to where you need it and give it a clear name. Make sure that the script capabilities do not exceed the security rights of the workgroup folder they are stored in.
Then type in the cygwin command line windows the following commands one after the other. If you do this in a workgroup folder, only the files in that folder will be proceed. If you type the commands in the lua root folder, then all workgroup folders, which you have listed in the local.make will be proccessed.
- make clean: this will remove all older results and temporary files
- make: This will convert all module XML files, all other templates and all *.lua files into Lua source files with the extension .luasource. Although normal *.lua files will only been copied by this, this renaming is necessary to not accidently delete some handwritten *.lua files when calling “make clean”
- make scripts: This will now start the real compile process, where the lua- compiler takes all luasource files and translates them into precompiled binaries (*.lbc). Hint: During the compilation of a script all include files are linked together in a temporary file called lua.tmp. If the compiler fails with any error message, that is the right file to look into why the compiler is complaining.
- make pack: When enabled and configured in local.make, this call will take all compiled *.lbc files and encrypt them accourding to the workgroup directory in which they are in, make up the folderstructure in the target directory, which is also configured in local.make, and and copy the results into it.
After that the final output can be found in the configured target directory, encrypted and ready to distribute.