Knowledge of the worst-case execution time (WCET) is important for reliable real-time software. Several tools exist for determining WCET bounds or estimates for a given program or its parts (eg. individual tasks). Some tools are based on pure static analysis of the program, others combine execution-time measurements with static analysis in various ways.
The purpose of the WCET Tool Challenge is to be able to study, compare and discuss the properties of different WCET tools and approaches, to define common metrics, and to enhance the existing benchmarks and collect and develop new benchmarks. The WCET Tool Challenge has been designed to find a good balance between openness for a wide range of analysis approaches, and specific participation guidelines to provide a level playing field. This should make results transparent and facilitate friendly competition among the participants. However, participants and other interested parties should be aware that results from different WCET tools will still be hard to compare directly, and there is not yet an established classification or set of performance metrics in this field. Therefore, the purpose of this Challenge is not to establish "winning tools".
The WCET Tool Challenge is both a continuous process and a periodic event. The collection, definition, and improvement of benchmarks, analysis problems, and analysis results is a continuous process. Periodically, a summary of the analysis results is prepared and presented. A deadline is set in advance, and all analysis results recorded before the deadline are included in the summary and presentation.
The first WCET Tool Challenge event took place in 2006. The results were presented at the ISoLA 2006 conference and a full report is available.
A new Challenge took place place in 2008, with a revised set of benchmarks. The status and initial results were presented and discussed at the WCET Workshop 2008 on July 1. The slides of the presentation are available. The final report was published in the proceedings of the WCET Workshop 2008.
The Artist2 Network of Excellence supported the WCET Tool Challenge in 2006 and 2008, as an activity of the Compilers and Timing Analysis cluster.
The WCET Tool Challenge is aimed at developers of WCET tools, whether academic groups or commercial tool vendors. The participants are expected to use their tools to analyze the specific benchmark programs chosen for the Challenge and to report their analysis results in a standard format to the Challenge website (a Wiki). The participants are also encouraged to submit new benchmarks and propose new problems for analysis, whether based on existing or new benchmarks.
The organizers of the WCET Tool Challenge 2008 collected the results from the participants, via the Wiki site, and prepared the summary report for the WCET Workshop 2008 (referenced above).
To find out more about the WCET Tool Challenge 2008, visit our Wiki. Anyone can read the Wiki contents, but not edit them. Participants in the Challenge are registered with editing rights. To register on the Wiki with editing rights, please contact niklas.holsti@tidorum.fi.
The WCET Tool Challenge 2008 was an activity of the Artist2 NoE, specifically of the Compilers and Timing Analysis cluster. The steering group for WCC'08 consisted of:
| This work has been funded in part by the ARTIST2 Network of Excellence (http://www.artist-embedded.org/). |
|
|
|
Last updated 2009-07-15/NH