9 April 2018 Västerås, Sweden


2nd International Workshop on Testing Extra-Functional Properties and Quality Characteristics of Software Systems

Co-located with ICST 2018
Extended Submission Deadline: Jan 21, 2018

Call For Papers

The workshop does not accept papers that focus purely on functional testing!

Best papers from the workshop will be invited to submit an extended version of their work to
the Special Issue on "Testing Extra-Functional Properties" in the Software Testing, Verification and Reliability Journal!

Download: Flyer as PDF ----- Call-for-Papers as text

As the presence and role of computer systems in our daily life increases, we rely more and more on the services that are provided by software. On one hand, more tasks and functions are delegated to software systems (e.g., in the automotive domain), and on the other hand, the expectations and demands on the variety of services provided by these systems are dramatically growing (e.g., in mobile phones). In this context, the success of a software product may not only be dependent on logical correctness of its functions, but also on their quality characteristics and how they perform. Such system characteristics, which are referred to and captured as Extra-Functional Properties (EFPs), or Non-Functional Properties, have determinant importance particularly in resource constrained systems. For instance, in real-time embedded domain there can be limitations on available memory, CPU and processing capability, power consumption, and so on, that need to be considered along with timing requirements of an application. Considering the rapid development towards increased integration of software with the social and physical world that we see today, quality aspects become more important in an increasing number of the systems and devices we use and depend on. These systems therefore, need to be tested with a special attention to EFPs such as safety, security, performance and robustness.

Testing a system with respect to its EFPs, however, poses specific challenges and traditional functional testing methods and approaches may not simply be applicable. Examples of such challenges are: fault localization, the need to have appropriate techniques for different types of EFPs, the role and impact of the environment in testing EFPs, observability and testability issues, coverage and test-stop criteria, modeling EFPs and generating meaningful test cases, mutation operators for EFPs, etc.

ITEQS provides a well-focused forum with the goal of bringing together researchers and practitioners to share ideas, identify challenges, propose solutions and techniques, and in general expand the state of the art in testing EFPs and quality characteristics of software systems and services.

Check the topics of interest for more details.

Towards the goal of ITEQS, the topics of interest for the workshop include, but are not limited to, the following:

  • Model-based testing of EFPs; e.g., choice of modeling languages to capture EFPs and their role on testability, model-based test case generation, etc.
  • Mutation-based testing for EFPs; e.g., application of mutation techniques for testing of EFPs particularly introduction of EFP-specific mutation operators
  • Search-based testing techniques for EFPs
  • Testability, observability, and the role of the platform; e.g., how choosing an operating system can impact testability of EFPs, for instance, a real-time operating system, introducing testability mechanisms into a platform, designing middlewares for testing of EFPs
  • Empirical studies and experience reports; e.g., on the importance of testing EFPs, evaluation of testing methods, case-study and reports on project failures due to EFPs, comparison of methods and techniques
  • Quality assurance, standards, and their impact on testing EFPs
  • Requirements and testing EFPs; e.g., identification and generation of test oracles for EFPs from requirements, requirements for testability, traceability
  • Coverage criteria in testing EFPs
  • Processes and their role in testing EFPs; e.g., agile and TDD
  • Fault localization for EFPs and debugging
  • Formal methods, model-checking, and reasoning about EFPs
  • Parallelism, Concurrency, and Testing of multicore applications
  • Performance, Robustness, and Security Testing
  • Testing real-time, embedded, and cyber-physical systems, and their challenges
  • Testing quality characteristics of distributed, mobile, and cloud applications

The workshop promotes two types of contributions in IEEE double-column format:

  1. 1) Position papers and also solid Work-in-Progress papers consisting of 4 pages. WiP papers should provide concrete (but not necessarily complete) solutions and results as well as detailed plans for future extensions. Position papers should provide a clear position about research in testing EFPs arguing for a particular research direction in this area, discussing open problems and challenges, emerging topics and applications, and so on.
  2. 2) Full research papers consisting of 6-10 pages.

Each submitted paper will receive at least three reviews. To be faithful to the scope and goals of the workshop, papers that focus only on functional aspects and fail to address any EFPs and quality attributes will not be accepted.

Papers must conform to the IEEE double-column format (http://www.ieee.org/conferences_events/conferences/publishing/templates.html). Authors can submit their papers through easychair at: https://easychair.org/conferences/?conf=iteqs2018. Accepted and presented papers will be included in the IEEE CPS Proceedings.

Important Dates

Submission Deadline Jan 21 (extended)Jan 12, 2018
Notification of Acceptance Feb 21, 2018
Camera-ready Deadline Feb 28, 2018
Workshop Date April 9, 2018


To be updated...

Towards Compositional Testing of Real-Time Systems (Keynote)

Prof. Brian Nielsen Aalborg University

In this talk, we discuss how we may achieve a methodology for compositional testing of real-time systems. We revisit the notion of compositional testing in the setting of real-time systems. In particular, we introduce crucial notions of real-time conformance testing and compositional verification of real-time systems. We illustrate these notions on a small example, and show how the tools Uppaal Tron (online real-time testing), Uppaal Ecdar (refinement-checking) and Uppaal SMC (statistical model-checking for stochastic hybrid automata) provide strong support for a compositional testing methodology.

Organizing Committee

Contact us: iteqs2018 [at] easychair [dot] org or mehrdad.saadatmand [at] ri [dot] se

Mehrdad Saadatmand

Program chair

RISE SICS Västerås, Sweden

Birgitta Lindström

Program chair

University of Skövde, Sweden

Bernhard K. Aichernig

Program chair

Graz University of Technology, Austria

Program Committee

Antonia Bertolino

CNR (National Research Council), Italy

Brian Nielsen

Aalborg University, Denmark

Lionel Briand

University of Luxembourg, Luxembourg

Mohammad Mousavi

University of Leicester, UK

Vahid Garousi

University of Luxembourg, Luxembourg

Jeff Offutt

George Mason University, USA

Yvan Labiche

Carleton University, Canada

Vittorio Cortellessa

University of L'Aquila, Italy

Markus Bohlin

RISE SICS Västerås, Sweden

Björn Lisper

Mälardalen University, Sweden

Mika Mäntylä

University of Oulu, Finland

Hadi Hemmati

University of Calgary, Canada

Bestoun S. Ahmed

Czech Technical University in Prague, Czech Republic

James H. Hill

Indiana University-Purdue University Indianapolis, USA

Wasif Afzal

Mälardalen University, Sweden

Pasqualina Potena

RISE SICS Västerås, Sweden

Heinz Schmidt

RMIT University, Australia


Nils Müllner

Mälardalen University, Sweden

Sahar Tahvili

RISE SICS Västerås, Sweden

General sponsors

organizations that support us


  • Aros Congress Center, Munkgatan 7, Västerås