11th International Software Engineering Research Network Meeting (ISERN 2003)

Program

Presentation slides

Click here to view

Friday, 3 October

8:10 Bus Pick-up (Frascati square and Hotel Villa Vecchia)
8:45 - 9:45 Welcome
Observer Presentations
Chair: Giovanni Cantone

* Observers (up to 5 minutes each) *
Tony Cowling, University of Sheffield, UK
Hakan Erdogmus, National Research Council, Canada
Des Greer, Queen's University Belfast, UK
Michele Marchesi, Univesità di Cagliari, Italy
Maurizio Morisio, Politecnico di Torino, Italy
Antony Powell, University of York, UK
Austen Rainer, Hertfordshire University, UK
Laurie Williams, North Carolina State University, USA
Silvia Abrahao, Universidad Politécnica de Valencia, Spain
9:45 - 10:45 Session 1: Experimentation and Simulation - How to Make Synergy? (Part I)
Chair: Günther Ruhe

The purpose of the session is to study the interplay between Experimentation and Simulation. Among others, the following topics are intended to be addressed:
  • What are the limitations of experimentation (in the real-world) and simulation ('experimentation in the virtual world')?
  • What are the strengths of both approaches?
  • How does simulation supplement experimentation?
  • How does experimentation contribute to simulation?
  • What would an integrated approach looks like?
  • What concrete results are currently known for practically integrating both approaches?
  • What are related (future) research questions?
Format:
The session will contain presentations and then a discussion session.
The following presentations are planned:
  1. Martin Höst: Experience from software process simulation studies
  2. Jürgen Münch: Creating an advanced software engineering laboratory by combining empirical studies with process simulation
  3. Günther Ruhe: Goal-oriented measurement plus system dynamics: A hybrid and evolutionary approach
  4. Raymond Madachy: Using Empirical Data in Software Process Simulation
  5. Forrest Shull: Using simulation to provide decision support for software inspection use
Discussion will be either as a panel or as an open discussion in conjunction with the presentations.
10:45 - 11:00 Coffee Break
11:00 - 12:00 Session 1: Experimentation and Simulation - How to Make Synergy? (Part II)
12:00 - 1:00 Lunch
1:00 - 3:00 Session 2: Empirical Study of Decisions in Software Engineering and Project Management
Chair: Stefan Biffl

Many empirical studies are run to help developers and project managers make better decisions in planning software development activities. This session will give empirical researchers a better understanding of how empirical studies can effectively achieve this goal by attempting to classify development decisions, and which decision models are available as drivers (especially relating those decisions to a company's business strategy). The session will also look at the implication of a company's strategy for its need/use of empirical data, and for the type of empirical studies that are most appropriate.

Topics of session
  1. Types of software engineering decisions
    1. Motivation for using decisions as drivers for empirical study: why support decision makers, how does data help?
    2. Overview on key decisions in industry and current practice of decision making
    3. Decision models that link decisions/need for decision support with empirical data. Classifying decisions and whether empirical data plays a different role for different decision types.
  2. Using empirical data to provide decision support
    1. Some examples of key decisions and successful decision support tools
    2. Aggregation of empirical studies to support decisions, gap analysis: What analysis techniques are available/have been used to abstract needed information from evidence?
  3. Implications for future empirical research
    1. What are most worthwhile focus points of interest in this area for the ISERN community?
    2. Invite people to participate in research on the most interesting focus areas: Development of theory, data collection, and empirical studies.
Format:
Presentations with some discussion; followed by brainstorming/discussion/voting on future ISERN research (with a short follow-up feedback session).

Presenters/Discussants:
  • Günther Ruhe
  • Hakan Erdogmus
  • Jyrki Kontio
  • Barry Boehm
  • Dan Port
  • Marcus Ciolkowski
3:00 - 3:15 Coffee Break
3:15 - 5:15 Session 3: Tool Support for SE Experimentation
Chair: Philip Johnson

We all know that current tool support for software engineering experimentation is terrible. In this session, we're going to do something about it.

We will begin with a small number of short "provocations", in which panel members will present their completions for the following sentences:

1. The top three things that really bother me about the tools I use for experimentation are: ___________. Here's why: ____________.

2. If I had a good graduate student, I would make him/her build the following experimental tool(s) for me: __________.

(Panelists can also dedicate a maximum of 60 seconds and 1 powerpoint slide to shamelessly promote their own tool, if they have one. However, panelists do not have to be tool developers, they just have to be frustrated experimental tool users!)

After the provocations, we will spend the remaining time working together to plan out a ten year research agenda that makes both evolutionary and revolutionary advances in tool support for experimentation.

Format:
Panel presentations, group discussion, research agenda definition.
5:15 Back to Frascati and Hotel Villa Vecchia. Bus Pick-up
6:00 Bus pick-up for ISERN Dinner at the Antique Thermae of Diana in Tivoli

Saturday, 4 October

8:10 Bus Pick-up (Frascati square and Hotel Villa Vecchia)
8:45 - 10:00 Welcome Back
Candidate Member Presentations
Chair: Dieter Rombach

* Applicants (10 minutes each) *
Katsuro Inoue, Osaka University, Japan
David Klappholtz, Stevens Institute of Technology
Jyrki Kontio, Helsinki University of Technology, Finland
Audris Mockus, Avaya Labs Research, USA
Mario Piattini, University of Castilla-La Mancha, Spain
Bernard Wong, University of Technology Sydney, Australia
10:00 - 10:15 Coffee Break
10:15 - 12:15 Session 4: Subjects/Objects and Validity of Studies
Chair: Martin Höst

We will discuss the use of students, toy examples and so forth. We think it is important that we increase our understanding of the influence of different types of subjects and objects. For example, if the difference between two competing methods is very large for students, is it likely that the results will not be generalizable to professionals? Under what circumstances do students make reasonable subjects? When are small programs/designs appropriate, etc.? What other comparable factors affect the validity?

Format:
10.30 - 10.35: Part 1, Martin Höst
  • Introduction
10.35 - 10.55: Part 2, Austen Rainer
  • What are the most important factors for achieving high external validity?
10.55 - 11.15: Part 3, Dag Sjøberg
  • Students versus Professionals as subjects in experiments
11.15 - 11.35: Part 4, Martin Höst
  • Classification of experiments with respect to subjects and objects
11.35 - 11.55: Part 5, Markku Oivo
  • Subjects and objects in large research projects; a new research strategy
11.55 - 12.30: Part 6
  • Discussion
12:15 - 1:15 Lunch
1:15 - 3:15 Session 5: Empirical Studies of Pair Programming
Chair: Erik Arisholm

This session will explore empirical studies of one key agile practice, Pair Programming, in detail.

Presenters will contribute with presentations on previous and planned empirical studies using different approaches, e.g. qualitative and/or quantitative data, controlled experiments, case studies, observations, etc. The presentations will be followed by a discussion during which ISERN members are invited to comment on the perceived strengths/weaknesses of the various approaches as a means to evaluate the most important (independent, dependent and contextual) variables of pair programming.

Format:
1:15 - 1:45: Laurie Williams (North Carolina State University)
  • Short overview of the history and main principles of Pair Programming
  • Description of one or more previous or planned empirical studies (TBD)
1:45 - 2:15: Pekka Abrahamsson (VTT - Technical Research Centre of Finland)
  • This presentation will describe a "controlled case study" on extreme programming in a close-to-industry setting. Four SW developers were acquired to develop an information system for the purposes of VTT Electronics. The timetable was tough - 8 weeks comprising all SW development phases including the system test. We monitored the data collection procedure in great detail and collected a large number of qualitative and quantitative data on the project's progress including pair programming data.
2:15 - 2:45: Hans Gallis (Simula Research Laboratory)
  • Presentation of a planned controlled experiment on Pair Programming, scheduled for spring 2004. Our plan is to compare the performance of 100 pairs (200 professional Java consultants from Scandinavia and USA) with results of an existing experiment with 190 individuals solving the same programming tasks. The dependent variables might include development effort and different measures of solution quality. The independent variables might include different pair configurations (e.g., novices versus experts), individual Java programming skill, pair programming experience and task complexity.
2:45 - 3:15: ISERN meeting participants
  • Discussion
3:15 - 3:30 Coffee Break
3:30 - 4:30 Open Session
4:30 - 5:00 ISERN Business and Closing
5:00 Back to Frascati and Hotel Villa Vecchia. Bus Pick-up