ISERN 2010 Program
Monday September 13th, 2010
| 8:00 - 9:00 | Breakfast and Registration |
|
| 9:00 - 10:00 | Welcome and new introductions (Chair: Dieter Rombach) Location: Room D101 ISERN is open to academic and industrial groups world-wide that are active in empirical software engineering research and willing to adopt the empirical research framework. ISERN members are pairs of organization and contact person. If the contact person leaves the organization, the organization must reapply for membership. Interested organizations may apply by sending an electronic proposal to "isern at informatik.uni-kl.de" describing their past experience in empirical software engineering research as well as their expectations from a future ISERN membership. Candidates will be invited to observe the ISERN Meeting following their application. The goal of the session is to facilitate the membership application process by giving an opportunity for candidates to present their research and for observers to introduce themselves. Membership is granted according to a 3-step procedure: 1. Attending as invited observer at an annual ISERN meeting. 2. Attending as invited candidate at the following ISERN meeting giving a presentation. Membership is granted if a two-thirds majority of current members approve the application in an email voting after the meeting. 3. Attending as a full ISERN member at following meetings. Current members present contact/affiliation changes (2 min each):
|
|
| 10:00 - 10:15 | Report from 2009 session chairs Location: Room D101
|
|
| 10:15 - 10:45 | Coffee break | |
| 10:45 - 12:15 |
What are the empirical results from the 20th century that are invariant of technology? (Chairs: Larry Votta, Mike Barker) Location: Room D101 In this kickoff session, we will look back at what we have accomplished in the last half-century. Starting with a report on the findings from Dagstuhl, we will look at what has been achieved and why it is important. We will then hold a group discussion, first confirming or modifying the results from Dagstuhl, then adding any "missing pieces," and finally looking at what use we believe this information can be in research, education, and industry. As a final consideration, we will look at how we think this information should be publicized and disseminated in research, education, and industry. We will also pose a challenge for the rest of the conference to identify and propose challenges for the 21st century in preparation for the closing session. |
|
| 12:15 - 13:30 | Lunch | |
| 13:30 - 15:00 |
Can we evaluate the quality of SE experiments? (Chairs: Dag Sjøberg, Dietmar Pfahl) Location: Room D101 To make progress in software engineering (SE) in general and empirical SE in particular, we need to improve the quality of experiments and other empirical studies. At many ISERN meetings we have discussed guidelines for improving such quality. Being able to evaluate the quality of empirical studies is a premise to evaluate guidelines and other means to improve quality. Moreover, when carrying out ordinary reviews for conferences or journals, as well as literature reviews, we are also supposed to be able to evaluate quality. However, what is the case at present? In a pilot study conducted by 8 experienced researchers (5 of them associated with ISERN), we disagreed on the quality of a set of four selected papers that reported experiments. When we had a joint discussion in pairs, the reliability (agreement) increased. In this session, we want to explore whether working as individuals, or in groups (with different sizes) affects the reliability of evaluating quality of SE experiments. |
|
| 15:00 - 15:30 | Coffee break | |
| 15:30 - 17:00 | What should software engineers be measuring in the future? (Chair: Barry Boehm) Location: Room D003 This session draws on ISERN's experience in addressing the question of "What should software engineers be measuring in the future?" Increasingly, we are finding from large Government projects to small e-services projects that software engineers are less involved with greenfield designing and programming, and more involved with the evaluation, integration, and evolution of combinations of off-the-shelf (OTS) products, cloud-computing services, and legacy systems, along with generating code from model directives. Terms like "delivered" or "equivalent" source lines of code, defect density, productivity, etc., that are gathered in many repositories are hard to compare. Terms like requirements volatility or code churn are increasingly important and hard to define or measure. The session begins by comparing across various industry sectors and countries the extent to which such trends are presenting problems, and what is best to do about them. This will be followed by having breakout groups discuss and summarize their experiences and initiatives in measuring software size, effort, duration, productivity, quality, etc. for different classes of emerging software development modes (traditional, agile, evolutionary, OTS/services-driven, legacy-driven, hybrid, etc.). |
Ontologies in empirical software engineering (Chair: Ye Yang) Location: Room D002 Ontology has been popular in many fields such as AI, Agent systems, Database or Web Technologies. In recent years, many discussions have been going on around the marriage of ontology and software engineering. One definition for ontology is "a formal explicit specification of a shared conceptualization". Every aspect of ESE needs to be better formally specified to conceptualize a common set of knowledge, from the methods we perform ESE studies, what data to collect and how, to a concrete description of the empirical findings to enable practicability and comparability. Moreover, lots of non-structured data such as documentation can be machine-understandable after annotation and can be used for further ESE studies. All in all, if we consider ESE as a part of reality construction, we may find adequate place for ontologies in it. In this session, we would like to hear the participants' opinions about this, and look for potential collaborators in this regard. |
| 17:00 - 17:15 | Wrap-up and plan for Tuesday Location: Room D101 |
|
| 17:15 - 18:15 | ISERN SC Meeting (By invitation only) Location: Room D003 |
|
| 19:00 - 23:00 | ISERN dinner Location: Laimburg Winery |
|
Tuesday September 14th, 2010
| 8:00 - 9:00 | Breakfast and Registration |
|
| 9:00 - 9:30 | Feedback from Monday's study: Can we evaluate the quality of SE experiments? (Chairs:Dag Sjøberg, Dietmar Pfahl) Location: Room D101 |
|
| 9:30 - 10:30 |
Systematic literature reviews in SE: Status and outlook (Chairs: M. Ali Babar, Reidar Conradi) Location: Room D003 A growing number of systematic reviews are being conducted and reported by software engineering researchers who have been reporting the methodological and technological challenges involved in conducting and reporting high quality systematic reviews. There is a vital need to discuss the challenges involved in doing systematic reviews in software engineering at a community platform like ISERN in order to identify the potential strategies and actions required for addressing the scientific and technological challenges involved. Moreover, it is also important to review and discuss the methodology adoption in software engineering and brainstorm the improvements required. This session aims to review the state-of-the-practice of systematic reviews over the past 6 years, stimulate a debate on the rigor and relevance of this relatively young research methodology in software engineering, and encourage an open dialogue between ISERN members to assess its relevance. After a set of questions to instigate discussion and brainstorming activities, we aim to discuss and assess the efforts and commitments required to provide the scientific and technological infrastructures required for maturing and using systematic reviews to advance software engineering research and practice based on evidence. |
Empirical studies in industry (Chairs: Andreas Jedlitschka, Laurie Williams, Günter Ruhe) Location: Room D002 A major role of empirical software engineering research is to provide decision makers in software development with the type of evidence about technologies that they need to support informed decision-making when introducing new technologies. From managers' perspective, results from studies performed in industry would be preferred above any other. Several models for cooperation between research and industry in the context of empirical studies were proposed in the late 1990 and early 2000. Further, the community discussed for several years about how to best perform studies in industry and about potential models for long-term collaboration. Yet, still there seem to be some challenges arising from putting the theory into practice. The goal of the session is to collect lessons learned from examples of studies performed in/with industry; identifying what went well and where a change is needed? The result of the session is list of lessons learned, e.g., items to think of already during initiation and study design. The session will start with a very brief wrap-up of the discussions from recent meetings (starting with Dagstuhl 2006). Then facilitators will present some examples of studies performed in/with industry answering a set of predefined questions (like for a project retrospective). The major part of the session is dedicated to the discussion of lessons learned and improvement suggestions both for planning, conducting, analyzing, and reporting empirical studies in/with industry. Results are also expected to contribute to the knowledge regarding long-term collaboration. |
| 10:30 - 11:00 | Coffee break | |
| 11:00 - 12:15 |
Qualitative and quantitative synthesis of research evidence (Chairs: Marcus Ciolkowski, Daniela Cruzes) Location: Room D003 Synthesizing the evidence from a set of studies that spans many countries and years, and that incorporates a wide variety of research methods and theoretical perspectives, is probably the single most challenging task of performing a systematic review. Research synthesis is a collective term for a family of methods for summarizing, integrating, combining, and comparing the findings of different studies on a topic or research question. Such synthesis can also identify crucial areas and questions that have not been addressed adequately with past empirical research. It is built upon the observation that no matter how well designed and executed, empirical findings from single studies are limited in the extent to which they may be generalized. Research synthesis is, thus, a way of making sense of what a collection of studies is saying. The goal of this session is to discuss research challenges in synthesis in ESE. The session will have the following structure:
|
Global sharing of empirical software engineering repository(Chair: Yoshiki Mitani) Location: Room D002 Recently, constructions of the software engineering repository have prospered in various countries. For example in Japan two kinds of documents were published in English, one was related to the benchmarking and the other was related to the visualization of software development projects: http://www.ipa.go.jp/english/sec/reports/20100507a.html http://www.ipa.go.jp/english/sec/reports/20100507b.html The goal of the session is to get some practical proposals (ideas) on how to share each domestic information repository globally, and to discuss how to share such kind of information to conquer the barrier of individuality and confidentiality of software engineering research. The session will consist of:
|
| 12:15 - 13:30 | Lunch | |
| 13:30 - 15:00 | "I've seen the future, and it's software-shaped" Software Factory - An empirical SE research infrastructure (Chairs: Pekka Abrahamsson, Juergen Muench) Location: Room D003 How does the software development of 2010's look like? How does it feel to develop software in a company with no history? How to develop software with no specs - with just the desire to be all you can be? Sounds like an ad-hoc company, doesn' it. But, no, wait a moment: Software of the 2010's targets for innovative software, which is rarely developed. Software of today & past try to meet the specs with poor success. It has been so for 30 years. Let's stop it and dig deep. Software engineering is one of the few disciplines, which continue to lack the university hospital of software development. There is no facility specifically designed to enable the research into software development that is realistic and open where data can be freely shared for verification or other purposes. We can keep up debating about the need for setting up such an infrastructure or just go ahead and begin building it. The Software Factory is a strategic investment to a new infrastructure supporting empirically driven software engineering research, education and entrepreneurship globally. The reference implementation of the Factory is now in place and it is currently expanding to its global capacity housing up to 150 software engineers by end of 2011. Early results are very promising. In the ISERN session, we seek feedback, criticism, concerns, ideas and of course potential Software Factory locations globally. In Europe, we have Factories being set up in several locations already. For reading in your long plane travel, go to http://www.softwarefactory.cc/magazines for details. |
Observability of engineering processes in distributed teams (Chair: Stefan Biffl) Location: Room D002 The main goal of this session is to identify and classify candidate measurement and analysis approaches for engineering process observation in heterogeneous environments to: (a) enable process monitoring and control and (b) identify process improvement approaches in automation systems domain projects. Introduction to the Engineering Service Bus concept with focus on engineering process definition, observation, and improvement, and example workflows: e.g., Continuous Integration and Test, Signal Engineering in industrial automation systems projects. Breakout groups will gather needs and ideas on processes to observe, data collection and analysis. Discussion of summary and further work in the ISERN context. For more details of the session, please click here |
| 15:00 - 15:30 | Coffee break | |
| 15:30 - 17:00 | What are the great challenges of the 21st century that can be resolved using empirical studies? (Chairs: Larry Votta, Mike Barker) Location: Room D101 This session will wrap up the conference, building on the summary of what was achieved in the 20th century that we will define during the takeoff, and on observations from the other sessions. It will look at our future in terms of great challenges. This session will start with some examples of great challenges, such as the 14 engineering challenges proposed by the National Academy of Engineering. We will then have proposals by a panel of experts, and do some team brainstorming. We will collect ideas from the group, and uses these to begin defining a sketch of what we consider the great challenges of the 21st century for empirical software engineering. |
|
| 17:00 - 17:30 | ISERN business (Chair: Dieter Rombach) Location: Room D101 |
|
To the printer-friendly version
ISERN 2010 Group Picture:
For more ESEIW2010 pictures, join the Facebook group of ESEIW2010 [
]
