Software Intensive Systems - Cordis

1 downloads 223 Views 356KB Size Report
opment of Europe's industry. They enable ..... ulation of European societies ages, e-health systems will become indis- p
Report of the Beyond the Horizon Thematic Group 6 on

Software Intensive Systems

Draft, January 25, 2006

Martin Wirsing Thematic Group Coordinator and Editor

Matthias H¨olzl Editor

Contributors: Jean-Pierre Banˆatre, Juan Bicarregui, Ed Brinksma, Simon Dobson, Peter Druschel, Jos´e Fiadeiro, Pierre Fraignaud, Fausto Giunchiglia, Manuel Hermenegildo, Stefan J¨ahnichen, Helen Karatza, Stephan Merz, Joseph Sifakis, Mikhail Smirnov, Christian Tschudin, Franco Zambonelli

Contents Executive Summary 1. Engineering Adaptive Software-Intensive Systems 2. Managing Diversity in Knowledge 3. Eternal Software-Intensive Systems

3 4 4 5

Chapter 1. Introduction—Motivation 1. Living in a Highly Interconnected World 2. Business in a Highly Interconnected World 3. Embedded Systems 4. Service-Oriented Systems 5. Software-Intensive Systems

7 7 7 7 8 8

Chapter 2. Vision and Grand Challenges 1. Knowledge, Interaction and Adaptation 2. Human-Oriented Systems 3. Software Quality 4. Beyond Conventional Software Engineering

11 12 13 13 14

Chapter 3. Proposals for new Research Programmes 1. Engineering Adaptive Software-Intensive Systems 2. Managing Diversity in Knowledge 3. Eternal Software-Intensive Systems

17 17 22 28

Chapter 4. Conclusions

33

Appendix A. Summary of Workshop Results 1. The Workshop and its Objectives 2. Challenges for Software-Intensive Systems 3. Workshop Agenda 4. Participants and CVs

35 35 35 37 38

1

Executive Summary Software has become a central part of a rapidly growing range of applications, products and services from all sectors of economic activity. Systems in which software interacts with other software, systems, devices, sensors and with people are called software-intensive systems. Examples include large-scale heterogeneous systems, embedded systems for automotive and avionics applications, telecommunications, wireless ad hoc systems, business applications with an emphasis on web services etc. Our daily activities increasingly depend on complex software-intensive systems that are becoming ever more distributed, heterogeneous, decentralised and interdependent, and that are operating more and more in dynamic and often unpredictable environments. This development has several important consequences: • There exist different kinds of complexity in the development of software. Historically, as software systems grew larger, the focus shifted from the complexity of developing algorithms to the complexity structuring large systems, and then to the additional complexities in building distributed, concurrent systems. In the next ten to fifteen years we will have to face another level of complexity arising from the fact that systems have to operate in large, open and non-deterministic environments: the complexity of knowledge, interaction and adaptation (see Figures 1 and 2). • Instead of developing computer-oriented systems where people have to adapt to the computer we have to develop human-oriented systems into which computers integrate seamlessly. • Requirements for software quality will dramatically increase. But our current methods are not sufficient to deal with adaptive software in a dynamic environment, especially not for large systems with complex interactions. We need to develop practically useful and theoretically well founded principles, methods and tools for engineering future softwareintensive systems. Current engineering methods and tools are not powerful enough to design, build, deploy, and maintain software-intensive systems with these properties. There is, however, no realistic alternative to such systems: we cannot afford to stop building software-intensive systems, and we can especially not afford to build inflexible, unreliable software-intensive systems. Today’s grand challenge is to develop practically useful and theoretically well-founded principles, methods, algorithms and tools for programming and engineering reliable, secure and trustworthy future software-intensive systems throughout their whole life-cycle. Among the many promising areas for future research, the participants in this Thematic Group have identified three complementary crucial areas: engineering adaptive software-intensive systems; managing diversity in knowledge; and eternal software-intensive systems. The first two areas focus on growing complexity: of services and systems; and of data and knowledge. The third area is transversal to 3

4

EXECUTIVE SUMMARY

Figure 1. Increasing Complexity of Software over Time

the former two; it arises from the current perception that software-intensive systems and their data a here to stay in the long term. 1. Engineering Adaptive Software-Intensive Systems The prevalent approach to software construction—where the full set of program behaviours and features is decided when software is designed—does not scale to pervasive, highly dynamic systems. Emergent properties and behaviours of systems and their environments are an unavoidable fact that must be taken into account, handled, and if possible even exploited throughout the system’s life time, in order to scale to the level and kind of complexity we are witnessing. Systems will no longer be produced ab initio, but more and more as compositions and/or modifications of other, existing systems, often performed at runtime as a result of a process of evolution. An adaptive system is a software-intensive system that can adjust to and respond to changes in its environment, evolving requirements, obsolescence of existing or introduction of new technologies, and newly gained knowledge. The challenge is to develop algorithms, methods, tools and theoretical foundations that enable effective design of adaptive systems which harness, control and use the effects of emerging system and environment properties. 2. Managing Diversity in Knowledge We are facing an unforeseen growth of the volume and complexity of the data, content and knowledge being produced. In knowledge management and engineering, the “usual” approach is to take into account, at design time, the possible future knowledge components, most commonly by designing a global reference representation schema. As applications become more and more open and complex, this top-down approach shows its limits. We need a new, bottom-up, approach for managing knowledge where the different knowledge parts are designed and kept “locally” and independently; new knowledge is obtained by the design- or run-time adaptation of such different knowledge parts. This calls for developing adaptive or even self-adaptive knowledge systems that are able to manage diversity in knowledge by harnessing, controlling and using the effects of emergent knowledge properties.

3. ETERNAL SOFTWARE-INTENSIVE SYSTEMS

5

Figure 2. Increasing Complexity of Data over Time 3. Eternal Software-Intensive Systems Information, and the tools to work with it, represent one of society’s most important assets. From a cultural as well as an economic point of view, it is essential to enable continuous and up-to-date access to long-lived and trustworthy information systems, as well as to guarantee that the corresponding information systems do not age and break. Systems should be able to preserve and update their original functionality and properties in a machine independent way by making it easy to re-program—or even by making it possible for the systems to re-program themselves—to take into account new contexts. In other terms, the challenge is to organise software-intensive systems so that they can survive and evolve in a constantly changing world.

CHAPTER 1

Introduction—Motivation Software-intensive systems—Systems in which software interacts with other software, systems, devices, sensors and with people—are playing an increasingly dominant role in our lives and daily activities. In the following sections we elaborate on some factors responsible for this development and show the challenges that these factors pose for the development of software-intensive systems. 1. Living in a Highly Interconnected World We are living in a highly interconnected world. In the last few years our arsenal of personal communication means has expanded from wire-bound telephone and mail—at first slowly by the introduction of the fax machine, then more and more rapidly with the Internet, mobile phones, portable computers with Bluetooth and WLan, and so on. Instead of writing letters we send mails and SMSes. Our documents are no longer only stored in private and public libraries, they are now often available on a network, accessible from our laptops or even mobile phones. For the first time, we can choose to communicate whenever, wherever and whatever we want. We are no longer tied to the opening hours of a post office, to the place where our phone is plugged into the wall, or to crude copies of black-andwhite pages printed on thermo-paper. We can send text, images or video; we can single-cast or broadcast; we can chose synchronous communication like ICQ or we can communicate asynchronously, e.g., via email. We are now starting to see the movement to more complex systems: Instead of having a video recorder or DVD player we have home entertainment centres that use the Internet to offer value-added services, like on-demand music or film downloads. We can access the network of our bank to manage our personal finances, etc. 2. Business in a Highly Interconnected World It is no wonder that the possibilities of an interconnected world have also transformed the way we do business: Letters and faxes have to a large part been replaced by email; employees “telecommute” by accessing the resources of their company from their personal computer; businessmen can freely interact with their company while they are travelling; increasingly business processes are restructured to involve more automatic communication by computers and less manual intervention by humans. Literally billions of Euros worth of financial transactions are effected each day by the networked computers of banks. We rely so heavily on these systems that even a few hours of downtime have severe repercussions, from the mild discomfort that many of us experience when we suffer a network outage for a few hours, to millions of Euros of damage when software-intensive systems used in international trade fail. 3. Embedded Systems The tools and machines we use in our daily lives are becoming increasingly sophisticated—mostly because they contain more computing power. A car from the 7

8

1. INTRODUCTION—MOTIVATION

1970s is mechanically not that different from a modern car, but chances are that the old car will contain very little electronics beside the radio, while the modern car will have a variety of embedded computer systems on board that perform tasks ranging from controlling the brakes (ABS) to helping the driver find the way to his destination, or keep the interior temperature comfortable. In most other areas technological progress also depends on the availability of cheap computer technology that allows the designers of hardware or electronics to go beyond the possibilities of hard-wired control and provide products that offer new and fascinating possibilities. 4. Service-Oriented Systems Service-oriented systems are the state-of-the-art technology for building business applications; they will be an important part of many future software-intensive systems. In the service-oriented computing paradigm, services are understood as autonomous, platform-independent computational entities that can be described, published, discovered, and dynamically assembled for developing massively distributed, interoperable, evolvable systems. Service-oriented computing facilitates personalisation of information to individual end users and applications by means of differentiated context-aware services where the content or quality of the service is organised according to the context of its use, customer type, transaction use, location, and so on. Service-oriented computing also addresses the requirements of dynamic e-business which encompass alternative modes of user interaction (e.g., portals and voice), innovative business models (e.g., e-marketplaces and auctions), access methods (wireless services), and devices (PDAs and cellular phones). Service-oriented computing allows for multiple-step transactions to be supported across any device, e.g., web browsers, mobile PDAs, Internet appliances, in-vehicle information systems, and set-top boxes. It aids in the modelling, integration and automation of the business processes that span an organisation. The visionary promise of Service-Oriented Computing is a world of cooperating services where application components are assembled with little effort into services that can be loosely coupled to create dynamic business processes and applications that span organisations and computing platforms. Networks of cooperating services will form the basis for developing a new generation of agile applications that will directly lead to wider adoption, broader availability and ubiquitous availability of services. 5. Software-Intensive Systems All the scenarios described in the previous sections have one thing in common: they depend on software that controls the behaviour of individual components and the interaction between components, and on software which interacts with other software, systems, devices, sensors and with people. In other words: they depend on software-intensive systems. Because of the unique malleability that software offers, it is possible to build more advanced systems more cheaply and more flexibly than ever before. One example is business software, where service-oriented computing promises new and simpler ways for dynamically composing systems; services evolve from relatively simple customer services to global complex business solutions. Another example are the possibilities afforded by embedded controllers: their computational power increases exponentially while their cost and energy consumption decreases; this will even more accelerate the transition from mechanical or non-programmable electronic control systems to computers in the next decades. Hence software-intensive

5. SOFTWARE-INTENSIVE SYSTEMS

9

systems will become even more ubiquitous in the coming years. The future prosperity of European countries will to a large extend depend on their competitiveness in the development of software-intensive systems. At the current state of the art there are (at least) two main application scenarios where the issue of system complexity and emergent behaviour is starting to show: monitoring and control of large systems or environments; and exploitation, integration and composition of highly complex software systems Monitoring and Control of Large Systems or Environments. These applications are characterized by the need to collect data from environments which can also be hostile, and to partially control the environment. The solution here is to develop adaptive embedded systems, namely reactive systems embedded in complex software-intensive systems which are capable to react to the changing operating conditions (e.g., appearance or disappearance of a node, positive and negative peaks of power consumption as opposed to the time varying high and low power needs, difficult communication, ad hoc-networking as a way to survive hostile operating conditions, and so on). • Embedded systems for automotive and avionics applications. Modern cars are increasingly dependent on built-in computers (motor control, antilock system, electronic stability control, etc.) and on communication with other software-intensive systems (navigation system with GPS, automated notification of emergencies, etc.). Increasingly cars adapt to their environment, e.g., to weather and traffic conditions, and the constitution of the driver. For example, systems are being developed that analyse the eyelid movement of the car’s driver to recognise beginning fatigue and tailor the actions of the car’s safety systems accordingly. The air-traffic control network is a software-intensive system in which stationary and airborne components cooperate. • Systems controlling critical infrastructures such as electricity, transport, and weather monitoring and prediction, which have to preserve functionality even if their hardware degrades or partially fails. • The health sector is relying on software-intensive systems to support tasks such as diagnosis and surgery. Also connected with this sector are proposals for E-Homes (ambient assisted living) in which software-intensive systems serve as domestic assistance for the elderly at home. Similar systems will also be used in hospitals to improve the medical supervision and care for patients. Exploitation, integration and composition of highly complex software systems. These applications are characterised by the need to adapt in response to unforeseen changes of requirements, technology, or environment, and to integrate highly dynamic unpredictable diverse knowledge (on this issue see the Grand Challenge ”Managing Diversity in Knowledge by Adaptation”), but also by the need to integrate and compose the execution of autonomous software systems. A possible solution here is to develop seamless adaptive service systems: namely systems, embedded in human-centred environments, which meaningfully interact with humans and offer seamless adaptive services, thus hiding the complexities of their internals. • Telecommunications, wireless ad hoc systems which are decentralised and have a dynamically changing topology, with nodes constantly entering and leaving the system. The cable television system is being combined with the Internet and therefore integrated into a software-intensive system by the introduction of multi-purpose home entertainment centres.

10

1. INTRODUCTION—MOTIVATION

• Company Infrastructure. The infrastructure of large companies becomes increasingly software-intensive with increasing integration of computeraided design (CAD), manufacturing (CAM), procurement, supply management, etc. Business applications are increasingly built from distributed, dynamically assembled services and endowed with decision-making capabilities. Systems combining both aspects. Many software-intensive systems do not belong exclusively into one of those categories; instead they combine the monitoring and control aspects with exploitation, integration and composition of highly complex software systems. An example are the systems used in the health sector. On the one hand, the infrastructure of hospitals is similar to that of other large companies, on the other hand future decision-support and surgery management software will have to analyse data from a multitude of sensors in real-time to alert the surgeon to possible problems and help resolve unexpected situations.

CHAPTER 2

Vision and Grand Challenges Software-intensive systems will have an important impact on the future development of Europe’s industry. They enable companies to conduct business more efficiently, their construction will lead to employment opportunities for highly skilled workers. Software-intensive systems have the potential to improve the quality of life of Europe’s citicens by providing means for entertainment, communication, and by helping them in their personal environment. Since software-intensive systems pervade our private and business lives, their dependability is essential for our technical progress, economic success and personal activities. But despite their importance, software-intensive systems are currently not well understood. While it is possible to design and analyse individual components, there is currently no reliable way to predict the behaviour of the complete system from the behaviour of the components. Software and system engineering do not offer the theories, methodologies and tools to reliably engineer these systems. These problems already manifest themselves in various ways: Modern cars fail because of software errors; networked computers succumb to malicious attacks; mobile phones become infected by viruses. Even critical systems such as the powergrid are becoming more and more software-intensive as this is the only way to manage their growing complexity. We cannot afford to compromise reliability and quality of service of these systems. The recent example of a computer virus infecting the control software of a nuclear power plant is a chilling example of the dangers inherent in engineering systems according to the current state of the art in software development. The ubiquity of software also raises new problems for the maintenance and evolution of programs. The authors of a program cannot foresee all circumstances and every environment in which the software will be deployed or used if the software is part of a large system or interacts with an open environment. Therefore we have to design software that can adapt to different circumstances with limited, or even without, intervention by a developer. Furthermore, we cannot shut down critical infrastructure to perform software upgrades, so we have to find engineering methods that allow us to guarantee that programs can be upgraded or enhanced while retaining their particular adaptations. It is not sufficient for software modifications to be “locally correct”: the whole software-intensive system has to continue working correctly after we have so modified some of its components—even if if the environment of the program is so large that we cannot completely understand it, and even if the environment is largely not under the control of the software developer or user. Of course software-intensive systems do not only pose problems, they offer huge possibilities to reduce the cost of living and increase the quality of life. For example, e-homes providing assistance for elderly people, seem like the only way to deal with the increasing expectations and costs of our health systems in the face of an ageing population. There is no realistic alternative to software-intensive systems. The issues described above are not merely academic exercises, they are real, pressing, and costing

11

12

2. VISION AND GRAND CHALLENGES

money every day. The question is not whether we want to solve them, but how many expensive mishaps will happen before we solve them. Today’s grand challenge is to develop practically useful and theoretically well-founded principles, methods, algorithms and tools for programming and engineering reliable, secure and trustworthy future software-intensive systems throughout their whole life-cycle. 1. Knowledge, Interaction and Adaptation As Figure 1 illustrates (see Executive Summary) the development of softwareintensive systems introduces a genuinely new kind of complexity into the software development process: the complexity of knowledge, interaction and adaptation. This is not just a difference in the size of the problems the developer has to solve, but a new form of complexity that differs from and goes beyond the forms of complexity encountered in traditional software engineering. Most of the problems in developing the first generations of software came from the complexity of developing algorithms that could solve the computational problems of the program with the available resources: Software was commonly developed in a top-down fashion, programs used mostly built-in data-types like integers or reals and most software was tied to one kind of hardware and a particular operating system. As the problems solved with the computer began to grow a new kind of complexity was encountered: Apart from algorithmic problems, the sheer size of the programs and the dependencies between different parts of the programs had to be mastered. This led to new development methodologies like iterative development, and to new programming abstractions like object-orientation, software modules and components. Software developed according to these principles is in general portable to some degree: It is possible to adapt the software to different environments with a limited amount of effort. The need to develop distributed systems with concurrently operating components introduced another level of complexity where designers have to reason about conditions that can only arise because of the interactions of components performing parallel computations. Software developed in the next decades will increasingly have to face a new challenge: It will depend on a constantly changing networked environment that can no longer be controlled—or even completely understood—by the developer or user of the software. Programs will have to harness the behaviour of their environment to achieve their tasks. This behaviour will most often emerge from interactions between independently designed components and not from a single, well-specified entity. And a program will in many cases only be able to influence the behaviour of the system and obtain the information it needs to complete its tasks by interacting with other systems that may not even exist when the software is designed. A similar development can be seen in the data dimension: Whereas the first systems that were developed contained small amounts of data, current softwareintensive systems often operate on large quantities of data. However this data is mostly homogeneous and recorded in central repositories. Future systems will have to handle many diverse, dynamically changing sources of knowledge (see Figure 2), and even work with mutually inconsistent knowledge repositories. To be successful in highly dynamic systems, programs have to adapt themselves and their knowledge. They have to take into account the emergent behaviour of the system and its reaction to their actions and change their own behaviour to induce system behaviour that achieves the desired results. But in spite of this adaptation of individual components we still want the system to provide guarantees about its behaviour, such as minimal requirements or quality of service.

3. SOFTWARE QUALITY

13

To successfully adapt it will—in many cases—no longer be possible to rely on a knowledge base that is statically determined at design time. Instead the program has to possess adaptive or even self-adaptive knowledge systems that can obtain and adapt knowledge from different sources. It is not realistic to suppose that applications can rely on a generally accepted universal reference schema; instead each application will need facilities to integrate data from different sources into its local knowledge base. 2. Human-Oriented Systems Most current systems are computer-oriented: people have to change their ways of working to fit into the paradigm imposed on them by the computer, and people have to manage their data and programs mostly manually. While this situation is a burden even today, the increase in the number of software-intensive systems will soon make it untenable. Systems cannot be said to be human-oriented unless they possess at least some capability to react to the behaviour of the person interacting with the system. This does not mean that systems have to be “intelligent” to be human-oriented. A radio that senses that its user is answering the phone and thus reduces its volume can already be said to react to a person’s behaviour. But most systems cannot react in a satisfying manner so easily without having some knowledge about their surroundings and the expectations of their user. Since neither surroundings nor user expectations are likely to remain constant the need for adaptation and managing diversity in knowledge is evident for the construction of human-oriented software. Furthermore, instead of users being located at the system boundaries, we will increasingly see collaborative systems, i.e., systems that feature complex interactions between people and computers. Collaborative systems will have to take into account the non-deterministic and often non-predictable behaviour of people. The system cannot simply stop working if a human actor takes some action that the system does not expect or understand. Instead it has to adapt to the new situation. For example, a physician may alter the treatment if he is aware of facts that cause him to suppose that some malady is not an isolated incident but rather the first case of a pandemic. In these cases the hospital’s software will have to adapt to the situation and still provide the services it is supposed to deliver, even if if the treatment is not within the parameters it expects. But adaptation and management of diverse knowledge are but one step to human-oriented systems. An increasing amount of our data is stored in digital form: Where the history of our personal lives was until recently contained in tangible objects like letters or photographic prints it is increasingly found only on the hard disks of our computers in the form of image files or emails. The possibility to easily replicate these files should make it easier to preserve this data, but in practice the opposite has happened. Common backup media like CDs have limited longevity and often the failure of our personal computer implies the loss of months or years of personal data. But even if we archive the data, we cannot guarantee that software that can understand this data will be available even a few years from now. One important step in the development of truly human-oriented systems is therefore the engineering of software that enables lasting access to data, even in the face of changing hardware and software environments. 3. Software Quality “Quality” is a notion that contains many different properties of software, like reliability, security, performance, reputation or trustworthiness. These properties are not easy to achieve, even for relatively simple programs let alone for complex

14

2. VISION AND GRAND CHALLENGES

programs operating in a dynamic, unpredictable environment. Given the complexity and importance of many software-intensive systems, quality is too important to be left to chance and impossible to achieve by ad-hoc methods. It has to be based on strong theoretical foundations, enabled by the development process, and supported by tools and techniques throughout the software’s life cycle—the initial development of a program as well as its subsequent modification, adaptation and evolution. Classical techniques to achieve software quality are mostly manual: code inspections and software testing, sometimes augmented by automatic coverage analysis. Recently more powerful semi-automatic or automatic methods like abstraction techniques, model checking or theorem proving have seen more widespread use. All these techniques will continue to be improved and refined in the future and they will continue to be important for software-intensive systems as well. However all mentioned methods rely to a large part on the compositionality of the system being analysed; it is as yet not clear whether these techniques can be usefully extended to deal with the amount of adaptation and emergent properties that we will see in future software-intensive systems. A promising approach could be to identify subsystems and system properties which by their compositional behaviour can make static and dynamic analysis feasible. For example, compositional compensation techniques could be used to ensure reliability in the face of failing system components or an unpredictable environment. If, e.g., a component becomes unavailable the system will try to compensate by using a replacement or reducing service quality if no adequate replacement is available. Analysis of quality of service, analysis of expected system behaviour in a range of environment conditions, and analysis of very large-scale systems requires the introduction of novel analysis methods such as probabilistic and stochastical methods, fuzzy logics and their combination with techniques for discrete systems. 4. Beyond Conventional Software Engineering While a large number of methods and tools for engineering software-intensive systems have been developed, they suffer from some severe deficiencies: pragmatic modelling languages and techniques lack formal foundations, thus inhibiting the development of powerful analysis and development tools, while formal approaches are not well-integrated with pragmatic methods, do not scale easily to complex software-intensive systems and are often too difficult to use. Analysis and modelling techniques used by engineers in different fields are often incompatible. Current engineering methods do not adequately support systems that have to adapt to a changing, highly dynamic, unpredictable environment while satisfying stringent requirements for reliability, quality of service, security, and trust. To rise to the challenge of developing practically useful and theoretically well founded principles, methods, algorithms and tools for engineering future softwareintensive systems, we have to find novel solutions to engineering problems like modelling data and processes; building adequate system architectures; ensuring reliability, dependability and compliance; supporting interoperability; managing change and enhancing usability. But simply improving software engineering practises will not be enough for building future systems. We have to advance beyond the conventional “engineering” metaphor for software development and support engineering activities with theoretical foundations, techniques and tools that support change throughout the whole system life cycle. Specific areas of opportunity are languages, algorithms and theories for change, and fine and course grained design evolution; tools that support change by modelling, analysing and transforming evolving software systems;

4. BEYOND CONVENTIONAL SOFTWARE ENGINEERING

15

tools that support co-evolution of artifacts at various levels of abstraction; as well as theoretical foundations and methodologies for supporting change.

CHAPTER 3

Proposals for new Research Programmes Among the many promising areas for future research, the participants in this Thematic Group have identified three crucial areas: engineering adaptive softwareintensive systems; managing diversity in knowledge; and eternal software-intensive systems. We give a description of each of the programmes in the following sections.

1. Engineering Adaptive Software-Intensive Systems 1.1. Rationale. Software has become a central part of a rapidly growing range of products and services in all sectors of economic and social activity. In such software-intensive systems, software applications are required to interact, in a seamless way, with other software components, devices, sensors, and people. Software-intensive systems like these mentioned in the examples in Chapter 1.5 are becoming ever more distributed, heterogeneous and decentralised. Their size, and the complexity of the interactions within them and with people, will continue to increase. We have now reached a point where their behaviour can no longer be fully understood and their evolution anticipated. Therefore, software applications will have to be designed to operate in highly dynamic and unpredictable environments; they can no longer be engineered to compute and deliver results to predefined clients, but to induce and react to properties that emerge from their interactions with other components. Although the main problem that has challenged software engineering for more than 40 years — the so-called “software crisis” — is one of complexity, the situation today and in the foreseeable future is made worse by two facts: (1) The complexity of distribution grows exponentially with the degree of concurrency as measured by the number and intricacy of the interactions among system components. (2) New, mostly unprecedented, issues arise that induce a new kind of complexity. Some of these issues are, among others: • Partial requirements: global system requirements are rarely complete and stable — the true potential of software-intensive systems is often realised only when it is in operation; behaviour and properties of (sub)systems, their environment, and their interconnections may not be known to designers for various reasons, including lack of documentation as in legacy applications, rights protection, security reasons, inadvertent usage, or because of sheer complexity; • Unpredictability and lack of adaptability: dynamic and timing behaviour of a system and its components are unknown at design time but also at run-time, during system operation; moreover, unforeseen changes of environment, technology, operating conditions, and requirements can occur. Neither are actual engineering techniques able to cope with such change and evolution occurring at all stages of system development nor do actual systems and run-time environments support self-adaptation and self-organisation of systems. 17

18

3. PROPOSALS FOR NEW RESEARCH PROGRAMMES

• Emergent behaviour : even if individual system parts and their interconnections fulfil well-defined requirements, the integrated system does not in the sense that there is often no set of properties or purpose that can characterise the behaviour of the system as a whole; the behaviour of the system is not compositional but emergent. • Extra-functional features: the behaviour of modern software-intensive systems increasingly depends on extra-functional aspects, such as real-time properties, performance, reliability, and quality-of-service, which complicates system design considerably because such properties are difficult to maintain in unpredictable environments. • Human-Oriented Systems: with software-intensive systems becoming more prevalent it is no longer feasible for people to adjust their behaviour to the expectations of the systems with which they interact. We have to design systems that seamlessly integrate into their environment and that conform to user expectations. One example for this are systems that involve complex interactions between software components and people, not as users of the software but as players engaged in common tasks—what are sometimes called collaborative systems. Such systems are playing an increasingly important role in many areas and situations that are critical for society, such as health or medical systems. In collaborative systems the software has to care for the possibility that human actors taking part in the system may not behave as expected by the software. We use a modern hospital system to illustrate some of the features that future software-intensive systems will exhibit. Even now these systems serve several diffent purposes, from supporting examinations, treatments and surgeries to handling all kinds of administrative tasks such as operation scheduling and billing. Such systems involve people performing different kinds of roles (nursing staff, doctors, administrators) interacting with different kinds of data and equipment. We will just look at two important but very different tasks of this system: supporting surgeries and billing. Surgeries already make use of software-controlled equipment such as endoscopical systems, tomographs, and robots and equipment for minimally invasive surgeries. A few years from now surgeons will be assisted by decision-support and surgery management software. During surgery the surgery-management software controls the various monitoring devices and the decision support system makes a real-time simulation based on recording all data from the multitude of sensors that will be deployed to support the surgery. When an unforeseen or critical situation occurs the software notifies the surgeon and offers help in resolving this situation. This may take the form of searching local and remote knowledge bases to find information about similar cases, or running predictive simulations—based on the collected data—of different ways to resolve the problem. This resolution has to happen within a few seconds, and therefore may need to use a powerful grid to perform the resource intensive computations. After the surgeon has taken the decision how to proceed the system has to adapt to the new unplanned situation in order to assist in planning and executing the further progession of the surgery. A different kind of adaptation occurs frequently in the financial administration where the modalities of billing have to stay consistent with changing legal regulations and insurance conditions. In this example adaptation is made necessary by changing requirements and not by a changing environment. The situations described above cannot be matched by the current practice and foreseen developments in Software Engineering. The standard development

1. ENGINEERING ADAPTIVE SOFTWARE-INTENSIVE SYSTEMS

19

approach where systems are mainly assembled, at design time, by interconnecting basic building blocks in tightly coupled architectures does not scale to the challenge of engineering components for pervasive, ubiquitous systems which have to engage in highly unpredictable dynamic interactions. Therefore, the proposed approach is somewhat opposite to the standard practice. Emergence of unanticipated behaviour is no longer seen as a defect to be corrected by improving design time decisions, but rather as an unavoidable fact that must be handled, compensated—and if possible exploited to increase the system’s performance—during system run time, in order to scale to the level of complexity that we will be witnessing. The proposal amounts to making a paradigm shift where software components are no longer produced ab initio to be integrated in well-defined contexts but, instead, endowed with increasing levels of autonomy and adaptability that allow them to evolve, at run time, in reaction to changes taking place in the rest of the system. Typically, we will be building on top of a landscape of existing highly interconnected systems. This process is not always controlled or planned externally but induced by changes perceived in the environment in which systems are operating. The challenge is to develop methods, tools and theoretical foundations for software-intensive systems that enable effective design by handling, harnessing, controlling and using the effects of emergent system properties while ensuring required levels of extra-functional properties such as availability, what we call Engineering Adaptive Software-Intensive Systems By being able to engineer adaptive software-intensive systems we will be able to address partial requirements and unpredictability of environment, technology, operating conditions, etc. To solve the problems posed by lack of compositionality, and to preserve extra-functional features in unpredictable environments we need to understand the behaviour of complex adaptive systems and devise techniques and tools to monitor and predict their behaviour. Adaptivity is also an important component in the development of human-oriented systems, as we have already argued in Chapter 2.2 1.2. Expected Benefits (Impact). Software-intensive systems are already playing an important role in the European and international economy, and why we expect their importance to grow significantly in the coming decades. The United States—realising their importance—have already started a number or research projects for software-intensive systems. The development of software-intensive systems will have significant economic consequences; research about engineering adaptive software-intensive systems has the potential to mitigate the negative effects and to increase the positive effects of these developments. • As software-intensive systems become more widely used in economically critical areas, e.g., in the energy, transportation, finance and commercial sectors, failures of software-intensive systems can have severe consequences. Unless we manage to build reliable, resilient software-intensive systems the possibility of a large-scale power outage or huge financial losses looms over the European Industry. • On the other hand, software-intensive systems are important ingredients to solutions of many of today’s and tomorrow’s problems: As the population of European societies ages, e-health systems will become indispensable for increasing the quality of health care and public welfare. The productivity gains provided by software-intensive systems will increase the competitiveness of companies using them.

20

3. PROPOSALS FOR NEW RESEARCH PROGRAMMES

• The development and deployment of software-intensive systems will become an increasingly important part of the economy. The information technology sector is already in the process of being transformed into a service business, and this trend will continue toward system delivery. 1.3. The Main Research Objectives. The main research objective is to develop scientific foundations, methods and tools for engineering adaptive softwareintensive systems. This will require research efforts in the following areas: • Change and adaptation: how to develop software-intensive systems that can adapt to unforeseen changes of requirements, technology, environment and operating conditions, both human and technical. • Quality and trust: how systems under change can preserve required overall constraints, including those that derive from operational and normative behaviour, and achieve good quality levels that are good enough with regards to extra-functional properties; and, symmetrically, how trust and reputation can be developed starting from measured quality while guaranteeing privacy; • Components and interoperability: how components can be made to interoperate in a way that is meaningful with only partial knowledge of their environment; how to design and analyse distributed algorithms specified by local interactions between such components. • Emergence: how to understand and build systems where useful behaviour emerges as the result of interactions of independent parts. • Human-Oriented Systems: how to represent the profiles and roles of the people who will be joint players with software components in collaborative systems; how to develop software-intensive systems where people do not have to adapt their behaviour to the requirements of the computer but where computers integrate seamlessly into the task at hand. 1.4. The Research Focus. To reach the objectives described in the previous section research should be focused on the following topics: • Scientific foundations and models for adaptation: theories for adaptability and evolution of software, supporting reasoning on adaptive behaviour and extra-functional features; scientific principles underlying software composition and adaptability including (on-line) algorithms for adaptation and reconfiguration. It is crucial to observe that the development of the scientific principles, methods and tools for supporting adaptation requires the techniques of computer science to be merged with similar methods and techniques in other disciplines. • Techniques and tools for adaptation: techniques and tools for modelling, analysing, simulating, transforming, and predicting the behaviour of evolving software systems are needed. Such techniques and tools should also support co-evolution of artifacts at various levels of abstraction, from code through design to requirements. In particular, they should be able to track, quantify and control change and help to modify and store development artifacts. • Processes and methodology for supporting adaptation: research is needed for providing ways to design systems stable under perturbation and able to recognise and react to unforeseen changes, to determine which software practices are most effective in coping with high rates of change, and to develop design techniques that allow one to interleave design time and run-time activities for adaptation purposes.

1. ENGINEERING ADAPTIVE SOFTWARE-INTENSIVE SYSTEMS

21

• Language support for adaptation: present-day languages focus on static design and offer few mechanisms to support evolution. It is necessary to develop high-level languages and mechanisms that can express fine and coarse grained evolution, cope with radical changes in design, and make design knowledge available to runtime behaviour and operation. • “Black-box” adaptation: To facilitate adaptation we have to move from the situation where systems are controlled by ”brain surgery” (opening them up and changing pieces of code) to the situation where systems are modified by observing and controlling (as done in control theory) their external behaviour. In particular, the development of adaptive embedded systems requires the integration of formal methods for reactive systems with discrete and continuous control theory, leading to an adaptive design theory for hybrid systems. This is the case not only because of the need to explicitly account for resources and continuous variables (most noticeably power consumption and time) but also because control theory can provide a new methodological approach where systems are taken as such (most often as black boxes) and controlled and managed from the outside. Dually, the development of seamless adaptive service systems requires a full integration of current work in software design for causal domains (e.g., on compositional theories for system interconnection, service-oriented development, requirements engineering, inter alia) with work in organisation science and the modelling of social interactions. This will result in new logical calculi and algorithms that can take into account both the coalitional and the non-cooperative characteristics of social relations. • Mathematical models for emergence: it is necessary to develop a mathematical framework in which different models of system behaviour can be brought together, reflecting the heterogeneity of components in softwareintensive systems, in order to be able to reason about emergent properties; examples include stochastic models that can reflect unpredictable behaviour, deontic logics for normative systems that can provide models for biddable domains, besides more traditional models for hybrid systems. • Self-adaptive software-intensive systems: The objective of self-adaptive systems is to minimise the degree of explicit management necessary for construction and subsequent evolution whilst preserving the required properties and operational constraints of the system. Sound approaches are needed to support system composition on the fly and dynamically reconfigurable services, and to help manage change, especially where it is required to take place in a deployed system. These new foundations, methods and techniques should maintain the right balance between genericity and the possibility to specialise them to diverse problem domains and applications. The research should therefore aim for both generic approaches at the general system level, and a related, rich spectrum of highly effective, domain specific elaborations. The long term objectives are to develop effective theories for the adaptive engineering in several concrete settings. 1.5. Why Now? (Feasibility). Software-intensive systems are not wishful thinking they are already widely deployed and used. Europe has traditionally been a market leader in the area of embedded systems. With embedded systems becoming increasingly software-intensive, Europe has to stay at the forefront of research or we are in danger of falling behind in this important economic sector. Currently there is no feasible way to address the inherent problems of adaptation and emergent behaviour of software-intensive systems. We need to develop

22

3. PROPOSALS FOR NEW RESEARCH PROGRAMMES

theoretical foundations and practical development methods as soon as possible. Current research, e.g., on service-oriented computing, is starting to address some of these problems but we need to increase the research effort in areas such as engineering adaptive software-intensive systems, managing diversity in knowledge or eternal software-intensive systems. Europe is well-positioned to address the issues posed by software-intensive systems: The European research community has traditionally been very strong in theoretical foundations, especially in areas such as distributed systems or process calculi. Recent European research initiatives have focused on Global Computing and especially on service-oriented computing; these initiatives are providing important foundations for future research in software-intensive systems that should be exploited by the European research community. In the next 5 years it will be possible to develop solid theoretical foundations— on which future analysis and research of software-intensive systems can be built— and prototypical implementations of some of the necessary tools. 2. Managing Diversity in Knowledge 2.1. The problem—Knowledge Complexity. We are facing an unforeseen growth of the sheer volume and complexity of the data being produced. Data can be generated in “standard” industrial applications (e.g., enterprise information systems), or from large sets of sensors monitoring indoor or outdoor environments or from textual and other forms of multimedia content files memorizing (camera) pictures, movies, sound, or any combination of them generated in the Web or in intranet applications by people, the electronic press and media, public administrations and companies. This data growth in turn generates a similar growth of content1, and an even bigger growth of knowledge2. From now we talk of (complexity of) knowledge implicitly considering the knowledge itself but also the (complexity of) data and content which generate it. Several factors lead to the growing complexity of knowledge: • Size: the sheer numbers (increase in the number of knowledge producers and users, and in their production/use capabilities); • Pervasiveness: knowledge, knowledge producers and users are pervasive in space and time and at all levels, from raw sensor data up to the most abstract knowledge items; • Elusiveness: in many applications (e.g., weather, space exploration, and finance), knowledge is automatically generated and made available in streams. This type of knowledge is elusive: there is a need to detect and maintain in time correlations, aggregated or selected data, and at the same time to be able to detect fraud, intrusion, anomalous behaviour, and so on. • Time unboundedness: this problem has two aspects. On one side, new knowledge items are continuously produced and used, with no foreseeable upper bound. On the other side, many knowledge items are produced to be used indefinitely in time (e.g., data about cultural heritage, the environment and people). The notion of eternal knowledge, namely of knowledge that must survive time, will soon become a major issue; and 1By content we mean metadata codifying (to some extent) the matter or topics the data are

about. 2By knowledge we mean some complex (computerized) structure which “glues together” multiple pieces of content thus creating the body of what is known.

2. MANAGING DIVERSITY IN KNOWLEDGE

23

this will happen also at the micro-level, and for knowledge which is now scarcely digitalized (e.g., my own family records and pictures); • Distribution: knowledge, knowledge producers and users are and will stay very sparse in distribution, with a spatial and a temporal distribution; • Dynamicity: new and old knowledge items, often referenced by other knowledge items, will appear and disappear virtually at any moment in time and location in space. Size, pervasiveness, and distribution of knowledge will all be highly dynamic; • Unpredictability: the future dynamics of knowledge (size, pervasiveness, distribution) will be unknown at design time but also at run time. The situation is made worse by the fact that the complexity of knowledge grows exponentially with the number of interconnected components. New, mostly unprecedented, issues which are now rising are, among others: • Lack of knowledge: often knowledge items, their properties, their environment, and their interconnections may not be known to the designer and to the user for various reasons, such as legacy problems, rights protection, security reasons, inadvertent usage, or simply because of sheer complexity or of not being aware of the existence of certain knowledge items; • Lack of compositionality: even if individual knowledge parts and their interconnections fulfil well-defined requirements, the integrated system does not in the sense that there is no set of properties or purpose that can characterise it as a whole. Classical examples are two knowledge items which cannot be merged because they are constructed under different usage assumptions, or because their union becomes inconsistent (even if the two knowledge items are locally consistent); • Extra-functional features: the way knowledge is codified, represented and managed increasingly depends on extra-functional aspects, such as specific usage goals, real-time properties, reliability, quality-of-service, etc. These aspects complicate knowledge design considerably and motivate the need for differentiating knowledge to the point that even knowledge items that are modelling the same phenomenon may turn out to be considerably diverse. 2.2. The solution—Managing Diversity in Knowledge. 2.2.1. The state of the art. In knowledge engineering and management the “usual” approach is to take into account, at design time, the possible future dynamics. The key idea is to design a “general enough” reference representation model, expressive enough to incorporate the possible future variations in the knowledge being represented, at all levels. So far, the most common solution has been to design a global representation schema and to codify into it all the possible diverse knowledge components. Examples of this approach are the work on (relational) databases, the work on distributed databases, and, lately, the work on information integration (both with databases and ontologies)3. There are many reasons why this top-down approach has been and is still largely successful. From a technological point of view it is conceptually simple, and it is also the most natural way to extend the technology developed for relational databases and single information systems. Database technology has been successfully and extensively used in the past, and there is a lot of know-how about it. From an organizational point of view, this approach satisfies the companies’ desire to centralize and, consequently, to be in control, of their data. Finally, from a cultural point of view, this approach is very much in line with the way knowledge 3A noticeable exception was the early work on multi-databases and federated databases.

24

3. PROPOSALS FOR NEW RESEARCH PROGRAMMES

is thought of in the western culture and philosophy, and in particular with the basic principle (rooted in ancient Greek philosophy) that it must be possible to say whether a knowledge statement is (universally) true or false. Besides being rooted in our culture and studies, this property is reassuring, and also efficient from an organizational point of view in that it makes it “easy” to decide what is “right” and what is “wrong”. However, experience has shown that the top-down approach scales with difficulty in open applications, where requirements are only partially known at design time. The standard solution so far has been to handle the problems which arise during the life time of a knowledge system as part of the maintenance process. The price to be paid has various facets. The first is an increased cost of maintenance as the global schema is exponentially more complex than the knowledge parts integrated inside it. The second is a decreased life time of systems, whose quality and maintainability decades with the number of maintenance updates. The third is an increased load on the users who must take charge, at least in part, of the complexity which cannot be managed by the system. Finally, various examples exist where this approach has failed simply because people did not come to an agreement on the specifics of the unique global representation (as a different but related issue think of the complexity of any effort on building standards in the representation of specific knowledge domains, e.g., the medical domain). The more applications become open and complex, the more the top-down approach shows its limits. The top-down approach cannot be pursued beyond a certain level of openess (and unpredictable dynamics); and this point has been reached by the current knowledge complexity. 2.2.2. The solution (part I) - Managing Diversity. The approach proposed here is somewhat opposite to the top down approach. The key idea is to make a paradigm shift and to consider diversity as a feature which must be maintained and exploited and not as a defect that must be absorbed in some general schema. People, organizations, communities, populations, cultures build diverse representations of the world for a reason, and this reason lies in the local context. It is hard to say what context exactly is. However, it can be safely stated that context has many dimensions: time, location, contingent goals, short term or long term goals, personal, community, cultural or historical bias, environmental conditions, etc. These contingent motivations are what makes the specific contextual knowledge parts locally optimal and in turn, what makes diversity a feature. Instead of designing the whole knowledge integrated, with a pure a-priori effort, we propose a bottom-up approach where the different knowledge parts are kept distinct and designed independently. This leads to the following articulation of the problem of managing diversity in knowledge: (1) Locality: On one side, we have a notion of contextual, local knowledge which satisfies, in a optimal way, the (diverse) needs of the knowledge producer and knowledge user. Any piece of local knowledge can be represented as (local) theory of the world (sometimes called a context) expressed in some proper (local) representation language. Local theories are associated with a set of (local) knowledge representation (e.g., reasoning) operations which allow us to derive locally new knowledge from the existing knowledge. (2) Compatibility: On the other side, we are left with the problem of how one or more theories can be used and possibly integrated for applications different from those they have been thought for. The proposed solution is to exploit and/or enforce compatibility among a set of (local) diverse

2. MANAGING DIVERSITY IN KNOWLEDGE

25

theories expressed in their (local) diverse representation language. Compatibility is ensured via inter-theory operations which allow us to derive new knowledge (e.g., a new theory, a new fact inside an existing theory) from the existing knowledge. To provide an example, consider a definition of context as it has been applied in the formalization of the language used, (for instance in the Web) and compare it with the definition of ontology. This is particularly interesting as the notion of ontology, and its intended uses, are paradigmatic of the current dominating approach to knowledge representation. We say that (compare the text in italic): • an ontology is an interpretation of some domain which is supposed to encode a view common to a set of different parties. An ontology is built to be shared 4; • a contex t is an interpretation of some domain which is supposed to encode a view of a party. A context is built to be kept local, where local implies not shared. Three observations. The first is that a context and an ontology which model the same domain are likely to be very different as they are built to serve different goals. As a matter of fact, different people will almost always build diverse interpretations, independently of whether they are building contexts or ontologies. The second is that ontologies are in principle better as they make the exchange of information easier. However the drawback is that consensus must be reached about their contents, and this can become arbitrarily hard when dealing with the kind of complexity highlighted above. Dually and third, contexts are easy to maintain as they can be constructed with little or no consensus with the other parties. However their weakness is that the exchange of information among contexts becomes very hard, also given the fact that they encode diverse knowledge. This weakness can be recovered, up to a certain extent, by moving from an ontology-centric view of the world to a view where information is exchanged via operators on constellations of ontologies. One first, relatively simple, example of operator are the mappings among ontologies and data bases which are being studied in the latest research in peer-to-peer information systems. These mappings allow to relate the meaning on one language element in a, say, an ontology to one or more language elements in another ontology. 2.2.3. The solution (part II) - Handling Dynamics by Adaptation. The bottomup approach provides a flexible, incremental solution where diverse knowledge parts can be built and used independently, with some degree of complexity arising in their integration. A second element of complexity arises also because of (the unpredictable) dynamics, and as a consequence of the fact that agreement and knowledge integration, which are not taken into account at design time, must be built in time, during the system life time. Typically, we will build knowledge on top of a landscape of existing highly interconnected knowledge parts. The proposed solution amounts to making a second paradigm shift from the view where knowledge is mainly assembled by combining basic building blocks to a view where new knowledge is obtained by the design- or run-time adaptation of existing, independently designed, knowledge parts. Knowledge will no longer be produced ab initio, but more and more as adaptations of other, existing knowledge parts, often performed in run-time as a result of a process of evolution. This process will not always be controlled or planned externally but induced by changes perceived in the environment in which systems are embedded. 4This is the notion of ontology commonly used within the Computer Science Community. This notion is much weaker than the notion of ontology which can be found in the philosophical literature, where ontologies are given (some kind of) universal status.

26

3. PROPOSALS FOR NEW RESEARCH PROGRAMMES

2.3. Expected Benefits (Impact). The proposed ideas will allow the handling of very high levels of complexity in the management of data and knowlege, such as: • Management of distributed highly hetereogeneous data and knowledge sources, as they can be found in the Web, and also in the management of large distributed corporations. • Management and integration of continuous (and continuously changing) data flows and their integration in preexisting data and knowledge structures. Examples are embedded systems in eHealth or even environment monitoring and control. • Management in time, under changing opertaing conditions, of important data and knowledge. This will become more and more of an issue with the exponential storage, index, use and retrieval of multimedia data. This will also lead to issues related to eternal knowledge. Relevant applications are related to personal (multimedia) data, data about cultural heritage, cultural and education materials • Management of future unpredictable uses and applications of data, as we start to see in many corporations and even in the Web, with the tendency to use data generated by others for completely different purposes (see for instance folksonomies and all the communities which we start to see in the Web). 2.4. Scope and Long Term objectives (10-15 years). The challenge is to develop design methods and tools that enable effective design by harnessing, controlling and using the effects of emergent knowledge properties. This leads to developing adaptive and, when necessary, self-adaptive knowledge systems and to the proposal of developing new methods, theories, algorithms, tools and systems for knowledge engineering and management, i.e. to Managing Diversity in Knowledge by Adaptation. Both the ideas of engineering and managing diversity and of doing it by handling (unpredictable) dynamics by adaptation are quite novel and represent a paradigm shift with respect to the state of the art. Achieving these ambitious goals will require dealing with the following research topics: (1) Scientific foundations and models for diversity and its dynamics, and for adaptation; (2) Language support for representing diversity, its evolution, and the data and constructs needed in order to implement adaptation; (3) Algorithms, techniques and tools for developing, re-using and integrating diverse knowledge components, and for getting meaningful answers; (4) Processes and methodologies for the development, reuse and integration of knowledge components; (5) Knowledge-based systems, able to exploit all the results developed in the previous items and integrate them with the pre-existing technology and systems. These research topics cannot be studied only at the technological level; many organizational and social issues must also be considered. The complexity in knowledge is a consequence of the complexity resulting from globalization and the virtualization of space and time produced by the current computing and networking technology, and of the effects that this has on the organization and social structure of knowledge producers and users. Moreover, the properties of knowledge that we want to emerge must be facilitated by the “proper” organizational and social settings.

2. MANAGING DIVERSITY IN KNOWLEDGE

27

The problem of diversity could then be handled according to the following three levels, each requiring the merging of computer science with similar methods and techniques in other disciplines: (1) Representation level, dealing with all the issues related to how local and inter-theory knowledge is represented, to its semantics, and to the definition of the operations which allow to manipulate it. It is important to observe that, once one assumes that local knowledge is represented as a (local) theory of the world then all the existing theories and techniques apply unchanged. The idea is that one can reason inside one local theory as if this were the unique global theory of the world. Instead, much research is necessary in order to study how global knowledge can emerge from the interaction of local theories. In this work, ideas could be drawn from results in philosophy of language, philosophy of science, and cognitive science (e.g., the work on mental spaces, or on partitioned representations, or on logics of context). (2) Organization level, dealing with the organization and interaction of interconnected knowledge parts and peer systems producing and using them. Work at this level will require, among other things, the development of models for community building based on semantics aware sharing of knowledge, in particular with the goal to show adaptive behaviour in presence of unpredictable dynamics. In this work, ideas can be drawn from results in organization science, economics and experimental economics (e.g., the work on organizational patterns, on game theory, on rational behaviour or on sunk costs). (3) Social level, dealing with the problem of how systems (incrementally) reach agreement, thus creating (sub)communities of shared or common knowledge. A further crucial issue will be how to allow for boundary crossing among communities which possess diverse knowledge, and how to deal with the consequences that boundary crossing has on old communities and on the creation of new communities. This work will need a strong interaction with the paradigms developed within the social sciences (e.g., social networks, communities of practice). 2.5. Focus issues. The above research topics will have to concentrate on a set of focus issues, which will include: • Local vs. global knowledge. The key issue will be to find the right balance and interplay between operations for deriving local knowledge and operations which construct global knowledge, between intra-theory and inter-theory operators, and, in the semantics, between having the proper local semantics and the proper compatibility among local semantics; • Autonomy vs. coordination, namely how the peer knowledge producers and users find the right balance between their desired level of autonomy of and the need to achieve coordination with the others. • Change and adaptation, namely and mainly, how inter-theory operators will have to adapt to the changing operating conditions. This will lead to the creation of adaptive knowledge. Research has to be done for developing organization models which facilitate the combination and coordination of knowledge and which can effectively adapt to unpredictable dynamics (e.g., in the number of participants of the organization, in the active participants, in the participants’ knowledge production/ use capabilities, in the network topology, and so on);

28

3. PROPOSALS FOR NEW RESEARCH PROGRAMMES

• Quality, namely how to maintain good enough quality (e.g., good enough answers in data management or in document management and search systems). A specific area will be the study of self-certifying algorithms, which should be able to demonstrate correct answers (or answers with measurable incorrectness) in the presence of inconsistent, incomplete, or conflicting knowledge components. • Trust and reputation, namely how to build, change and adapt trust and reputation of knowledge and knowledge communities, for instance as a function of the measured quality. 2.6. Feasibility. The emerging levels of complexity call for new methodologies. However the time is ripe as certain ideas stemming mainly from peer-to-peer, agent and autonomic technologies are going somehow in the direction here proposed. They provide a firm background which, integrated with the pre-existing know-how in data and knowledge management, should produce a new generation of techniques. Europe is very well positioned given the investment done before in the above mentioned areas. Some first results at the representation level and some early forms of adaptivity, still at the representation level, which could be ready for industrial IT development and exploitation, should be achievable in the time frame of 3 years. More advanced results, also leading into the organization and social level wil require longer periods, from 5 to 7 years, with some early results maybe in 5 years. More research and theory oriented results could be obtained in the first couple of years and then continuously produced. Substantial solid results at the organization and social level will have to wait for longer time (maybe 3–4 years).

3. Eternal Software-Intensive Systems 3.1. Challenge. In the current age of the information revolution, “information” and the tools to work with it represent one of a society’s most important assets. From a cultural as well as economic point of view it is essential to enable continuous and up-to-date access to long-lived and trustworthy information services, as well as to guarantee that the corresponding information systems don’t age and break but are able to evolve. The challenge that we have identified is to organize decentralized softwareintensive systems such that they can survive in a constantly changing world. Literally, they have to run forever and must become “eternal” systems whose content and functionality can be passed from one generation to the other. As a simple example including only one system, consider a personal video recorder (PVR) that was programmed to suit the owner’s preferences: It stores many recorded programs with annotations; It organizes the content in a personal way; It will contain active rules how to search for and select new content in an automated way; It features heuristics how to complement stored programs with information (e.g. movie databases) that can be found on the Internet. What users wish is that all this information and personalized logic should be independent from the device: The PVR service should be able to persist across a series of software releases; It should be upgradable but be immune against accidental feature interactions from later third party extensions; It should also be portable from one hardware generation to the other as well as across manufacturer barriers. In other words, such an active “PVR personality” software should be future proof. Today we are not able to write software that is capable of such behavior.

3. ETERNAL SOFTWARE-INTENSIVE SYSTEMS

29

We define “eternal software-intensive systems” as software systems which can survive changes in their execution environment without (or with as little as possible) human intervention regarding their code base. Changes include e.g., new usage patterns (self-optimization), functionality upgrades (that can be added without reverse engineering the running software), new versions of libraries or of the embedding operating system (discovery and exploitation of improved functionality) and hardware replacements (portability and network context). At the end, a system must preserve its functionality as well as its configuration and meta data: an eternal software system becomes an eternal information system and vice versa. 3.2. Impact. From an end user as well as a scientist point of view it is difficult to accept that computerized systems are so brittle and short lived. Taking the expectancy of life in our western world as a benchmark, active computer systems and applications fade away (i.e., must be replaced) in 1/20 of this time; Passive digital content, if accessible at all, must be manually saved every 1/10 of the human lifespan. This translates directly into either loss of functionality and content or economic efforts to keep “things alive”: Corresponding disenchantment regarding the computer revolution will be growing quickly as people realize its high churn and decay rate. Software systems, or more precisely, their content and services, that are able to accompany humans at least for decades if not a lifespan, will be essential to keep the acceptance of ICT at a high level. Economically it is clear that being able to extend the life time of software intensive systems (which currently translates into cost intensive systems), and being able to easily extend such systems, gives a competitive advantage over those parties which have to bind resources just to keep systems running in a changing environment, or which have to redo them on a periodic basis. Finally we point out that the massive deployment of (ubiquitous) devices and services is in peril if it is not possible to reach a sustainable management level. With 100 to 1000 computerized devices per household and a service MTBF of 5 years, one would have to fix or replace 20 to 200 of them each year, which is not an acceptable market proposition. Assuming that hardware can be made to run for a decade or more, the problem currently lies on the software side. 3.3. Scope and Long Term Objectives (15 years). The challenge is to develop a framework, both theoretical as well as practical, that enables new software intensive systems to be extremely long-lived although requiring minimal intervention and management. We back this challenge by pointing to four trends in the area of software production which require a radically rethinking of software and service life cycle management: (1) From ab initio design to incremental extensions: In the long run, all new software will rely on already existing and deployed functionality. Today this is already the case at the level of libraries and tools to create software. In the future, networked systems can be designed only when interacting with existing services and systems. New functionality will always be an extensions of already existing software. Interfacing to running software without complete reverse engineering must be supported. (2) From centrally coordinated software evolution to loosely coupled software changes: With many parties extending and enriching existing systems and services with add-ons, according to their own pace, there will be no entity that can oversee the many versions and variations of a service. Software-intensive

30

3. PROPOSALS FOR NEW RESEARCH PROGRAMMES

systems must be empowered to handle interoperability, backwards compatibility and future-proofness by themselves. (3) From controlled requirement changes to combinatorial configurations: In an open environment, design decisions on software updates are taken with a local view only: How and by whom a service could be extended and combined will become more and more difficult if not impossible to anticipate. This leads to the necessity to delegate to the software essential adaption and self-configuration logic as well as to enhance software with the capabilities to work around unforeseen constraints and feature interactions with other services. (4) From human engineered solutions to automated optimization and software synthesis: We envisage that there will be a shortage and a limitation of human expertise for creating, maintaining and adapting software-intensive systems. Beyond developing better tools (for humans) to create and extend such systems, we must work towards “emancipating” software from human engineers and make it less dependable on scarce know-how and error prone hand tuning. These trends raise fundamental challenges relating not only to the methodology of software production but also, and most important, to the run-time behavior of software intensive systems themselves. The following research topics should be investigated: • Models for core software properties, e.g. in form of meta-data, that are needed to document a software’s functional, performance and implementation envelope. • Methods to analyze self-describing software and to automate the modification of software as well as its meta data according to new requirements. • Medium term transition technologies and processes where the evolution of software intensive systems is carried out in a hybrid fashion through humans as well as automated processes. • Design principles that enable the writing of long-lived software at several stages, starting from graceful degradation in face of changing conditions up to active compensation and recombination logic that enables a service to survive. • Scientific foundations for self-modifying software systems at all levels of configuration, documentation and coding, permitting to understand the automated evolution of software. • Long term innovative theories for “alive software” and “alive information” which together are able to rewrite themselves eternally. 3.4. Focus issues. Today “software portability” is the main strategy when it comes to enhancing the chances that a system can continue to operate over long time spans. Achieving portability always takes human engineers for granted and relates to coding practices and documentation standards (which also help in the short term maintenance problem) as well as the use of emulation technologies. However, we are not aware of any attempt to “self-porting” software. Fundamental research is needed in this direction, which should be concentrated on the following set of focus issues: • “Self-description”: Any software change decision, including porting considerations, relies on knowing how the software operates and what requirements it has. Today, software consists of executable code only as well as

3. ETERNAL SOFTWARE-INTENSIVE SYSTEMS

31

human readable comments and source code references for debugging purposes. An eternal software intensive system needs to have access to an internal documentation in order to avoid “self-reverse-engineering”. The key issue is to identify the description elements beyond reflection needed for automated software change processes, both static (code dependencies) as well as dynamic (run-time behavior). • A second issue is “self-observation” which is essential for a system’s adaptivity. First results on dynamic code rewriting exist (e.g. runtime software morphing) but need to be generalized beyond their restricted application areas and simplifying safety assumptions. Based on self-observation functionality there is the need for analysis and goal-preserving strategies regarding system configurations as well as code and data transformation. Also, runtime goal-checking is needed in order to let a software system evaluate and incorporate imported functionality in an automated way. • Finally, harnessing “self-modification” is a key element for letting a system “re-invent” itself, potentially making it an eternal software-intensive system. While it might be feasible to bound the effects of the adaptive configuration and transformation actions of a system mentioned above, more powerful theories are needed. Specifically, a “living software” framework should be conceived where all elements, including the self-description, selfobservation as well as the self-steering logic can be subject to adaption. 3.5. Why now? (Feasibility). Quite some experiences on keeping software intensive systems alive have been accumulated together with the insight that automated solutions must be found: This has happened to an extend such that it is now possible to draw a large picture of the research landscape and to identify the pertinent questions. We point to industry efforts like IBM’s autonomic computing initiative (2001) which produced among other things a clear framework on where research efforts have to be focused, as well as the industrial “Networked European Software & Services Initiative” (NESSI) that was launched in 2005 and which identifies “aware computers” as well as “alive software” as key evolution elements for a future ICT landscape. Due to previous research in compilers, software engineering, operating systems, performance monitoring and network management, the “self-description” and “selfobservation” focal areas are immediately accessible for exploring the automation of software modification and configuration processes that are currently carried out by humans. This means that efforts can be concentrated on the dynamics of automated software changes with a fair chance to produce strong results, including demonstrators, within 3 to 5 years. In the area of “living software”, only early results can be expected in the same time frame. This is due to the more fundamental challenges addressed here, to the banning of self-modifying code several decades ago, as well as to the new topics of networking and massive scale which make self-evolving software even more challenging. However, related research e.g., on autonomic communication and bio-inspired approaches, is existing and thus preparing the field. It is therefore important to bring more software system related viewpoints into existing research agendas and to put enough emphasis on this important basic research issue.

CHAPTER 4

Conclusions Software-Intensive Systems are already an important factor in our economic activities and our daily lives. The European infrastructure, e.g., the power grid, European companies, and European citizens all rely on Software-Intensive Systems. It is certain that they will become more wide-spread, more important and more indispensable in the future. Unless the European Union undertakes a determined effort to stay competitive, future developments in SIS will occur without European participation and with little regard to the needs of the European Union. The challenges identified during the workshop—engineering emergent behaviour via effective adaptation, eternal Software-Intensive Systems, and invisible software— could be first steps of European research towards a leading role of Europe in this strategic area.

33

APPENDIX A

Summary of Workshop Results 1. The Workshop and its Objectives The workshop on Software-Intensive Systems was part of a series of workshops of the Beyond the Horizon Coordinated Action of the European Commission. Beyond the Horizon sets out to identify the emerging trends and strategic research areas that will be the future of IST. The project is coordinated by ERCIM, the European Research Consortium for Informatics and Mathematics with the support of the Future Emerging Technologies unit of the European Commission. The workshop on “Software-Intensive Systems” took place at the University of Koblenz, Germany, on September 9–10, 2005, and was held as a co-located event of the Second International Conference on Software Engineering and Formal Methods, SEFM 2005. Participation in the workshop was by invitation only. About 20 leading experts from Europe presented and discussed future R&D directions, challenges, and visions in the emerging area of Software-Intensive Systems. The first day of the workshop consisted of a series of presentations from the participants in which each participant presented those topics and developments he or she considered most relevant for the future aims of the Beyond the Horizon Coordinated Action. On the second day the participants divided into three working groups according to the thematic areas that had been identified in the first day’s talks. 2. Challenges for Software-Intensive Systems The workshop consisted of a two-step process: At first each participant presented those topics and developments he or she considered most relevant for the future aims of the Beyond the Horizon Coordinated Action. Then the participants divided into three working groups according to the central thematic areas that had been identified in the initial presentations. In the following section we summarize the talks, the next section contains summaries of the results of the working groups. 2.1. Summary of Presentations. The presentations covered a wide area of topics, but some central themes appeared throughout the presentations and in the discussions. These are • • • • • • • • •

Change and adaption at all levels of SIS Moving to the knowledge level Emergent design Quality and trustworthiness Next generation infrastructure A science of Software-Intensive Systems A new form of complexity Autonomous sensor networks Multi-disciplinary design methods

2.1.1. Change and Adaption at all Levels. 35

36

A. SUMMARY OF WORKSHOP RESULTS

Dependable Systems Evolution. SIS are not static, and their environment is ever changing: To adapt to new requirements SIS are subject to continuous modifications, upgrades and reconfigurations; As they are communicating across global infrastructure like the Internet their environment is continuously changing. SIS have to adapt to these condition while still remaining dependable. Interoperability On-The-Fly. Many interoperability decisions that are now made at design-time will have to be postponed to the run-time of the system. This will require, e.g., dynamic assessment of components when they are brought into the system, or repositories of verified components. 2.1.2. Moving to the Knowledge Level. To design SIS we will have to move from the software level to the knowledge level, i.e., by moving from program to plan or from database schema to ontology. 2.1.3. Emergent Design. Non-Mechanical Attitude. We will no longer be able to adopt a “mechanical” attitude when developing software systems: Currently software is designed as a multi-component machine that is capable of providing the required functionality in an efficient and predictable way. The increasing complexity and dynamics of SIS will force us to deal with systems where only parts of the systems are under control of the designers and where new and different abstractions are needed. Macro-Level Modelling. In software systems consisting of a large number of autonomous and distributed components it will no longer be possible to model and analyze the behaviour of the system in terms of the behaviour of its components. It will instead be necessary to focus on the macro-level behaviour of the systems as a whole, e.g., by statistical methods. Self-Organization and Emergence. In the presence of autonomous components situated in an open environment it will no longer be possible to design software systems that exhibit specific, predictable and deterministic behaviours. It will be necessary to build systems that achieve the desired behaviour by emergence and self-organization. Adoption as First-Class Phenomenon. The emphasis on adaption requires the next generation of methods and tools to treat adaption as a first-class phenomenon to be modelled and analysed as completely as possible. Adoption should be separate from the application’s normal control flow. Many adoptions will be based on probabilistic inferences and therefore may have to be reversed (and their effects undone) when their premises become false. 2.1.4. Quality and Trustworthiness. Point- and Process-Correctness. Traditional applications and systems exhibit what one might call point-correctness: They are designed with a well-defined notion of correct behaviour. Systems that adapt to their environment will have a more complex definition of correctness. The correct behaviour in a set of circumstances must be chosen depending on a range of past, present and future conditions: the adaption must be process correct with respect to the history of the system as well as point-correct at each instant. 2.1.5. Next Generation Infrastructure. Software should be able to evolve, but still stay conformant to its creator’s purpose. This requires the software to sense, perceive and understand what the current purpose is. This requires automatic support for network software evolution as well as resilient protocol software that handles internal as well as external failures. To this end we will have to build autocatalytic software that supports continuous code rewriting and code transition instead of state transition systems. 2.1.6. A Science of Software-Intensive Systems. A science of Software-Intensive Systems has to provide theory, methods and tools for ensuring software adaptivity.

3. WORKSHOP AGENDA

37

These should enable predictability in the presence of uncertainty, characterized as the difference between average and worst-case behaviour. A science of SIS should also provide the means for coping with deviations from nominal behaviour, including errors, attacks and failures in the underlying physical platform. 2.1.7. A new Form of Complexity. Thirty years ago, the research community was faced with the problem of dealing with what we call physiological complexity: software applications that are very complex entities in the sense that they require an intricate interlacing of parts to provide the solution to a problem. The challenge that Software-Intensive Systems is raising is not so much concerned with the development of “large chunks of software” but, rather, what we call social complexity: the fact that (even very simple) software applications are being required to join, at run-time (i.e. dynamically), existing systems in which they have to interact with other entities, very often in ways that have not been planned and having to rely on heterogeneous networks of physically distributed and dynamically changing locations, connected through often-unreliable communication infrastructures. 2.1.8. Autonomous Sensor Networks. Ubiquitous computing, ubiquitous communication and intelligent user interfaces will lead to new kinds of systems, e.g., autonomous sensor networks. These are networks of countless very small, distributed, autonomous, ubiquitous and invisible micro-computer systems. These systems will lead to technical problems, such as miniaturization, energy supply, and wireless communications, and also to software problems like distributed algorithms, self-organization and scalability of large networks. Control, preprogramming and centralization will be replaced with autonomy, distributed functioning and emergence. 2.1.9. Multi-Disciplinary Design Methods. A major characteristic of embedded systems, and SIS in general, is the growing amount of software that controls or interacts with mechanical devices. As software is not subjects to the physical constraints of mechanical or electrical devices, the use of software to control systems creates almost unlimited complexity in component interactions. Software engineers lack methods to deal with constraints imposed by what should be a multi-disciplinary design and development process. The challenge is to produce theory, methods and tools for software development which guarantee that extra-functional requirements for a given platform are satisfied. 2.2. Discussion Groups. From the material discussed in the talks the participants identified three areas that are likely to hold strategic importance: engineering emergent behaviour via effective adaptation, eternal software-intensive systems and invisible software. The results of these working groups were subsequently refined into the three great challenges described in Chapter 3. 3. Workshop Agenda Friday, September 9, 2005 14:00 Martin Wirsing (LMU Munich): Welcome 14:05 Dimitris Plexousakis (ERCIM/FORTH): Presentation of the Beyond the Horizon initiative 14:15 Thomas Skordas (Future and Emerging Technologies, European Commission): FET’s vision of BTH and FP7 14:30 Presentations of Workshop participants 15:30 Coffe break 16:00 Presentations of Workshop participants 19:15 End of first day

38

A. SUMMARY OF WORKSHOP RESULTS

Saturday, September 10, 2005 8:30 9:30 12:30 14:00 14:15 16:15 16:30

Definition of working groups Working group sessions Lunch Jessica Michel (ERCIM): BTH project management Presentation and discussion of working group results Martin Wirsing: Close Departure of participants 4. Participants and CVs

4.1. Participants. Jean-Pierre Banˆ atre University of Rennes and IRISA/INRIA Rennes, France Email: [email protected] Juan Bicarregui Business and Information Technology Department, CCLRC Rutherford Appleton Laboratory, UK Email: [email protected] Ed Brinksma Embedded Systems Institute and University of Twente, The Netherlands Email: [email protected] Simon Dobson Trinity College Dublin, Ireland Email: [email protected] Peter Druschel Rice University and Max Planck Institute for Software Systems, USA and Germany Email: [email protected] Jos´ e Fiadeiro University of Leicester, UK Email: [email protected] Pierre Fraignaud CNRS and Universit´e de Paris Sud, France Email: [email protected] Fausto Giunchiglia University of Trento, Italy Email: [email protected] Manuel Hermenegildo Technical University of Madrid Email: [email protected], [email protected] Matthias H¨ olzl Ludwig-Maximilians-Universit¨ at M¨ unchen, Germany Email: [email protected] Rapporteur for TG6

4. PARTICIPANTS AND CVS

39

Stefan J¨ ahnichen Fraunhofer Institute for Computer Architecture and Software Technology (FIRST) and Technical University of Berlin, Germany Email: [email protected], [email protected] Helen Karatza Aristotle University of Thessaloniki, Greece Email: [email protected] Stephan Merz INRIA Nancy, France Email: [email protected] Jessica Michel Administrative and Financial Coordinator of the BTH project, ERCIM Email: [email protected] Dimitris Plexousakis Foundation for Research and Technology-Hellas (FORTH), Heraklion, Crete, Greece Email: [email protected] Scientific Coordinator of the BTH project Joseph Sifakis VERIMAG, Grenoble, France Email: [email protected] Thomas Skordas Future and Emerging Technologies Unit, European Commission Email: [email protected] Mikhail Smirnov Fraunhofer Institute for Open Communication Systems (FOKUS), Berlin, Germany Email: [email protected] Christian Tschudin University of Basel, Switzerland Email: [email protected] Martin Wirsing Ludwig-Maximilians-Universit¨ at M¨ unchen, Germany Email: [email protected] Thematic group coordinator (TG6) Franco Zambonelli Universit` a di Modena e Reggio Emilia, Italy Email: [email protected] 4.2. Input from. Siobhan Clark Trinity College Dublin, Ireland Email: [email protected]