D5.4.3 - MODAClouds

Grant Agreement N° FP7-318484

!

Title:

Prediction and cost assessment tool – Final version

Authors:

Danilo Ardagna, Michele Ciavotta, Giovanni Paolo Gibilisco (Polimi), Juan Pérez (Imperial), Román Sosa González (ATOS)

Editor:

Danilo Ardagna, Michele Ciavotta (Polimi)

Reviewers:

Florin FORTIS (IeAT), Andrey SADOVYKH (Softeam)

Identifier:

Deliverable # D5.4.3

Nature:

Prototype

Version:

1.0

Date:

02/04/2015

Status:

Final

Diss. level:

Public

Executive Summary Cloud technologies provide tools to create dynamic systems capable to react to workload fluctuations by adapting themselves in order to keep the general performance unchanged. However, if on one hand Cloud Computing offers many advantages, on the other it introduces some important issues and new challenges in application development. In fact, the Cloud technologies and the pricing models show a high variability and are so complex to evaluate that the selection of the configuration responding to certain requirements and minimising execution costs, can result in a tremendous task. To carry on such a task the QoS (Quality of Service) engineer should consider multiple possible solutions and for each of them evaluate costs and performance. Thus, clearly there exists a serious need for analytical techniques and automatic or semi-automatic tools to support design time decisions. One of the targets of MODAClouds project is to meet that need by developing a module consisting of set of tools integrated into Creator 4Clouds (a.k.a. MODAClouds IDE). The tools are designed to interact and cooperate to each other in order to help the user to find a satisfactory configuration for the application at design time. This document accompanies the final version of this module. It is structured in such a way that two different audiences, application users and developers, are addressed. From the point of view of the first kind of users, this document is interesting as it presents a handbook of functionalities and a detailed guide for the tool installation and use. The developer, instead, would also be interested in the architectural and technological choices made in the development. Such information is also valuable for the open source community that wants to extend MODAClouds tools. Copyright © 2015 by the MODAClouds consortium – All rights reserved. The research leading to these results has received funding from the European Community's Seventh Framework Programme [FP7/2007-2013] under grant agreement n° 318484 (MODAClouds).

MODAClouds


MOdel-Driven Approach for design and execution of applications on multiple Clouds

Members of the MODAClouds consortium: Politecnico di Milano Stiftelsen Sintef Institutul E-Austria Timisoara Imperial College of Science, Technology and Medicine SOFTEAM Siemens srl BOC Information Systems GMBH Flexiant Limited ATOS Spain S.A. CA Technologies Development Spain S.A.

Italy Norway Romania United Kingdom France Romania Austria United Kingdom Spain Spain

Published MODAClouds documents These documents are all available from the project website located at http://www.modaclouds.eu/

Public Final version 1.0, Dated April 2, 2015

2

MODAClouds Deliverable # D5.4.3


Contents 1

Introduction 1.1 Context and objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Objectives of the current deliverable and their achievement . . . . . . . . . . . . . . . . 1.3 Structure of the document . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

5 5 5 6

2

SPACE 4Clouds for Dev - QoS Modelling and Analysis Tools: Overview

7

3

SPACE 4Clouds: Performance and Costs Assessment Tool 3.1 Overview of the Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Features implemented in the final release . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Improvements with respect to the initial version . . . . . . . . . . . . . . . . . . . . . .

10 10 11 12

4

LINE: Fluid performance engine 4.1 Introduction . . . . . . . . . . . . . . . . . . . 4.2 Features implemented in the final release . . . . 4.3 Improvements with respect to the initial release 4.4 encoding="UTF-8"?> c3.xlarge c3.xlarge 4

http://www.palladio-simulator.com/fileadmin/user_upload/palladio-simulator/ documents/Introduction-Chapter-PCM.pdf 5 https://github.com/deib-polimi/modaclouds-qos-models

Public Final Version 1.0, Dated 02/04/2015

72

MODAClouds MOdel-Driven Approach for design and execution of applications on multiple Clouds


Listing 1: An example of Resource Model Extension.

E.3.2

Usage Model Extension

The usage model extension XML file is a file describing the workload that we are considering for the studied application. Each hour (going from 0 to 23) has a workloadElement, composed by the information of what hour it refers, the population considered at that time instance, and the think time. In the example 2 we have a fixed think time of 10 ms, while the population is changing every hour (ranging from 1000 at 18:00 to 100 at 1:00 and 9:00).


73



Listing 2: An example of Usage Model Extension.

E.3.3

Multi Cloud Extension

The multi cloud extension file is a XML file describing how the workload is divided between the chosen providers. Each provider element is associated to a cloud provider, and has 24 workloadPartition elements (one for each hour of the day) defining the percentage of workload for the specific hour (e.g. 60 is 60%). In the example 3, the two providers, Amazon and Microsoft, have a fixed workload percentage of 60% and 40%, respectively. It is a given that for a given hour the sum of the workload percentages of all the providers should give 100%.


74


Listing 3: An example of Multi Cloud Extension.

E.3.4

Solution

The output of SPACE 4Clouds is divided into two separate files. The first one containts information on the application deployment and conforms to the schema of the resource environment extension; the second contains information related to the application performance and conforms to another schema, available in the models repository. An instance of a Performance output file is example 4.

[...] [...] [...] [...]


75



Listing 4: An example of a Performance output file.

E.3.5

Development Environment Setup

Since Space4Cloud is based on Palladio Bench which itself is a collection of Eclipse plugins, the natural IDE to develop new Space4Cloud features is Eclipse itself, in particular the Plug-in Development Environment (PDE). In order to reduce the dependencies required to build all the plugins Space4Cloud and Palladio we suggest to download Palladio Bench and use it as a starting point for the development. The source code of Space4Cloud is hosted on the github repository at https://github.com/deibpolimi/modaclouds-space4cloud. The Palladio Bench installation of Eclipse already contains the git connector so it is just sufficient to checkout the master branch of the repository. To do so click on File → Import.. then select Projects from Git, click on Next and select Clone URI. In the import window, shown in Figure 38, insert the url of the repository and click Next. Select the desired branch, usually the master, and, in the next window, select the checkout directory. Select to import the project and conclude the import operation.

Figure 38: Import the SPACE4Cloud project from github. In order to ease the development process the project is managed using Maven. Palladio Bench does not come with the maven plugin so it is necessary to install it from eclipse marketplace. Click on Help → Eclipse Marketplace and then search for the maven integration and install the Maven Integration for Eclipse. When the installation procedure is completed restart eclipse and wait for eclipse to build the workspace. In order to integrate Space4Cloud and LINE with Palladio one of they key Palladio features, the PCM Solver, has been patched. If the developer is not interested in further developing the integration the easiest solution is the follow the installation instructions in Section F.2 and instead of installing the Public Final Version 1.0, Dated 02/04/2015

76



SPACE4Cloud expand the Uncategorized category, shown in Figure 51 and install the update of the PCM Solver feature. If the developer want to modify the integration of LINE in Palladio then he/she should follow the instructions on the github repository of SPACE4Cloud in order to checkout the code of the PCM Solver plugin from https://svnserver.informatik.kit.edu/i43/svn/code/ Palladio/Core/trunk/Solver/de.uka.ipd.sdq.pcmsolver/ and apply the Patch contained in the Palladio/patch folder of the SPACE4Cloud project. Either installing the patched PCM Solver feature or importing the project and applying the patch will correct all the compilation errors. PDE allows the debug of plugins using an Eclipse Application run configuration. To create such a configuration click on Run → Run Configurations.. then double click on Eclipse Application. More information on the specification of this kind of run configuration can be found in the official PDE documentation6 . Under the arguments tab use -Dosgi.requiredJavaVersion=1.5 -Dhelp.lucene.tokenizer=standard -Xms64m -Xmx512m -XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=128M as VM arguments then launch the run configuration. A new Eclipse window will appear with the SPACE4Cloud plugin installed. These instructions have been updated and tested against the version of SPACE4Cloud tagged as D5.4.3 for more updated information please refer to the official github repository of SPACE4CLoud. SPACE 4Clouds is developed in Java, the Javadoc documentation can be found on the github repository7 .

6

http://help.eclipse.org/luna/index.jsp?topic=%2Forg.eclipse.pde.doc.user%2Fguide% 2Ftools%2Flaunchers%2Feclipse_application_launcher.htm 7 http://deib-polimi.github.io/modaclouds-space4cloud/ Public Final Version 1.0, Dated 02/04/2015

77


E.4


L INE Developer Documentation

This section provides a detailed description of L INE at the moment of writing. An up-to-date version of this documentation can be found in http://line-solver.sourceforge.net. L INE is a tool for the performance analysis of software applications, with an emphasis on cloud applications. L INE automatically builds and solves a performance model of the cloud application from a high-level description. This description is assumed to be available as a PCM instance, which includes the application architecture, the deployment characteristics, and the workload in the form of a usage model. The PCM instance can be built in the Palladio Bench tool, which also provides an implementation of the PCM2LQN transformation [6], that translates the PCM instance into a Layered Queueing Network (LQN) model [7]. L INE makes use of the LQN description obtained, in the form of an XML file, from which the performance model underlying L INE is derived. This model is subsequently solved and the relevant performance measures are computed and returned to the QoS engineer. This process is described in more detail in the next section. E.4.1

An overview of L INE

The sequence diagram in Figure 39 depicts the main steps in the operation of L INE. From version 0.5, L INE operates as a server, accepting connections from clients. In addition to the client, Figure 39 includes the four main components that make up L INE: • L INE Server: Manages the connections with the Client, and performs the calls necessary to the other components to build and solve the L INE PM. • L INE Parser: Reads the input model description and transforms it into a L INE PM. • L INE Main Solver: Receives the PM and calls a solver to obtain the steady state distribution of the PM. It uses this distribution to compute mean performance measures. It also calls other solvers to obtain other metrics such as response time distributions. • L INE Fluid Solver: Provides routines to determine the steady state distribution of a PM. It also provides routines to estimate response-time distributions for a PM. Figure 39 illustrates how a Client first establishes a connection with the L INE Server. After a connection is established, the Client can submit models for solution. A model specification is composed of two parts: a main LQN model, and an optional extension (EXT) file, both in XML format. Once L INE receives a model for solution it parses the basic LQN model first, and then parses the EXT file. Parsing the LQN model has two main steps: first parsing the LQN XML file to reconstruct the LQN model, and then obtaining the L INE performance model (PM). This PM is then extended with the definitions included in the EXT file. After these steps, the L INE PM is ready to be solved by the L INE solvers. The L INE Main Solver calls the Fluid Solver in two steps: first to find the steady state distribution of the PM, which it then uses to determine mean performance metrics; and second, to determine the response time distribution. After these step are completed, the L INE Main Solver returns the performance metrics to the L INE Server, which in turn exports them in an XML file and sends the client a “Model Solved” message. The main advantage of this operation is that L INE only needs to be initialized once to accept connections from different clients, and to solve any number of models during any connection. E.4.2

L INE classes

We now describe the main classes and scripts that make up the L INE components. As L INE is implemented in MATLAB, it not only consists of a set of classes, but also of purely procedural scripts. We divide the classes in three groups, depending on whether they are used to describe LQN models, PMs, or others. The scripts are instead divided according to the components they belong to, as listed in Figure 39. Public Final Version 1.0, Dated 02/04/2015

78



LineServer: Establish connection Connection established Submit model

Client:

LineParser:

LineMainSolver:

LineFluidSolver:

Parse model Perf. Model Parse extensions Extended Model Solve model Solve mean metrics Solve RT distribution Model solved Close connection Connection closed

Performance Metrics

Figure 39: L INE Sequence Diagram LQN classes This set of classes are used to describe LQN models, and their relations are illustrated in Figure 40. The classes are • Processor: describes a processor in the LQN model. Its properties include a list of the tasks deployed on the processor. • Task: describes a task in the LQN model. Its properties include a list of the entries and activities within the task. The activity graph and precedences describe how the activities are executed. • Entry: describes an entry in the LQN model. Its properties include a list of the activities executed when the entry is called. • Activity: describes an activity in the LQN model, which is the basic execution unit. Its properties include the mean demand of the activity on the resource where it is executed. • Precedence: provides a link between two or more activities, which are used to describe the activity graph. EXT classes These classes, depicted in in Figure 40, are used to describe two extensions to the LQN models: Coxian distributions for the processing times, and the Random Environments (RE) for reliability modeling. The classes are • RE: describes a random environment, including its stages, transition rates, reset rules, and the parameters in the LQN model affected by the RE. • COX: describes a Coxian distribution, including its states, transition rates, and completion probabilities, as well as the activities in the LQN model that follow this distribution. PM classes This set of classes, depicted in Figure 41 are used to describe the performance model (PM) underlying L INE. There are four classes that describe different versions of the PM: • CMCQNCS: describes the basic PM underlying L INE. CMCQNCS stands for Closed Multi-Class Queueing Network with Class Switching, which is the basic PM. Its description can be found in [5]. Public Final Version 1.0, Dated 02/04/2015

79



LQN classes

EXT classes

processor

task 1

entry 1

1..*

COX

1..* 1

1

precedence

RE

1..*

1..*

activity 1

1..*

Figure 40: L INE LQN and EXTClasses • CMCQNCSRE: describes a CMCQNCS extended with a random environment (RE), which is used when an RE is specified. • CMCQNCSCox: describes a CMQNCS extended to handle general (Coxian) distributions for the processing times. • CMCQNCSRECox: describes a CMCQNCS extended to handle general (Coxian) distributions for the processing times, and a random environment.

PM classes CMCQNCS

CMCQNCSRE

LINE Main class LINE obj SEFF

CMCQNCSCox

CMCQNCSRECox

Figure 41: L INE PM and Solver Classes

L INE Main class As depicted in Figure 41, the main L INE class is LINE obj SEFF, which is part of the L INE Server, and is in charge of interacting with the L INE parser to obtain the PM, and its extension, and with the L INE Main Solver to obtain the performance metrics. The LINE obj SEFF class is also in charge of implementing one of the key features of L INE: the parallel evaluation. There are three types of operation: Sequential, Parfor, and Batch Engine. The LINE obj SEFF class has these 3 modes, and one of them is used according to the configuration parameters. To support the Batch Engine mode, this class creates the connection to the local cluster, and maintains a list of the jobs processed and their status. E.4.3

L INE scripts

In addition to the set of classes described in the previous section, L INE mainly consists of a number of procedural scripts, which implement the main functionalities belong to each of the main components. L INE Server The scripts in this component are depicted in Figure 42. • LINE: This script is called to start the L INE application. It receives the location of the configuration file as argument. • LINEserver: This script manages the connections and receives the commands from the Client. Public Final Version 1.0, Dated 02/04/2015

80



• LINEprotocolXML: This script parses the commands received from the client and submits the models for solution to the L INE Main class. • writeXMLresults SEFF: This script exports the results to an XML file. It is used by the L INE Main class.

LINE Server scripts LINE

LINEserver

LINEprotocolXML

writeXMLresults SEFF

Figure 42: L INE Server Scripts

L INE Parser The scripts in this component are depicted in Figure 43. • parseXML LQN: this script reads the XML file containing the LQN description of the model to solve, and builds a set of objects using the LQN classes. • readXML CMCQNCS SEFF: this is the main script to perform the transformation from the LQN objects to the PM description, which is returned in the form of a CMCQNCS object. • readXML CMCQNCS addEntries SEFF: this script supports the previous one by recursively exploring the LQN objects to build the PM. • parseXML COX: this script parses an extension (EXT) file to build one or more COX objects, which describe the Coxian distributions to extend the LQN model. • parseXML RE: this script parses an extension (EXT) file to build one or more RE objects, which describe the Random Environment (RE) to extend the LQN model.

LINE Parser scripts parseXML LQN

readXML CMCQNCS SEFF

parseXML COX

parseXML RE

readXML CMCQNCS addEntriesP SEFF

Figure 43: L INE Server Scripts

L INE Main Solver The scripts in this component are depicted in Figure 44. • CMCQN CS analysis SEFF: this is the main script of the transformation and operates on a CMCQNCS object as PM. It calls the Fluid Solver to obtain the stationary distribution of the PM, which it uses to obtain the mean performance measures. It also calls the Fluid Solver to obtain the response time distributions.


81



• CMCQN CS RE analysis SEFF: this is a similar script as the one above, but operates on a CMCQNCSRE object as PM, thus considering an extended PM with RE. • CMCQN CS Cox analysis SEFF: this is a similar script as the first one, but operates on a CMCQNCSCox object as PM, thus considering an extended PM with Coxian distributions. • CMCQN CS Cox RE analysis SEFF: this is a similar script as the first one, but operates on a CMCQNCSRECox object as PM, thus considering an extended PM with both RE and Coxian distributions. • CMCQN CS respTime SEFF: this script supports all the other scripts above by computing performance metrics for the intermediate LQN tasks, referred to as SEFF in Palladio.

LINE Main Solver scripts CMCQN CS analysis SEFF

CMCQN CS Cox RE analysis SEFF

CMCQN CS respTime SEFF

CMCQN CS Cox analysis SEFF

CMCQN CS RE analysis SEFF

Figure 44: L INE Main Solver Scripts

L INE Fluid Solver The scripts in this component, depicted in Figure 45, can be divided in two groups: the first group, composed of the scripts on the left column in Figure 45, focuses on PM models without RE; the second group, composed of the scripts on the right column in Figure 45, focuses on PM models with RE. The scripts that solve PMs without RE are • CMCQN CS fluid ps: this script sets up the fluid model that represents the PM (CMCQNCSCox object), and solves it using MATLAB ODE solvers. Returns the stationary distribution of the PM. • CMCQN CS fluid ps RT: this script sets up a fluid model to compute response time distributions. • CMCQN CS fluid analysis: this script defines the ODE system of the fluid model, which is used by the MATLAB ODE solvers. The scripts that solve PMs with RE are defined similarly.

E.5

SLA tool Developer Documentation

E.5.1

API Introduction

The REST interface of the sla-core system has the following conventions: • Every entity is created with a POST to the collection url. The body request contains the serialized entity in the format specified in the content-type header. The location header of the response refers to the url of the new allocated resource. The return code is a 201 on success. Templates and agreements have special considerations (see the corresponding section). Public Final Version 1.0, Dated 02/04/2015

82



LINE Fluid Solver scripts CMCQN CS fluid ps

CMCQN CS RE fluid ps

CMCQN CS fluid analysis ps

CMCQN CS RE fluid analysis ps

CMCQN CS fluid ps RT

CMCQN CS RE fluid ps RT

Figure 45: L INE Fluid Solver Scripts • A query for an individual item is a GET to the url of the resource (collection url + external id). The format of the response is specified in the http header with the accept parameter. The return code is 200. As expected, a not found resource returns a 404. • Any other query is usually a GET to the collection url, using the GET parameters as the query parameters. The result is a list of entities that match the parameters, despite the actual number of entities. The return code is 200, even if the list is empty. • Any unexpected error processing the request returns a 5xx. • An entity (or list) is serialized in the response body by default with the format specified in the Content-type header (if specified). The request may have an Accept header, that will be used if the resource allows more than one Content-type. • Updating an entity involves a PUT request, with the corresponding resource serialized in the body in the format specified in the content-type header. The return code is 200. E.5.2

Generic operations

The generic operations of resources are shown below. Each particular resource (in following sections) show the supported operations, and any deviation from the behavior of generic operations. The format of a resource can be modified by a project by using serializers. GET /{resources}/{uuid} Retrieve an entity by its uuid. Request

GET /resources/{uuid} HTTP/1.1

Response in XML

HTTP/1.1 200 OK Content-Type: application/xml ...


83



Response in JSON

HTTP/1.1 200 OK Content-Type: application/json { ... }

Usage (for JSON and XML)

curl -H "Accept: application/xml" http://localhost:8080/... ...sla-service/resources/fc923960-03fe-41 curl -H "Accept: application/json" http://localhost:8080/... ...sla-service/resources/fc923960-03fe-41

GET /resources{?param1=value1¶m2=value2...} Search the resources that fulfils the parameters All resources are returned if there are no parameters. Request

GET /resources?param1=value1 HTTP/1.1

Response in XML

HTTP/1.1 200 OK Content-type: application/xml ... ... ...

Response in JSON

HTTP/1.1 200 OK Content-type: application/json [{...},{...},{...}]


curl [-X GET] -H "Accept: application/xml" localhost:8080/... ...sla-service/resources curl [-X GET] -H "Accept: application/xml" localhost:8080/... ...sla-service/resources?name=res-name curl [-X GET] -H "Accept: application/json" localhost:8080/... ...sla-service/resources curl [-X GET] -H "Accept: application/json" localhost:8080/... ...sla-service/resources?name=res-name


84



POST /resources Create a new resource. The created resource will be accessed by its uuid. Request in XML

POST /resources HTTP/1.1 Content-type: application/xml ...

Request in JSON

POST /resources HTTP/1.1 Content-type: application/json {...}

Response in XML

HTTP/1.1 201 Created Content-type: application/xml Location: /sla-service/resources/{uuid}

Request in JSON

POST /resources?param1=value1 HTTP/1.1 Content-type: application/json {"code":"xxx", "message": ...}

Usage (for JSON and XML) curl -H "Accept: ...-d@ curl -H "Accept: ...-d@

application/xml" -H "Content-type: application/xml"... -X POST localhost:8080/sla-service/resources application/json" -H "Content-type: application/json"... -X POST localhost:8080/sla-service/resources

UPDATE /resources/{uuid} Updates an existing resource. The content in the body will overwrite the content of the resource. The uuid in the body must match the one from the url o not being informed. Request in XML

PUT /resources/{uuid} HTTP/1.1 Content-type: application/xml ...

Request in JSON Public Final Version 1.0, Dated 02/04/2015

85



PUT /resources/{uuid} HTTP/1.1 Content-type: application/xml {...}

Response in XML

HTTP/1.1 200 Ok Content-type: application/xml ...

Response in JSON

HTTP/1.1 200 Ok Content-type: application/json {...}

Usage

curl -H "Accept: application/xml" -H "Content-type: application/xml"... ...-d@ -X PUT localhost:8080/sla-service/resources/{uuid} curl -H "Accept: application/json" -H "Content-type: application/json"... ...-d@ -X PUT localhost:8080/sla-service/resources/{uuid}

DELETE /resources/{uuid} Deletes an existing resource. Request

DELETE /providers/{uuid} HTTP/1.1

Response in XML and JSON

HTTP/1.1 200 Ok Content-type: application/[xml | json] ... (free text indicating that the resource has been removed)


curl -H "Accept: application/xml" -X DELETE localhost:8080/... ...sla-service/resources/fc923960-03fe-41 curl -H "Accept: application/json" -X DELETE localhost:8080/... ...sla-service/resources/fc923960-03fe-41


86


E.5.3


Providers

• Provider collection URI: /providers • Provider URI: /providers/{uuid} A provider is serialized in XML as:

fc923960-03fe-41eb-8a21-a56709f9370f provider-prueba

A provider is serialized in JSON as:

{"uuid":"fc923960-03fe-41eb-8a21-a56709f9370f", "name":"provider-prueba"}

GET /providers/{uuid} Retrieves a specific provider identified by uuid GET /providers Retrieves the list of all providers POST /providers Creates a provider. The uuid is in the file beeing send DELETE /providers/{uuid} Removes the provider identified by uuid.

E.5.4

Templates

• Templates collection URI: /templates • Template URI: /templates/{TemplateId} The TemplateId matches the TemplateId attribute of wsag:Template element when the template is created. A template is serialized in XML as defined by ws-agreement. Refer to the SLA Mediator Guide for more details about the format and the generation of the templates at each SLA level. GET /templates/{TemplateId} Retrieves a template identified by TemplateId.


87



GET /templates{?serviceIds ,providerId} The parameters are: • serviceIds: string with coma separated values (CSV) with the IDs of service that is associated to the template. • providerId: id of the provider that is offering the template. POST /templates Creates a new template. The file might include a TemplateId or not. In case of not beeing included, a uuid will be assigned. PUT /templates/{TemplateId} Updates the template identified by TemplateId. The body might include a TemplateId or not. In case of including a TemplateId in the file, it must match with the one from the url. DELETE /templates/{TemplateId} Removes the template identified by TemplateId.

E.5.5

Agreements

• Agreements collection URI: /agreements • Agreement URI: /agreement/{AgreementId} The AgreementId matches the AgreementId attribute of wsag:Agreement element when the agreement is created. Refer to the SLA Mediator Guide for more details about the format and the generation of the agreements at each SLA level. GET /agreements/{AgreementId} Retrieves an agreement identified by AgreementId. GET /agreements/ Retrieves the list of all agreements. GET /agreements{?consumerId, providerId, templateId, active} The parameters are: • consumerId: uuid of the consumer (value of Context/AgreementInitiator if Context/ServiceProvider equals “AgreementResponder”). • providerId: uuid of the provider (value of Context/AgreementResponder if Context/ServiceProvider equals “AgreementResponder”). • templateId: uuid of the template the agreement is based on. • active: boolean value (value in {1,true,0,false}); if true, agreements currently enforced are returned. Public Final Version 1.0, Dated 02/04/2015

88



POST /agreements Creates a new agreement. The body might include a AgreementId or not. In case of not being included, a uuid will be assigned. A disabled enforcement job is automatically created. DELETE /agreements/{AgreementId} Removes the agreement identified by AgreementId. GET /agreements/active Returns the list of active agreements. GET /agreements/{AgreementId}/context Only the context from the agreement identified by AgreementId is returned. Request in XML

GET -H "Accept: application/xml" /agreements/{agreement-id}/context HTTP/1.1

Request in JSON

GET -H "Accept: application/json" /agreements/{agreement-id}/context HTTP/1.1

Response in XML

HTTP/1.1 200 OK RandomClient provider02 AgreementResponder 2014-03-07T12:00:00CET template02 service02

Response in JSON

HTTP/1.1 200 OK {"AgreementInitiator":"RandomClient", "AgreementResponder":"provider02", "ServiceProvider":"AgreementResponder", "ExpirationTime":"2014-03-07T12:00:00CET", "TemplateId":"template02", "Service":"service02"}


curl -H "Accept: application/xml" http://localhost:8080/sla-service/...


89



...agreements/agreement01/context curl -H "Accept: application/json" http://localhost:8080/sla-service/... ...agreements/agreement01/context

GET /agreements/{AgreementId}/guaranteestatus Gets the information of the status of the different Guarantee Terms of an agreement. Request in XML

GET -H "Accept: application/xml" /agreements/{agreement-id}/guaranteestatus HTTP/1.1

Request in JSON

GET -H "Accept: application/json" /agreements/{agreement-id}/guaranteestatus HTTP/1.1

Response in XML

HTTP/1.1 200 OK

Response in JSON

HTTP/1.1 200 OK {"AgreementId":"agreement02", "guaranteestatus":"FULFILLED", "guaranteeterms": [{"name":"GTResponseTime", "status":"FULFILLED"}, {"name":"GTPerformance", "status":"FULFILLED"}] }


curl -H "Accept: application/xml" http://localhost:8080/sla-service/agreements/... ...agreement01/guaranteestatus curl -H "Accept: application/json" ... http://localhost:8080/sla-service/agreements/... ...agreement01/guaranteestatus

E.5.6

Enforcement Jobs

An enforcement job is the entity which starts the enforcement of the agreement guarantee terms. An agreement can be enforced only if an enforcement job, linked with it, has been previously created and started. An enforcement job is automatically created when an agreement is created, so there is no need to create one to start an enforcement. In MODAClouds, the agreement enforcement MUST be started through the ”modaclouds” resource. Public Final Version 1.0, Dated 02/04/2015

90



• Enforcement jobs collection URI: /enforcements • Enforcement job URI: /enforcements/{AgreementId} An enforcement job is serialized in XML as:

agreement02 true 2014-08-13T10:01:01CEST

An enforcement job is serialized in JSON as:

{"enabled":true, "agreement_id":"agreement02", "last_executed":"2014-08-13T10:01:01CEST"}

GET /enforcements/{AgreementId} Retrieves an enforcement job identified by AgreementId. GET /enforcements Retrieves the list of all enforcement job. POST /enforcements Creates and enforcement job. Not required anymore. The enforcement job is automatically generated when an agreement is created. PUT /enforcements/{AgreementId}/start Starts an enforcement job. Not to be used in MODAClouds. See ”modaclouds” resource. Request

PUT /enforcements/{agreement-id}/start HTTP/1.1


HTTP/1.1 200 Ok Content-type: application/[xml | json] The enforcement job with agreement-uuid {agreement-id} has started


curl -H "Accept: application/xml" -X PUT localhost:8080/sla-service/... ...enforcements/fc923960-03fe-41/start curl -H "Accept: application/json" -X PUT localhost:8080/sla-service/... ...enforcements/fc923960-03fe-41/start


91



PUT /enforcements/{AgreementId}/stop Stops an enforcement job Request

PUT /enforcements/{agreement-id}/stop HTTP/1.1


HTTP/1.1 200 OK Content-type: application/[xml | json] The enforcement job with agreement-uuid {agreement-id} has stoppped


curl -H "Accept: application/xml" -X PUT localhost:8080/sla-service/... ...enforcements/fc923960-03fe-41/stop curl -H "Accept: application/json" -X PUT localhost:8080/sla-service/... ...enforcements/fc923960-03fe-41/stop

E.5.7

Violations

• Violations collection URI: /violations • Violation URI: /violations/{uuid} A violation is serialized in XML as:

ce0e148f-dfac-4492-bb26-ad2e9a6965ec agreement04 Performance 2014-08-13T10:01:01CEST 0.09555700123360344

A violation is serialized in JSON as:

{"uuid":"e431d68b-86ac-4c72-a6db-939e949b6c1", "datetime":"2014-08-13T10:01:01CEST", "contract_uuid":"agreement07", "service_name":"ServiceName", "service_scope":"", "metric_name":"time", "actual_value":"0.021749629938806803"}

GET /violations/{uuid} Retrieves information from a violation identified by the uuid.


92



GET /violations{?agreementId, guaranteeTerm, providerId, begin, end} Parameters: • agreementId: if specified, search the violations of the agreement with this agreementId, • guaranteeTerm: if specified, search the violations of the guarantee term with this name (GuaranteeTerm[@name]), • providerId: if specified, search the violations raised by this provider. • begin: if specified, set a lower limit of date of violations to search, • end: if specified, set an upper limit of date of violations to search.

E.5.8

Business penalties

• Penalties collection URI: /penalties • Penalty URI: /penalties/uuid A penalty is serialized in XML as:

ec7fd8ec-d917-49a2-ad80-80ff9aa8269c agreement-a 2015-01-21T18:42:00CET

A penalty is serialized in JSON as:

{ "uuid":"bfc4bc66-d647-453a-b813-d130f6116daf", "datetime":"2015-01-21T18:49:00CET", "definition":{ "type":"discount", "expression":"35", "unit":"\%", "validity":"P1D" }, "agreement":"agreement-a" }

GET /penalties/{uuid} Retrieves information from a penalty identified by the uuid. GET /penalties{?agreementId,guaranteeTerm,begin,end} Parameters: • agreementId: if specified, search the penalties of the agreement with this agreementId,


93



• guaranteeTerm: if specified, search the penalties of the guarantee term with this name (GuaranteeTerm[@name]), • begin: if specified, set a lower limit of date of penalties to search, • end: if specified, set an upper limit of date of penalties to search.

E.5.9

MODAClouds

This is an special endpoint to provide specific MODAClouds operations. • Modaclouds root URI: /modaclouds PUT /modaclouds/{AgreementId}/start Request

PUT /modaclouds/{agreement-id}/start HTTP/1.1


HTTP/1.1 200 Ok The enforcement job with agreement-uuid {agreement-id} has started

E.5.10

Metrics

This is an special endpoint to receive notifications from the Monitoring Platform. • Metrics root URI: /metrics POST /metrics/{AgreementId} Request

POST /metrics/{agreement-id} HTTP/1.1


HTTP/1.1 200 Ok Content-type: text/plain Metrics received


94


E.6


SLA Mediator

E.6.1

Template generation

One template is generated in the Customer - Application Provider SLA level. Only the high level constraints are being considered, i.e. constraints that directly affects the end user. A clear example is a constraint where the Application Provider wants to offer an availability of 99.9% to the Customer. In the case of the Application Provider - Cloud Provider SLA level, one template is generated per tier, The details of the generation are: • An autogenerated UUID is assigned as TemplateId. • The context is filled with the following name="quickaddRT"> EkA9EMhoEeKON4DtRoKCMw Method ResponseTime 2000

1


95



METRIC > 2.0 qosConstraintc1Violated

the following guarantee term is generated:

RequestHandler/quickadd ResponseTime {"constraint": "qosConstraintc1Violated NOTEXISTS", "qos": {"hasMaxValue":2.0}}

A full example of generated template is:

ATOS AgreementResponder eHealth Services RequestHandler/quickadd ResponseTime {"constraint": "qosConstraint_c1_Violated NOTEXISTS", "qos": {"hasMaxValue":2.0}} RequestHandler


96



ResponseTime {"constraint": "qosConstraint_c2_Violated NOTEXISTS", "qos": {"hasMaxValue":5.0}} ResponseTime {"constraint": "qosConstraint_c3_Violated NOTEXISTS", "qos": {"hasMaxValue":1.0}}

E.6.2

Agreement generation

Several agreements can be created based on a single template, although in the context of MODAClouds, only one agreement is generated from a template in the C-AP SLA level. The agreement is a copy of the template, with some fields that are not present in the template: • the customer • the expiration time of the agreement • the id of the template which this agreement is based on

random-consumer provider AgreementResponder 2015-09-03T16:04:52GMT 59ce6aaf-49e3-4f5a-b9c5-5e63df8388c5 modaclouds RequestHandler/quickadd ResponseTime


97



{"constraint": "qosConstraint_c1_Violated EXISTS", "qos": {"hasMaxValue":2.0}} RequestHandler ResponseTime {"constraint": "qosConstraint_c2_Violated EXISTS", "qos": {"hasMaxValue":5.0}} ResponseTime {"constraint": "qosConstraint_c3_Violated EXISTS", "qos": {"hasMaxValue":1.0}}


98


F


Installation instructions

This section explains how to install the tool, it gives brief information about the pre-requisites for the installation and give pointers to the documentation relative to those. Procedures reported in the following sections have been tested on different versions of the Windows operating system including Windows XP, 7 and 8. Since this deliverable refers to a prototype implementation of the tool no procedure for the installation on a Unix/Linux platform have been included. These procedures are expected as part of future releases.

F.1

Pre-requisites

In order to store information regarding the cloud provider offers and the characteristic of cloud resources the SPACE 4Clouds tool uses a relational database. To be able to successfully run the tool it is necessary to install a RDBMS like MySQL. MySQL can be found at http://dev.mysql.com/downloads/mysql/ the tool has been tested against version 5.6. In order to import the dump of the database containing the information about the supported cloud providers it is strongly suggested to install also MySQL Workbench. Even though this is not a strict requisite the guide in Section F.2 shows the procedure to import the database using this tool. To install MySQL Workbench follow the guide at http://dev.mysql.com/downloads/ tools/workbench/ SPACE 4Clouds is distributed as a plugin of the Eclipse IDE, in particular it relays on the PalladioBench analysis tool as explained in Section 2. Palladio-Bench itself is a collection of plugins developed for Eclipse Modeling Framework. To get a copy of Eclipse Juno with a Palladio Bench 3.4 visit http://www.palladio-simulator.com/tools/download/. A free registration is required to download the tool. To run Palladio unzip the file and launch the eclipse.exe executable. LINE is written using Matlab. In order to run LINE users have to install the Matlab Compiler Runtime 8.0 (MCR) R2012b which is available at: http://www.mathworks.com/products/ compiler/mcr/

F.2

Installing and building SPACE 4Clouds

To import the database with the information about the supported cloud providers first download the dump of the database from: http://www.modaclouds.eu/software/public-deliverables/ and extract it. Start MySQL Workbench and select the Manage Import Export feature as shown in Figure 46. Select the folder in which the files of the dump have been extracted, select the schema labelled cloud,mysql, and performance schema as shown in Figure 47. Start the import process and when asked for a password insert modaclouds as in Figure 48; SPACE 4Clouds is build on top of the Palladio Modelling Framework, the current version needs Palladio 3.58 . It makes use of the analysis and the optimization feature of SPACE 4Clouds the LQNS9 or the LINE10 solver is needed. The suggested choice is LINE since it includes some advanced functionality specifically built for cloud application analysis that well integrate with SPACE 4Clouds. The current version of SPACE 4Clouds uses version 4.5.7.2 of LQNS solver and 0.6 of LINE. The installation of SPACE 4Clouds over Palladio-Bench 3.5 is performed using the install new software feature of eclipse. It can be accessed as shown in Figure 49 by the menus: Help → Install New Software.. 8

http://www.palladio-simulator.com/tools/download/ http://www.sce.carleton.ca/rads/lqns/lqn-documentation/ 10 http://code.google.com/p/line/ 9


99



Figure 46: Database import process 1/3.



100




Figure 49: SPACE 4Clouds - Install new software menu


101



Figure 50: SPACE 4Clouds - Update site configuration In the Install windows click the add a new update site by clicking the Add button, input the desired name for the updatesite (e.g. SPACE 4Clouds) and use https://raw.githubusercontent. com/deib-polimi/updateSite/master/site.xml as location, as shown in Figure 50. Select the SPACE 4Clouds feature under the SPACE 4Clouds category and click on Next, as shown in Figure 51. Eclipse will compute the required steps to perform in order to install the tool, in particular it will update the Eclipse Standard/SDK and the PCM Solver features, in order to support the evaluation with LINE also inside Palladio, and then it will install the SPACE 4Clouds feature, as shown in 52. Eclipse will now fetch the required components from the updatesite and perform the installation operations and warn the user that it is about to install unsigned content. Click OK on the warning and proceed with the installation. A restart of eclipse is suggested after the installation has been completed.


102



Figure 51: SPACE 4Clouds - Feature selection window

Figure 52: SPACE 4Clouds - Installation steps Public Final Version 1.0, Dated 02/04/2015

103


F.3


Installing L INE

To install L INE, it is enough to download the binaries and configuration files from http://code. google.com/p/line/ and place them in a location exported in the PATH system variable. To use L INE as a standalone application please refer to the installation instructions on the same website.

F.4

SLA Core Installation Guide

Requirements The requirements to install a working copy of the sla core are: • Oracle JDK ≥ 1.6 • Database to install the database schema for the service: Mysql ≥ 5.0 • Maven ≥ 3.0 Installation All commands shown here are ready to be executed from the root directory of the project (i.e., the one with the configuration.properties file) 1 - Download the project Download the project using a git client from sla core repository

$ git clone https://github.com/modaclouds/modaclouds-sla-core.git

It is recommended to use the master branch as it is always the latest released version. 2 - Creating the MySQL database From mysql command tool, create a database (with a user with sufficient privileges, like root):

$ mysql -p -u root mysql> CREATE DATABASE atossla;

Create a user:

mysql> CREATE USER atossla@localhost IDENTIFIED BY '_atossla_'; mysql> GRANT ALL PRIVILEGES ON atossla.* TO atossla@localhost; ... ... --* optional WITH GRANT OPTION;

From command prompt, create needed tables:

$ mvn compile exec:java -f sla-repository/pom.xml

Another option to create the database is execute a sql file from the project root directory: Public Final Version 1.0, Dated 02/04/2015

104



$ bin/restoreDatabase.sh

The names used here are the default values of the sla core. See configuration to know how to change the values. 3 - Importing the code into eclipse The core of the ATOSSLA has been developed using the Eclipse Java IDE, although others Java editors could be used, here we only provide the instructions about how to import the code into Eclipse. The first step is to tell Maven to create the necessary Eclipse project files executing this:

$ mvn eclipse:eclipse

The previous command is going to generate the eclipse project files: .settings, .classpath, and .project. Again, please never upload those files to the repository, it is going to deconfigure the eclipse of other developers (it is easy to fix, just an annoying waste of time). After it, from your eclipse you can import the project. Go to ”import project from file”, go to the trunk folder, and you should see the ”ATOSSLA” project ready to be imported in your Eclipse. Configuration The project is made up of five main modules: • SLA Repository • SLA Management • SLA Service • SLA Tools • SLA Personalization A configuration.properties.sample that is placed in the parent directory has to be copied to configuration.properties. Several parameters can be configured through this file. 1. tomcat.directory when building, war will be automatically copied to this directory, 2. db.* allows to configure the database username, password and name in case it has been changed from the proposed one in the section Creating the mysql database. It can be selected if queries from hibernate must be shown or not, 3. log.* allows to configure the log files to be generated and the level of information, 4. manager.enforcement.* several parameters from the enforcement can be customized, 5. service.basicsecurity.* basic security is enabled These parameters can be used to set the user name and password to access to the rest services.


105



Compiling

$ mvn install

If you want to skip tests:

$ mvn install -Dmaven.test.skip=true

The result of the command is a war in sla-service/target. The war is also copied to the directory pointed by tomcat.directory in the configuration.properties file. Running If the war was successfully copied to tomcat.directory, then start your tomcat to run the server. Alternatively, you can run an embedded tomcat:

$ bin/runserver.sh

that is just a shortcut for:

$ mvn tomcat:run -f sla-service/pom.xml

Some configuration parameters can be overriden using environment variables or jdk variables. The list of parameters overridable is: • DB DRIVER; default value is com.mysql.jdbc.Driver • DB URL; default value is ‘jdbc:mysql://${db.host}:${db.port}/${db.name} • DB USERNAME; default value is ${db.username} • DB PASSWORD; default value is ${db.password} • DB SHOWSQL; default value is ${db.showSQL} F.e., to use a different database configuration: $ $ $ $

export DB_URL=jdbc:mysql://localhost:3306/sla export DB_USERNAME=sla export DB_PASSWORD= bin/runserver.sh

Testing Check that everything is working:

$ curl http://localhost:8080/sla-service/providers

The actual address depends on the tomcat configuration. The embedded tomcat uses http://localhost:8080/slaservice/ as service root url.


106


F.5


Batch Engine Installation Guide

Considering the nature of the Batch Engine (BE) manual installation is not recommended, the users are advised to use one of the provided batch appliances. Appliances should be available for Eucalyptus and Amazon EC2. Download links can be obtained from the sub-project site11 . Manual Installation Although not recommended the BE can be manually installed from source using standard Python methods. It is recommended to use a separate virtualenv for holding an installation of BE. Retrieve the sources from Git:

git clone \ https://bitbucket.org/modaclouds-team/modaclouds-batch-engine

Create an virtualenv for the package:

virtualenv --no-site-packages ˜/be/

Install the BE from source:

# Switch to the directory containing the sources cd $BE_DIR ˜/be/bin/python setup.py install

After the installation BE is available in the virtualenv as:

˜/be/bin/batch-engine

11

https://bitbucket.org/modaclouds-team/modaclouds-batch-engine


107