Experimental: Models for cloud performance and cost analysis and optimization

As part of the EU FP7 project MODAClouds, we have been experimenting with modelling support for provider independent cloud applications within Modelio.

MODAClouds is a research project started at September 2012 and funded by the European Commission. Led by the Italian university POLIMI and under the scientific steering of the Norwegian research institute SINTEF, it aims at using a model driven approach to mitigate the problem of vendor lock-in when building cloud applications. The project provides methods, a decision support system, an open source IDE and a runtime environment for high-level design, deployment and management of cloud provider independent applications.

The present tutorial is the third one on a series to present MODAClouds approach, methodology and tools. Each tutorial presents one scenario in which modelling can simplify the development of cloud applications.

In the first tutorial, we presented the MODAClouds model-driven cloud application design approach, materialized by Creator 4Clouds. Creator 4Clouds is a Modelio module that extends UML with concepts for modelling cloud provider independent applications. Creator 4Clouds is also the main integration hub of MODAClouds tools. As displayed in the picture below, it represents the central piece integrating design and runtime tools.

MODAClouds tools

In the second tutorial, we presented the integration of Creator 4Clouds with CloudML and its approach for using models to define and enact cloud provider independent deployments.

In the present tutorial, we show the integration of Creator 4Clouds with SPACE 4Clouds. This tool has been developed by POLIMI as an extension of the Palladio Workbench and, as shown in the figure below, it produces either cost/performance estimations or multi cloud performance optimized deployment plans. In order to present SPACE 4Clouds, we will walk through the model of an actual application, the MiC (Meet in the Cloud) Application. We will see the details of this application at CCIM and CPIM levels and will use SPACE 4Clouds to evaluate its performance and cost, and to optimize its deployment.

The role of SPACE 4Clouds and Creator 4Clouds

If you need more information on the MODAClouds approach and tools, please refer to the videos and training sections of MODAClouds website.

In particular, we recommend watching our presentation at the Multi-Cloud Tutorial, available at the videos section; and the first tutorial available here.

Why models for cost and performance assessment and optimization?

With the proliferation of cloud provider offerings it has become more and more difficult to a single development team to assess the similarities and differences among all of them. Let alone, gathering info on cost structure and deployment procedures and performing actual benchmarking runs. SPACE 4Clouds reduces application assessment and optimization cost by providing a database of data on cloud provider cost and performance data, so that application models can be evaluated against this database.

It also automates the assessment of different providers, combinations of providers and deployment plans in order to find optimized parameter sets for a given application, which would be cumbersome to do manually. To do so, SPACE 4Clouds uses advanced mathematical models to consider users and application behaviour throughout the day, coupled with different cloud provider offerings in terms of VM and service types. It provides then sets of provider, resource type and size that would make room for improved infrastructure cost and preserved QoS constraints.

The MiC application in a nutshell

The MiC application (see picture below) is a web application for social networking that, during the user registration process, gathers information on user’s topics of interest. It then identifies the most similar users in the network according to the registered users’ preferences. More specifically, during the registration process, the new user provides a preference for each topic of interest in the range 1 to 5. At the end of the registration process, the MiC application calculates the Pearson coefficient for the new user based on the preferences expressed, and based on that, it identifies the users in the system with the most similar interests, and then creates a list of contacts for the newcomer.

Registered users can then log into the MiC portal and interact with other users by writing and reading posts on selected topics. Users can also change their interests refining their profiles. In this case the system reacts re-evaluating the similarity and updating the list of recommended contacts.

The application comprises two services deployed on the same virtual machine: a Webservice, to process the incoming http requests; and a database, to store users’ profiles, messages, and contact list.

The MiC application in a nutshell

In this tutorial, we will use SPACE 4Clouds in two ways:

  • to assess performance metrics of the MiC application. Metrics include CPU utilization of resources hosting the application or response time of the operations provided by the application to the user;

  • to derive an optimized deployment plan of the MiC application in the cloud environment in order to minimize the cost of the infrastructure and guarantee QoS.


For this tutorial, you need the docker file of the MODAClouds design time tools.

After running this docker file, you will have a docker container exposing a version of Modelio Open source and a version of SPACE 4Clouds along with all its dependencies. You will then be able to connect to this container either via RDP or VNC.

Modelio 3.1.2 open source can also be found here.

The models of the MiC sample application, which can be found here. The provided zip file contains all needed modules and models.

In this tutorial, we are not going to present the basics of Modelio usage, for that, refer to our Modelio specific documentation. We are not going to provide further details on MODAClouds modelling approach and concepts behind cloud independent deployment models as these concepts can be found in the first and second tutorials of this series.

Youtube Video

The content of this tutorial can also be found in the following video:

Opening the MiC project

For this tutorial, we provide a Modelio project containing all necessary modules and models for this tutorial.

  1. Import the project into your workspace.
  2. Open the imported project.

Modelling the MiC application in Creator 4Clouds

In order to correctly estimate cost and performance, SPACE 4Clouds needs an accurate model of the application structure, behaviour and of how users interact with it. The model for the MiC application in Creator 4Clouds includes a CCIM services oriented description of the application and a CPIM cloud provider independent description of the application deployment.

Defining application architecture and behaviour with CCIM models

The figure below illustrates the simplified model of the MiC application that we will consider at this tutorial. At the CCIM level, it is composed of two services, a Webserver that provides a web front end and a Database that stores the application data.

In order to obtain a model that can be analysed by SPACE 4Clouds, in addition to the definition of services and their assembly, we provided: an orchestration scenario, a usage model, and a set of QoS constraints that apply to the application.

All these elements will be explained in the following sections.

CCIM model for the MiC application

Defining application high-level architecture with the Service assembly diagram

We start by defining the services that compose the application, and the way they connect at runtime. In this figure we also show the interfaces required and provided by each component.

The Webserver service provides the MiC Interface interface, containing the operations answerQuestions(), saveAnswer() and register(). Users call these operations indirectly to get lists of questions during registration, save answers to these questions and register as a new user. In this picture, we can also see the Database service that provides the Database Interface interface containing three operations: insertUserProfile(), getTopic(), and updatePreferences(). They are used by the Webserver respectively to create a new user profile, obtain the list of topics and update user preferences.

Service assembly diagram

Modelling application operations execution time with the resource demand specification

Each operation in the database includes a resource demand specification, which is a numeric representation of the time (in ms) needed to run each operation when running on a reference machine. A reference machine is defined as a machine with processing rate of 1,000. Processing rates for different kinds of resources can be obtained from the cloud resource model (see below), so that running times can be scaled up or down in order to be fed to the CCIM model.

Processing rates in the resources model

In this example, the operations answerQuestions(), saveAnswer() and register() run respectively in 23ms, 22.7ms and 71.5ms on an ideal machine. During optimization and analysis phases, as SPACE 4Clouds considers different kinds of VM to run each service instance, resource demands are then rescaled up or down taking into consideration the capacity of the considered machines.

Resource demand specification

Defining application internal behaviour with Orchestration scenarios

An orchestration scenario defines the behaviour of part of a cloud application. In the next picture, we can see that a scenario is defined as an activity diagram which refers to action in the model. In this example, we see that the behaviour of the answerQuestions() operation is to call the getTopic() operation, the behaviour of the saveAnswer() operation is to call the updatePreferences() operation and the behaviour of the register() operation is to call the insertUserProfile() operation.

Orchestration scenario

Defining application external (user) behaviour with Usage models

A usage model defines the behaviour of the users of the application. They are also modelled as activity diagrams with references to the operations in the application as called by users. In this example, we consider a simplified usage model in which users repeatedly register themselves, answer questions and then save their answer.

Usage model

A usage model is accompanied by a workload matrix that defines for each moment of the day, the number of users that use the application and the amount of time between actions (called thinktime). In the commercial version of Modelio, a matrix is available to edit the workload of the application.

Workload specification matrix

In the open source version of Modelio, no matrix is available, one needs to fill the workloadElements tag value of the Workload stereotype of the workload model with a string composed of items in the format Population; Thinktime, separated by ;’s as below.


Defining the application deployment architecture with CPIM or CPSM models

In this tutorial, we will perform a CPIM level analysis and optimization of the model. We will consider a very simple deployment, where both services are deployed on the same node. For more information on how to build CPIM models, take a look on our CloudML tutorial.

CPIM deployment model

Defining application target QoS constraints

Finally, in this example let us consider a set of QoS constraints attached to the model elements, either at CCIM or CPIM level. The appearance of QoS constraints on Modelio’s tree view is shown below.

Overview of one constraint

The picture below illustrates all defined QoS constraints. Concerning the constraints on operations, we defined the maximum response time for register() to be 400ms, while for answerQuestion() and saveAnswer() it is of 140ms. Concerning the constraints on the cloud node, we defined its minimum RAM to be 2GB and its minimum resource type to be equivalent to m3.large.

QoS constraints

Action time: Exporting the MiC application model to SPACE 4Clouds

In order to analyse your application models, you need to export it from Creator 4Clouds and import it in SPACE 4Clouds.

  1. In order to export the Creator 4Clouds model so that it can be imported in SPACE 4Clouds, please right click on the desired CPIM or CPSM deployment model, and then select:

    Creator 4Clouds > SPACE 4Clouds > Export CPI(S)M models to SPACE 4Clouds…

  2. Create a new general Project in SPACE 4Cloud’s Eclipse.

    File > Project… > General > Project

  3. Import the files generated by Creator 4Clouds in the new project. You will also need the existing LINE.properties and localDBConnection.properties from one of the projects provided along with SPACE 4Clouds to your new project.

Basic SPACE 4Cloud configuration

To launch the SPACE 4Cloud UI click on the cloud shaped icon button in the top bar.

On the Model and Extensions tabs, select the files generated by Creator 4Clouds in the following way:

  • Repository model: the *.repository file.

  • System model: the *.system file.

  • Allocation model: the *.allocation file.

  • Resource environment model: the *.resourceenvironment file.

  • Usage model: the *X.usagemodel file. Where X is a counter starting from 1.

  • Usage model extension: the *.UsageModelExtensionX.xml file. Where X is a counter starting from 1.

  • Resource environment extension: the ResourceContainerExtension.xml file.

  • Constraints: the QoSConstraints.xml file.

  • Multi cloud extension: the MulticloudExtension.xml file.

IMPORTANT: Only one usage model can be assessed/optimized at a time, that is why usage models and their respective extensions are numbered (starting at 1). When configuring SPACE 4Clouds, make sure to use corresponding model and extension files for usage models.

Assessment start Assessment start extension

The Functionality tab allows to select either the assessment or optimization functionalities.

Application performance and cost assessment

The most basic functionality of SPACE 4Clouds is to assess a fully specified application deployment in order to evaluate how it will react to different working conditions.

For this part of the tutorial select the LINE performance engine and the Assessment operation.

Assessment start functionality

Click on Start to start the assessment process.

When the assessment is complete a window with the performance results is shown. This window reports three images, from top to bottom, they represent:

  1. the number of VMs needed to run the application through our the day,
  2. the expected average CPU utilization for each VM and
  3. the expected average response time for each functionality modelled in the application.

In the example below, we see that the MiC application requires from 1 to 30 VMs to run throughout that day, and that’s the VMs peak utilisation time is from 7h to 19h. Finally, operations' response time average on less than 300ms.

Assessment result

The Details tab reports the workload specified by the user and the computed availability of the application.

In the example below, we see that the assessed scenario considered from 1,000 to 24,000 simultaneous users, and that the application would be available all the time.

Assessment result details

Application deployment optimization

Another important functionality of SPACE 4Cloud is the automated exploration of possible deployment alternatives. It allows to find an optimized the application deployment plan to minimize costs and guarantee QoS requirements.

In order to use such functionality the user needs to load all the models discussed in the configuration section and select the Optimization operation in the Functionality tab and then clicking at the Start button.

The progress of the optimization process is shown in the Optimization window and summarized in three images. The leftmost image shows the total number of VM used to host each application tier for the entire day, the centre image shows the daily cost of the solution in USD, the rightmost image shows the number of constraints violated by the solution under analysis.

Optimization on going

Good candidate solutions, as well as the final solution, can be explored in more details by clicking on the Inspect Solution button. The best solution window explorer shows the number of VMs used to host each tier of the application on each hour of the day, the cost of the solution and the size of the cloud resource chosen. In case of deployment on multiple clouds this information is reported for all the selected cloud providers.

In the example below, SPACE 4Clouds suggests allocating from 1 to 30 VMs throughout the day on Amazon EC2 c3.large VMs. Daily provisioning cost should be of $ 32.85.

Optimization final

Related material

A longer version of this tutorial was published as a one hour tutorial available in the following Youtube video:

assesment_details.PNG (29.6 KB) Marcos Almeida, 19 October 2015 15:33

assesment.PNG (52.8 KB) Marcos Almeida, 19 October 2015 15:33

functionality_assesment.PNG (31.2 KB) Marcos Almeida, 19 October 2015 15:33

model_extension.PNG (30.9 KB) Marcos Almeida, 19 October 2015 15:33

model_specification.PNG (35.4 KB) Marcos Almeida, 19 October 2015 15:33

optimization.PNG (26.7 KB) Marcos Almeida, 19 October 2015 15:33

optimization_running.PNG (70.7 KB) Marcos Almeida, 19 October 2015 15:33

optimization_solution.PNG (30.3 KB) Marcos Almeida, 19 October 2015 15:33

solver.PNG (21.2 KB) Marcos Almeida, 19 October 2015 15:33

MiC.png (123 KB) Marcos Almeida, 19 October 2015 15:56

SAD.png (12.6 KB) Marcos Almeida, 19 October 2015 16:57

ResourceDemandSpecification.PNG (15.6 KB) Marcos Almeida, 19 October 2015 16:58

OS.png (15.5 KB) Marcos Almeida, 19 October 2015 17:05

UD.png (7.86 KB) Marcos Almeida, 19 October 2015 17:09

IDM.png (7.98 KB) Marcos Almeida, 19 October 2015 17:11

QoS.png (17.9 KB) Marcos Almeida, 19 October 2015 17:29

OneConstraint.PNG (8.03 KB) Marcos Almeida, 19 October 2015 17:34

CCIM.PNG (8.75 KB) Marcos Almeida, 19 October 2015 17:43

Workload.PNG (14.1 KB) Marcos Almeida, 19 October 2015 17:47

cloudButton.PNG (855 Bytes) Marcos Almeida, 20 October 2015 11:10

assesment.PNG (67.7 KB) Marcos Almeida, 20 October 2015 11:49

assesment_details.PNG (41.2 KB) Marcos Almeida, 20 October 2015 11:49

model_extension.PNG (46 KB) Marcos Almeida, 20 October 2015 11:49

model_specification.PNG (60 KB) Marcos Almeida, 20 October 2015 11:49

optimization_running.PNG (89.1 KB) Marcos Almeida, 20 October 2015 11:49

optimization_solution.PNG (49.4 KB) Marcos Almeida, 20 October 2015 11:49

OS.png (44.3 KB) Marcos Almeida, 20 October 2015 11:49

ResourceDemandSpecification.PNG (34.4 KB) Marcos Almeida, 20 October 2015 11:49

functionality_assesment.PNG (47.1 KB) Marcos Almeida, 20 October 2015 11:49

model_extension.PNG (51.4 KB) Marcos Almeida, 20 October 2015 11:51

functionality_assesment.PNG (52.7 KB) Marcos Almeida, 20 October 2015 11:51

ResourceModelDemands.PNG (67 KB) Marcos Almeida, 21 October 2015 11:23

ResourceModelDemands.PNG (45.4 KB) Marcos Almeida, 21 October 2015 11:32

MODAClouds_Tutorial_SPACE 4Clouds.zip (132 MB) Marcos Almeida, 21 October 2015 14:57

ChainS4C.PNG (21 KB) Marcos Almeida, 09 November 2015 17:33

MiC.PNG (19.4 KB) Marcos Almeida, 09 November 2015 17:33

ChainS4C.PNG (32.5 KB) Marcos Almeida, 09 November 2015 17:34