Troubleshooter is reporting an alert because some clusters in the farm have too few hosts (less than 5)
The best practice is to have few clusters with many hosts, in order to make the best use of Eco4Cloud Workload Consolidation features, along with DRS and HA.
Troubleshooter suggest to merge clusters in order to improve efficiency.
Posted in: VMware vSphere
Status of the task: Complete
Title: Pilots assessment and feedback
Activities carried out during the task:
We discussed the results of task T4.2 with TIM administrators. We agreed that, when the improvements are projected on one year and on the entire TIM environment, they could lead to enormous savings: about 1600 MWh, 185,000 € and more than 500,000 kg of carbon emissions per year. This task also aimed to develop the beta version of the EcoMultiCloud software. Starting from the alpha version and exploiting the results of Tasks 4.1 and 4.2, the software was finalized, improving the implementation of some functionalities and the overall stability and reliability. The main changes on the software concern: (i) the possibility of defining custom constraints on virtual machines to be migrated and on hosts that can accommodate new workload; (ii) the refinement of algorithm implementation, exploiting the results of the simulation experiments; (iii) modifications aimed to improve the usability of the dashboard. We finally automated the “build” process, in order to quickly create and distribute new releases of the software.
Status of the task: Complete
Title: Pilots setup, deployment and execution
Activities carried out during the task:
At each data center we deployed the EcoMultiCloud local dashboard, which operates on the lower layer and manages the intra-DC workload consolidation, as well as the EcoMultiCloud orchestrator, which operates at the upper layer, with the aim of managing the global workload and performing inter-DC migrations. The business and technical goals for the TIM environment, as specified by the TIM administrators, were: reduction of energy consumption, reduction of carbon emissions, reduction of monetary cost, workload balancing among the three data centers. We executed the software in its passive mode for one week, in order to obtain a reference set of results. Then, we executed the simulation tool for two scenarios:
(1) only intra-DC migrations were enabled;
(2) both intra-DC and inter-DC migrations are enabled.
Finally, we turned the software modality from “passive” to “active”, and collected the results of a one-week period. The EcoMultiCloud software proved able to achieve a very significant performance optimization. Specifically, the reduction of daily energy consumption, with respect to the scenario existing before the activation of EcoMultiCloud, was about 463.6 kWh, equal to about 47.3%; the reduction of daily monetary cost was about 52.76 €, equal to about 47.3 %; the reduction of carbon emissions was about 143.8 kg, equal to about 47.3 %.
Status of the task: Complete
Title: Pilots design and implementation plan
Activities carried out during the task:
In this task, we defined the environment of the pilot experiments. The scenario is composed of three interconnected TIM data centers across Italy. We referred to the real-time information extracted from the TIM data centers, as the results are intended to be inherent to the specific environment chosen and configured for the pilot test. We also defined the pilot plan, which is composed of three phases: (i) execution of the EcoMultiCloud software in “passive” mode, i.e., without enabling VM migrations; (ii) use of the simulation tool of the software to analyze two different scenarios: with only intra-DC migrations enabled and with both intra- and inter-DC migrations enabled; (iii) execution of the software in “active” mode. The objective is to compute the energy, cost and carbon emission savings achieved during the pilot test, and estimate those that are achievable when activating the software with all its capabilities for one year on the entire TIM infrastructure.
Status of the task: Complete
Title: Integration and security /reliability features
Activities carried out during the task:
This task aimed to develop the alpha version of the EcoMultiCloud software. Starting from the prototype, obtained as the integration of the system modules, we enriched and upgraded the software, following the iterative and incremental development design. We added some missing features, and we also improved some aspects in order to obtain a more stable and reliable version. Our efforts were mainly focused on: user management and network management. Finally, we performed minor improvements and fixed some bugs, detected during the test phases.
Status of the task: Complete
Title: Service Level Agreement (SLA) Management
Activities carried out during the task:
We performed an extensive set of experiments to compute all the relevant business goals, for the entire environment and for the single data centers. We discussed the results were discussed with the TIM administrators, which were very impressed, but also suggested us to consider the necessity of (i) assessing the performance of the EcoMultiCloud algorithm when varying the available bandwidth and (ii) making an additional effort to reduce the number of migrations. The first aspect was analyzed by testing several values of the inter-DC bandwidth, to inspect the impact of this parameter on the business and technical goals. The second aspect was tackled by considering an improved method for selecting the VMs to migrate.
Status of the task: Complete
Title: End-to-end monitoring and tuning
Activities carried out during the task:
In cooperation with TIM administrators, we individuated the business goals that are of primary interest for the company. They are: reduction of energy consumption, reduction of monetary cost, and load balancing. We also performed a set of end-to-end monitoring and tuning activities aiming at individuate the technical constraints that need to be matched. The algorithm for workload assignment and migration was customized to take into account both technical constraints and business goals, and select the best data center for the accommodation of a new or a migrating virtual machine, i.e. the data center that can better help to achieve the goals while respecting the constraints.
Status of the task: Complete
Title: Simulation and Scenario Assessment
Activities carried out during the task:
In this task we customized the algorithms for workload assignment and redistribution starting from the characteristics and peculiarities of the multi-data center environment managed by TIM, formerly Telecom Italia. The analyzed scenario consists of three interconnected DCs, located respectively in Rozzano (MI) – DC1, Pomezia (RM) – DC2 and Bari (BA) – DC3. In this phase, we made an effort to abstract the workload distribution from the specific architectures and sizes of the three data centers and use them as a testbed for a deep analysis assessment and for the tuning and optimization of the EcoMultiCloud software.
To assess the effectiveness of the algorithm for inter-DC migration, we defined the following three scenarios: (i) No migrations scenario: the migrations among remote data centers are disabled; (ii) Random policy: the migrations among remote data centers are enabled, and the choice of the VMs to migrate is made randomly; (iii) Energy saving policy: the migrations among remote data centers are enabled, and the choice of the VMs to migrate is made in accordance with the “Energy saving policy”, which aims to select the CPU-intensive VMs for migration, because the energy consumption is strictly correlated to the CPU utilization.
Status of the task: Complete
Title: Data Analysis
Activities carried out during the task:
In this task we analysed the results of experiments on remote migrations performed among two TIM data centers. Firstly, starting from the results of Task 3.1, we defined and configured the test environment, focusing our efforts on network virtualization technology and security policies. Successively, we defined a set of experiments involving long-distance migrations among the data centers of the test environment. Exploiting the categorization of virtual machines discussed in Task 2.3, we executed the experiment with a set of virtual machines for each class of virtual machines. Finally, we analyzed the impact of migrations on the whole system and specifically on the quality of service perceived by users, aiming to evaluate some aspects such as resources utilization overhead, required bandwidth, possible downtime and effects perceived by final users.
Status of the task: Complete
Title: Run Time Environment
Activities carried out during the task:
This task aimed to define the multi-DC environment, by selecting a subset of TIM data centers. The main activities were devoted to (i) the collection of infrastructure and performances data from 15 TIM data centers, (ii) preliminary analysis on collected data, in order to individuate the main differences between the data centers in terms of infrastructure and workload, (iii) the selection of a subset of data centers among which long-distance migrations are enabled. We installed a software module to collect the data from each data center. An agent connected to the virtualization platform (vCenter) and collected the data on hosts and virtual machines. Then, we analysed the main metrics regarding the workload (CPU, RAM and Disk utilization), and the performance indexes: CPU-Ready Time, Active and Ballooned Memory and the data on server power consumption. Finally, the analysis of this data allowed us to select two data centers that are the most suitable for performing migration experiments.
Status of the task: Complete
Title: Project Coordination with EC
Activities carried out during the task:
With regards to the communication aspects, the beneficiary took care of informing the appointed Project Officers any time it was requested by the project circumstances, in occasion of the Amendment procedures to be prepared for the project to solve any issue before submitting the official requests, or for information, clarifications and updates regarding the Grant Agreement rules and conditions.
Whenever needed, interactions were also established with the PO through the dedicated functionalities provided on the Participant Portal.
Based on the internal management and quality standards defined by the beneficiary, the development of the technical reports and of the financial statements for P1 and P2 has been carried out on specific templates generated by E4C, and according to the main EC legal and administrative requirements and guidelines provided by the EASME. In particular, these templates include:
The activation of the coach, Mr. Roberto Carella, who had already supported the beneficiary during Phase 1 of the project, was prosecuted during the second period of the project, and led to a specific supporting work on the following aspects of the company business (also detailed in the coaching plan drafted at the beginning of the project):
Status of the task: Complete
Title: Project Management and Coordination
Activities carried out during the task:
COMMUNICATION ASPECTS: in the course of EcoMultiCloud project (M1-M24) and from the very beginning of the project lifetime, a collaborative atmosphere between the resources allocated to the project was established. A close interaction among the technical resources was maintained through frequent periodic internal meetings and checkpoints, in the view of providing and receiving feedback from each other and of being able to orientate the efforts /the activities towards a commonly shared direction.
A general supervision was carried out by the PM to ensure the involvement and the maintenance of a professional and proactive attitude of each resource during the whole project. During technical meetings and calls, special attention was paid by the appointed PM to the contractual aspects and the basic rules for the proper implementation of the EcoMultiCloud solution development. This allowed to respect the Contract with the EC and to successful achievement of the project goals.
The open, smooth and collaborative working environment created also supported the team during working peaks.
The monitoring and the overall supervision of the activities planned in the EcoMultiCloud work plan was also performed by E4C in relation to the involvement of the involved subcontractor LABOR.
Further details can be found in the Project Management Report submitted at M24 (Deliverable 6.1).
QUALITY CONTROL: conscious of the importance of a smooth and linear project implementation, Eco4Cloud staff and Management devoted efforts in applying adequate quality assurance processes to ensure the quality of the produced work and results. Key actions were put into practice to do it, which can be mostly reassumed as:
In order to maintain and guarantee a proper quality of the outputs produced throughout the project lifetime, the beneficiary E4C developed and used a set of templates and documents supporting the gathering of all the necessary information for the preparation of official Deliverables and reports expected as from GA.
The preparation of such documents was done through a cyclic procedure, which foresaw:
Finally, specific forms were created for the costs recording, which helped the Consortium keep an eye on the project expenditures and monitor the use of the resources consumed and remained for the project.
PROJECT PLANNING AND STATUS: the assessment of the progress of the technical work performed by the beneficiary E4C and the third parties/ subcontractors involved in the Phase 2 project was object of all the meetings and checkpoints held during the project lifetime.
The time schedule, indicating not only the list of milestones to be reached but also the interdependency among the different tasks and sub-tasks, was used as the main tool for monitoring the overall progress of the work. Percentage of completion as well as tasks slippage or any other deviation could be then assessed and corrected in time.
As partially anticipated during the first reporting period, minor changes had to be done to the EcoMultiCloud Description of Action (Annex 1) as corrective measures activated to solve the assessed difficulties.
Status of the task: Complete
Title: Commercialisation plan and Communication
Activities carried out during the task:
In T5.3, we defined a communication plan to spread the word about the EcoMultiCloud offering and strengthen the go-to-market activities. We also engaged cloud providers and system integrators to gather feedback about our proposition and fix it accordingly.
We passed then to action, implementing the communication plan in order to generate sales leads. The communication activities comprised a dedicated website for the project, an informative video about the benefits provided by the platform, a video tutorial to delve into the technicalities, a newsletter, social media activities, an advertorial on the main Italian business newspaper, and an online lead generation tool.
Status of the task: Complete
Title: Continuous Business Innovation Plan revision and IPR review
Activities carried out during the task:
In T5.2 we revised the business plan and reported the major change in the market, the partnership between VMware and Amazon AWS, a move to get more enterprises to move to Amazon AWS over other Cloud vendors, and to protect VMware’s leadership in Enterprise-level virtualization. With VMware Cloud on AWS, it will be easy for customers to operate a consistent hybrid IT environment using their existing VMware tools on AWS, and without having custom hardware, applications, or models. On the other hand, this adds up a new use case for EcoMultiCloud: the Hybrid Cloud. With VMware Cloud on AWS, a geographically dispersed data center environment can include an environment operated in the Cloud. The Data Center Transformation to the Cloud is process already started for many enterprises, so being able to offer a solution that walks with the customer in the journey is a great asset for the project.
We defined a business model to close a “complex sale” such as the EcoMultiCloud one, which is a sale that involves many buyer personas. The business model is based on a two-stage strategy: lead generation and ROI sale. We defined a lead generation strategy to generate a funnel of qualified leads with the presence of an empowered and committed internal sponsor, who will help to win the customer. The second phase is the actual sale; we provide a detailed forecast of the return on the investment to the prospect, so that EcoMultiCloud price point will be favourable to the customer.
We updated the financial forecasts of EcoMultiCloud, taking into account costs to be sustained and revenues foreseen with the defined business model.
Status of the task: Complete
Title: Customer Development Activities
Activities carried out during the task:
In T5.1 we analyzed in-depth the Corporate IT market, with a specific focus for data centers, virtualization and cloud computing. Enterprise Datacenters market is increasing steadily at a nearly 6% CAGR pace. Virtualization market is increasing at a 5% yearly pace. It is the technology used on top of physical hosts running inside datacenters, and is a de-facto standard in datacenters. The main vendors are VMware and Microsoft. Cloud Computing is the hottest market in the Enterprise-level IT, with a growth measurable with a 30% CAGR. The main vendors are Amazon, Google, Microsoft, IBM and VMware.
We defined the ideal customer for EcoMultiCloud. We defined an indirect go-to-market strategy, based on the creation of a value-added-channel, made of companies already working in the IT of large enterprises. We defined the communication plan needed to build the channel, and generate awareness and preference around EcoMultiCloud.
Finally, we listed the categories of competitors, and we analysed in depth VMware and VMTurbo, the main competitors, as they offer a subset of the features of EcoMultiCloud, while we explained EcoMultiCloud competitive differentiation through the analysis of the features that are unique of EcoMultiCloud.
Status of the task: Complete
Title: Validation Metrics and Plan
Activities carried out during the task:
The main business advantages of EcoMultiCloud solution are: Inter-Cloud management, risk mitigation, real-time adaptation and scalability. In accordance with these business advantages, the goal of this task has been to define a set of metrics and procedures to assess the capabilities of the devised solution.
The main activities in this task have been:
(ii) we individuated the quantitative and qualitative targets related to such metrics;
(iii) we defined a validation plan containing the activities and the experiments that will be performed by Eco4Cloud to assess the defined metrics and confirm the achievement of the related targets.
Moreover, we recognized that there is a clear lack of standard metrics about the efficiency of the computational components of data centers. Accordingly, we introduced a new metric, the HUE (Host Usage Effectiveness) that intends to fill this gap. This metric will be used to assess the efficiency of the strategies and policies for the assignment and redistribution of the workload among the data centers.
Status of the task: Complete
Title: Distributed Architecture Specification
Activities carried out during the task:
The objective of this task was to describe and detail the architecture of the EcoMultiCloud solution, which is logically organized in two layer:
Status of the task: Complete
Activities carried out during the task:
In this task we expanded the high-level analysis of Task 1.1 by eliciting and formalising the main system requirements. Specifically, we identified the main technical constraints, such as the capacity of single DCs and their ability to perform inter-DC migrations, the latency and bandwidth requirements of long-distance migrations, the possible downtime perceived by users, the security policies etc.
The requirements analysis has been performed following the software engineering principles. The requirements have been identified and described following a classification on three points of view.
Since EcoMultiCloud has a hierarchical architecture, the requirements were divided into two categories:
Local Requirements and Global Requirements.
Status of the task: Complete
Activities carried out during the task:
The objective of the task was to analyze the relevant business scenarios and objectives in the scenario of multi-site data centers, which are typically dislocated on a geographical scale and are remotely connected through a wide area network.
The main activities performed during the task are:
Status of the task: complete
Title: Software Integration and Testing
Activities carried out during the task:
The objective of this task is to integrate the software modules deployed in the previous tasks, specifically in tasks T2.1, T2.2 and T2.3, in order to combine them and produce a complete prototype. In particular, this task allowed us to integrate the Upper Layer and the Lower Layer, so that the policies for workload management can be automated in accordance to the business goals. During the first period, we started the integration of the modules that collect the data from single data centers and send them to the Orchestrator module of the Upper Layer, so as to enable the latter to execute the algorithms for workload distribution. In the second period, we completed the integration phase and implemented the dashboards of the System, which allow users to interact with the System and operate on it in order to control and verify System status. In particular, we implemented two dashboards, for Local and Orchestrator nodes. We tested the software devoting special attention to communications between modules, and reported in the final deliverable (D2.4) a typical flow execution, demonstrating the correctness of the overall process.
Status of the task: complete
Title: Preliminary Analysis of the Policies for Workload Allocation and Redistribution
Activities carried out during the task:
This task aimed to define different policies for the management and redistribution of the workload across the data centers, and to assess these policies through a set of simulation experiments. During the first reporting period, the activities focused on the individuation of the different policies, differentiated in accordance to the business goals, and on the definition of a set of scenarios on which each of the policies should be applied. In the second period, we completed the task by assessing some relevant performance such as: involved costs, load balancing, energy consumption, quality of service, etc. For the experiments, we adopted a Java simulator that was validated with respect to the real results obtained in the Telecom Italia data centers. The results proved the advantage of performing remote migrations in terms of energy consumption and cost reduction with respect to the basic scenario with no migrations, and also showed the additional advantage deriving from the use of the Energy/Cost Reduction Policy.
Status of the task: Complete
Title: Implementation of the LM and DCM Communication Protocols
Activities carried out during the task:
The objective of this task was to define and implement the main modules of the Communication Layer of the EcoMultiCloud platform. In particular, we defined the communication protocols that allow the nodes to interact with each other, while addressing the security issues involved in the architecture. Specifically, we individuated the security aspects regarding the development of the EcoMultiCloud solution: following the best practices in the state-of-the-art of the security field, we adopted a multi-tier architecture in order to ensure data protection and system inviolability.
The main modules implemented in the Communication Layer are:
We produced a user guide that includes an installation manual and a guide for the installation and utilization of the Communication Layer modules.
Status of the task: Complete
Title: Implementation of the DCM Data Models and Analysis Features
Activities carried out during the task:
The objective of this task was to define and implement the main modules of the Upper Layer of the EcoMultiCloud platform and, in particular, the high level consolidation Data Center Manager (DCM) module.
We deployed the VMware vApp that performs the logic and functionalities if the DCM module, in a VMWare environment. As for the implementation of the modules of the Lower Layer, we adopted the software engineering model referred to as “incremental model”, which allowed us to take advantage from aspects learned during the development of earlier versions of the software.
We also defined the software interfaces for the connections between the EcoMultiCloud Orchestrator, belonging to the Upper Layer, and the different Local Data Centers.
Specifically, the implemented modules are:
As for the modules of the Lower Layer, we produced a user guide that includes an installation manual and a guide for the installation and utilization of the Upper Layer modules.
Status of the task: Complete
Title: Implementation of the LM Data Models and Monitoring Features
Activities carried out during the task:
The objective of this task was to define and implement the main modules of the Lower Layer of the EcoMultiCloud platform, in particular, the local consolidation manager (LM) module.
We first designed and implemented the VMware vApps, deployed in a VMWare environment, which performs the functionalities of the LM module. We defined the development environment and the technology adopted for the deployment.
To support the implementation activities, we adopted a well-known software engineering model, i.e.,the “incremental model”, which allows the developers to take advantage from aspects learned during the development of earlier versions of the software.
We also produced a user guide that includes an installation manual and a guide for the installation and utilization of the Lower Layer modules.