To optimize the utilization we have created a first draft pre-configured with the initial structure of MyDBaaS Framework.

• 1. Download

  You can download the zip file or do the fork from the repository on GitHub.

• 2. Import and Run

  Use the Eclipse to import the project. Once imported you can now carry out the monitoring of metrics already implemented by the framework. You can also extend and adapt to your needs as required. To run the project you need to add a Server in the workspace used (we indicate the Apache Tomcat 7). Then add the project to the server. Now run!.

  2.1. Now you need to configure access to the database where it will be created the knowledge base. Use MySQL, you can change to another DBMS if you want.

  2.2. Access the url http://localhost:8080/mydbaas/starter. A form will appear as the first image below. You must enter the access data and then press Setup.

  2.3. The data will be checked and if they are correct the option to create the initial tables will appear, as the second image. Click on the Create button.

  2.3. The initial tables are created and the states will all green, as the third image.

  • 
  • 
  • 

  The monitoring server is running and you can get the collection of existing metrics. There is also a repository with only the MyDBaaS Framework, here.

With the server running just run the agents on desired resources. Set the context file for the agents.

• 1. Send

  The file mydbaas-agent.jar to the resource to be monitored, as well as the context file configured. You need to configure the file so or dll equivalent to operating system of the resource. Below we show an example of this configuration:

  #Example of configuration on Ubuntu 64-bit
  #Put the file libsigar-amd64-linux.so inside the directory /usr/lib/
  sudo mv [path]/libsigar-amd64-linux.so /usr/lib/

• 2. Run

  Now to run the monitoring agent access the directory that the files were placed.

  Now run the JAR file. Below we show two options:

  #First option: run and display the output on the console
  java -jar mydbaas-agent.jar context.conf
  #Second option: run and not display the output and lock the console
  nohup java -jar mydbaas-agent.jar context.conf >/dev/null 2>&1&

A metric is an entity that can be evaluated. Each metric is composed of one or more measures collected. Consequently, each metric must be clearly identified and defined his collection procedure. The collect of measures is performed repeatedly, at least once per iteration. The collected metrics are part of a profile evaluation, to be exploited in evaluating the progress and integrity of the environment. They can also be accumulated for future use of estimates and trends over resources.

To create a new metric we need to use common module:

• 1. Create a new class.

• 2. If the metric is for Virtual Machine or Host extend the class with AbstractMetric (details). If the metric is to DBMS or Database extend the class with AbstractDatabaseMetric (details).

import main.java.br.com.arida.ufc.mydbaas.common.metric.common.AbstractMetric;

public class Cpu extends AbstractMetric {
  //TODO
}

import main.java.br.com.arida.ufc.mydbaas.common.metric.common.AbstractDatabaseMetric;

public class Size extends AbstractDatabaseMetric {
  //TODO
}

The measures of the metric are represented by the fields of the class, but the class can have other fields not intended measures. So we need to identify measures, you just put the class name in front of the fields that represent measures. Why is it needed this pattern? Because this identification allows the monitoring agent automatically create the HTTP request parameters to send to the server.

import main.java.br.com.arida.ufc.mydbaas.common.metric.common.AbstractMetric;

public class Cpu extends AbstractMetric {

  private double cpuUser;
  private double cpuSystem;	
  private double cpuNice;
  private double cpuWait;
  private double cpuIdle;
  private double cpuCombined;

  //Getters and Setters
}

import main.java.br.com.arida.ufc.mydbaas.common.metric.common.AbstractDatabaseMetric;

public class Size extends AbstractDatabaseMetric {

  private double sizeUsed;

  //Getters and Setters
}

Finally It is necessary to implement the inherited method toString() and jsonToList(). In the first implementation need only return the type of metric, e.g. machine, host or database. The second is necessary to implement a conversion from a json in a list of objects of the metric.

@Override
public String toString() {
  return "database";
}

@Override
public List jsonToList(String json) {
  Gson gson = new Gson();
  List sizeList = gson.fromJson(json, new TypeToken>(){}.getType());
  return sizeList;
}

Now we have a new metric is necessary to create a collector for the monitoring agent and a receiver on the server.

A new collector is necessary to change the monitoring agent. A collector is used to collect information, which will then be used in a specific system on demand. Each collector is responsible for collecting the data of a particular metric. It has a cycle that determines how often the collection will be performed.

To create a new collector is required:

First step: in this step we will create an extension of the metric.

• 1. Create a new class. To maintain a patterning the name should be the class name of the metric plus Metric.

• 2. It should extend the class of the new metric created earlier and implement the LoadMetric interface, details.

• 3. With the created class implements the Singleton pattern. Why do I need this? Thus each metric will only have an instantiated object while the agent is active. Using less memory and reducing overhead.

import main.java.br.com.arida.ufc.mydbaas.agent.entity.common.LoadMetric;

public class SizeMetric extends Size implements LoadMetric {

  private static SizeMetric uniqueInstance;

  private SizeMetric() {}

  public static SizeMetric getInstance() {
    if (uniqueInstance == null) {
      uniqueInstance = new SizeMetric();
    }
    return uniqueInstance;
  }

  @Override
  public void loadMetricProperties(Properties properties) {
    //TODO
  }
}

In loadMetricProperties() method is where the configuration properties of the metric, as url and cycle, are loaded. You can configure new properties if necessary. In the metrics related to the database you need to upload identifiers which DBMSs and/or instances of databases that will be collected.

Second step: in this step we will create the collector.

• 1. Create a new class. Following the pattern the name should be the class name of the metric plus Collector.

• 2. It should extend the abstract class AbstractCollector, details. And as parameter of the extension pass the class created above.

import main.java.br.com.arida.ufc.mydbaas.agent.collector.common.AbstractCollector;

public class SizeCollector extends AbstractCollector<SizeMetric> {

  public SizeCollector(int identifier, String type) {
    super(identifier, type);
  }

  @Override
  public void loadMetric(Object[] args) throws Exception {
    // TODO Auto-generated method stub    
  }

  @Override
  public void run() {
    // TODO Auto-generated method stub    
  }
}

The class will inherit three methods. The first is the default constructor of the collector, it is not necessary to do anything. The second is the loadMetric(): in this method should be implemented the way collect the new metric. The third is the run() which is the heart of the collector. In this method is implemented the operation of the collector. Inside it called all other methods needed to collect and send to the server. Your new class also inherits two other methods that are already implemented. They are logMetric() and sendMetric(), details. One to save a log of the metric case an error occurs preventing the sending to the server and the other to send the metric for the server, respectively.

Here we show abstractly calls within the run method:

@Override
public void run() {

  //First collected metrics
  this.loadMetric(new Object[] {...});

  //Second creates the request parameters to send to the server
  List<NameValuePair> parameters = null;
  parameters = this.loadRequestParams(...);

  //Third sends the collected metric to the server
  HttpResponse response;
  response = this.sendMetric(parameters);
}

The method logMetric() should be used in error checking of the methods loadRequestParams() and sendMetric(). Here you can see a real implementation. Now we have a new collector is necessary to create a receiver on the server.

Assuming that a new metric was created following the step creating a new metric. Then also created a collector following step creating a new collector. A receiver is a method that receives the measures collected from a particular metric sent by a collector. Measures are sent via HTTP Post requests and stored in a knowledge base through the metric repository.

To create a new receiver is required:

First step: in this step we will create the receiver.

• 1. Create a new class. To maintain a patterning the name should be the type of the metric plus ReceiverController, in this case the metric is related to the machine. If the receiver of a particular type of metric already exists, skip to step two.

• 2. It should extend the abstract class AbstractReceiver, details.

• 3. With the implemented class, use the @Resource annotation of VRaptor framework. Resource is what we might think of as resources to be made ​​available for access by our collectors. Once you insert this annotation in class, all public methods of it become accessible through calls GET or POST to specific URIs.

• 4. Finally use the @Path annotation on the class, also of VRaptor. Allows the customization of URIs for accessing the methods. The parameter should be the name of the metric type of the receiver, in this case machine. With that all URIs from the receiver will be identified by the type of metric.

import br.com.caelum.vraptor.Path;
import br.com.caelum.vraptor.Resource;
import br.com.caelum.vraptor.view.DefaultStatus;
import main.java.br.com.arida.ufc.mydbaas.core.repository.MetricRepository;
import main.java.br.com.arida.ufc.mydbaas.core.controller.receiver.common.AbstractReceiver;

@Resource
@Path("/machine")
public class MachineReceiverController extends AbstractReceiver {

  public MachineReceiverController(DefaultStatus status, MetricRepository metricRepository) {
    super(status, metricRepository);
    // TODO Auto-generated constructor stub
  }
}

Second step: In this step we will create the method that receives the metric.

• 1. Create a new method with the same name as the new metric. The method must be public and void.

• 2. This new method can have three variations: when is virtual machine or host, or DBMS/database instance (as examples below). By default the method should contain a parameter that represents what metric the method receives, the name must be metric. The second parameter is a string that represents the exact time when the metric was collected, the name should be recordDate. The last parameter is the resource which the metric belongs and must be of type integer. This parameter varies with the kind of the metric. If the metric is about Virtual Machine the name should be machine and if it is about Host the name should be host. If the metric is about DBMS/database instance you need to add two integer fields, dbms and database. This happens because this type of metric can be collected at both resource levels.

• 3. Use the @Path annotation on the method to identify its URI. The parameter should be the name of the metric, which method is responsible.

• 4. The implementation is simple, just use the inherited object metricRepository and call the method saveMetric(), details.

• 5. The method saveMetric() returns a boolean. Check and if the return is true use the inherited object status and call the method accepted(). This method will return Accepted Status (202) to the collector who made the request.

@Post("/memory")
public void memory(Memory metric, int machine, String recordDate) {
  try {
    if (repository.saveMetric(metric, recordDate, machine, 0, 0, 0)) {
      status.accepted();
    }
  } catch (NoSuchMethodException e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
  } catch (IllegalAccessException e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
  } catch (InvocationTargetException e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
  }
}

@Post("/size")
public void size(Size metric, int dbms, int database, String recordDate) {
  try {
    if (repository.saveMetric(metric, recordDate, 0, 0, dbms, database)) {
      status.accepted();
    }
  } catch (NoSuchMethodException e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
  } catch (IllegalAccessException e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
  } catch (InvocationTargetException e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
  } 
}

@Post("/hostdomains")
public void hostDomains(HostDomains metric, int host, String recordDate) {
  try {
    if (repository.saveMetric(metric, recordDate, 0, host, 0, 0)) {
      status.accepted();
    }
  } catch (NoSuchMethodException e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
  } catch (IllegalAccessException e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
  } catch (InvocationTargetException e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
  } 
}

Here you can see real implementations.

The context file has the properties of a monitoring agent must perform on resources. This file is loaded by the agent that from the information automatically configures itself. This file contains information such as the URL of the server, the identifier of the resource, the resource type and configurations of the metrics.

1. The first information is about the server and the monitored resource:

# Unique code to identify the resource on the server
identifier =

# Resource type
type =

# Server URL
server =

The identifier field must have the resource code registered in the knowledge base. The type field should have the type of resource (machine or host). The server field must have the default url of the server .

2. Field that allows the monitoring agent to collect information about the physical configuration of a machine or host:

# URL to send request about machine system:
about = machine/info

or

# URL to send request about host system:
about = host/info

This option is loaded based on the content type field. The value of this field is the url for receiving this information on the server. It is concatenated to the value of the server field.

4. Setting the information about the metrics that will be collected:

# Example: metric of virtual machine
cpu.url = machine/cpu
cpu.cycle = 

# Example: metric of host
hostDomains.url = host/hostdomains
hostDomains.cycle =

# Example: metric of DBMS/database instance
size.url = storage/size
size.cycle =
size.dbms =
size.databases =

The metrics of virtual machine or server follow the same pattern. Each metric should have two fields in the file: .url and .cycle. These fields are completed with the name of the class that was created to represent the metric, as the example above. The first field contains the url provided in the server to receive the particular metric. The second field contains the value in seconds of the monitoring cycle. Represents the time window in which the agent will perform the monitoring of the metric. In the case of metrics for DBMS/database instance is possible to add two more fields: .dbms and .databases. They are also completed by the name of the metric. The value of these fields are composed by the codes of the DBMSs/databases that will be monitored. The codes are separated by a comma.