HOWTO: Save Monitoring Data on the Command Line
To extract the application performance samples, use monextract all, which will instruct CB to get all the performance data from the Metric Store and dump it on four comma-separated value files (.csv), under a newly created directory named after the experiment (i.e., ~/cbtool/data/<experiment_identifier>). These files are briefly describe as follows:
-
VM_management_<experiment_identifier>.csv: contains information regarding the provisioning time, capture time, among others, all extracted directly from the cloud management system. -
VM_runtime_app_<experiment_identifier>.csv: contains information regarding the application performance, such as latency, throughput and bandwidth, all generated directly from the VM. -
VM_runtime_os_<experiment_identifier>.csv: contains OS metrics (CPU, memory, disk I/O, network I/O) all generated by the VM. -
HOST_runtime_os_<experiment_identifier>.csv: contains OS metrics (CPU, memory, disk I/O, network I/O) all generated by the HOSTS. It can only be used on Clouds where there is direct access to the hosts (this means that this data cannot be collected on EC2, for instance), and where the Ganglia monitoring tool was manually configured on the hosts.
In addition to the four comma-separated value files (.csv) described above, CBTOOL will also create a helper script, ~/cbtool/data/<experiment_identifier>/plot.sh. This script invokes the CBTOOL - experimental GNU R code - ~/cbtool/util/plot/cbplotgen.R which will produce multiple graphs in PDF format, as exemplified
The R code ~/cbtool/util/plot/cbplotgen.R is quite experimental at this point. The dependencies for it are not part of the main CBTOOL installation at this point. They are basically:
* pdftk (in Ubuntu, ```apt-get install pdftk``` is enough)
* the following R libraries (it is recommended that you install these as root, by executing ```~/cbtool/util/plot/cbplotgen.R --help``` once).
To use the tool (again, unlike the rest of CBTOOL, this tool is not part of any regular regression testing :-), just execute it with the appropriate parameters: ~/cbtool/util/plot/cbplotgen.R -d ~/cbtool/data -e <experiment_name> -c -p -r -l. The -p tells the tool to generate plots using provisioning performance data, the -r, to generate plots using application performance data, and the -l, to overlay provisioning events on the same time axis as the application performance data (useful to show the effect of provisioning on application performance). Check the full range of options by running the tool with the --help command line option.
Here are some example graphs generated with this tool:
This first plot was generated as part of the Experiment 4, from VM_management_<experiment_identifier>.csv
This second and third plots were generated as part of the Experiment 2 from VM_runtime_app_<experiment id>.csv
These plots were generated as part of the Experiment 4
These plots were generated as part of the Experiment 3
-
VM_runtime_os_<experiment id>.csv: contains OS metrics (CPU, memory, disk I/O, network I/O) all generated by the VM.
-
HOST_runtime_os_<experiment id>.csv: contains OS metrics (CPU, memory, disk I/O, network I/O) all generated by the HOSTS. It can only be used on Clouds where there is direct access to the hosts (this means that this data cannot be collected on EC2, for instance), and where the Ganglia monitoring tool was manually configured on the hosts.
Again, the small script located in ~/cbtool/util/plot/cbplogen.R, written in the R language, can be used to quickly and automatically produce plots using the data supplied by the csv files. In order to use it, you need a relative recent version of R (2.X) with the following libraries installed: stringr, reshape2, ggplot2, data.table, optparse, hash. To see the script's help, just run it with "--help" option.
These commands (monextract) and the script (cbplotgen.R) are provided just for convenience. The same performance data can be accessed directly from the "Metric Store" (MongoDB), through a native client/driver.