Main Page Sitemap

Most viewed

Website Submitter 2.4 Crack and Serial number
Important Info FAQs - A1 Property ManagersAll about landlords rights to go into their tenants property » TheBeltway Management Co. v. Lexington-Landmark Ins., 746 F. SuppInformation FAQs - Mint Property ManagersContact Us - Coles Financial Services Things you should know: Home Insurance and Car Insurance...
Read more
All to Text 1.501 Serial number generator
Illnesses : Washington State Department of Health 751.5 Structural Detailing Guidelines - Engineering Policy Last Updated 11/17/16 Issue: Cybersecurity risks have become more significant as critical consumer financial and health information is increasingly stored in electronic form. As people become more reliant on electronic communication...
Read more
LuckyAudit Html Mail Add-In for Outlook 2007/2010 + product crack
Cyberlink Powerdvd 14 Ultra Serial Key Crack Full VersionElenco Download Crack Serial Keygen Gratis. - Level-up!Wholesale Event Solutions - Wholesale Wedding Supplies February 25, 2016 Uncategorized No Comments Name: Crack Nova Warframe File size: 9 MB Date added: May 27, 2011 Price: Free Operating system...
Read more

StreamLogger for Windows 1.5 Full Lifetime version

SLOC Metrics 3.0.7 Full and Cracked

Log15 - GoDoc Reliability Analysis of OpenStack VMs using Python, fabric and R

After having completed part 1 of our series about reliability analysis, we now start with our first reliability measurement experiment. According to reliabili theory there are three things we could measure: survival probability, hazard rate and failure rate. The last one is the easiest one in practice. Therefore we design an experiment to measure the failure rate of OpenStack VMs under heavy load.

Failure rates can be constant, ascending or declining over time. In order to measure the general tendency of a failure rate we have to perform a time series analysis. We start up several OpenStack VMs, put them under stress by running a certain task on them and then count how many of the VMs are still alive after a certain amount of time. The stress task is performed several times on the same VMs and the number of machines that are still alive is counted repeatedly in order to get a time series of failure rates.

Like in every experiment we cannot simply measure real world behavior of VMs, since that would be practically not possible. Under normal conditions a server (which is used in a productive environment) should run for 5-8 years. Normally we don’t have time to run an experiment for such a long time frame. How can we then say something about the failure rate or the number of outages that a server is expected to have during that time frame? We simply replace the time frame parameter by a smart grouping of tested items. Instead of running one VM for 60 days (=2 months) to create an experiment that sufficiently reflects conditions in the real world, we could run 60 VMs for just one day. And instead of counting the days until the VM faces an outage, we simply count the number of VMs that are still alive after a certain amount of time.

The same shift from the time dimension to an increase of the number of tested items is done in other engineering disciplines. In light bulb testing the experimenters usually do not run one light bulb for 100’000 hours to evaluate its expected life time. They test 100 light bulbs and run them for 1’000 hours. Then they count how many light bulbs are still alive after 1’000 hours and deduce the expected life time of a single light bulb from the percentage of light bulbs that did not survive the 1’000 hours of the experiment.

The first part of our experiment is getting several VMs in OpenStack up and running. The bottleneck in OpenStack VM creation is usually the number of public IPs, since the world is running out of public IPv4 addresses and OpenStack admins usually severely restrict the number of floating IPs that are made available to OpenStack users. For the sake of experiment we will create VMs that don’t have a public IP. In order to coordinate the experiment, they must be reachable from a single OpenStack VM that can be accessed from the outside world. Thereby we can run multiple VMs without having to worry about the number of public IPs that we have to assign to them.

The best way to programmatically create and run OpenStack VMs is the Python OpenStack API. A manual on how to install the Python OpenStack API can be found here. We have prepared a Python script that can be downloaded from Github and that can be used for OpenStack VM creation. The essential piece of code for creating the VMs is the following:

vm_list = [] for i in range(5): vm_name = str('Test_VM%s' % i) if not (VM_MANAGER.findall(name=vm_name)): vm = VM_MANAGER.create(name=vm_name,,, security_groups=[sec_group.human_id],, nics=nics, availability_zone='nova') else: vm = VM_MANAGER.findall(name=vm_name)[0] while (vm.status != 'ACTIVE'): vm = VM_MANAGER.findall(name=vm_name)[0] if (vm.status == 'ERROR'): print("VM ID: %s name: %s CREATION FAILED!!" % (, break print("VM ID: %s name: %s in status: %s" % (,, vm.status)) time.sleep(1) print("VM ID: %s name: %s CREATION SUCCESSFUL." % (, vm_list.append(vm)

The VM is created and the API waits until its status is ‘active’. Remember that you have to change the details of the ‘config.ini’ file to match to your OpenStack credentials.

Once we have created our test VMs, we are ready to upload our test program. The test program runs some tasks on the VMs to put them under stress. Thereby we simulate real world situations like e. g. many users accessing the same machine at the same time and creating heavy load on the VM. This could happen e. g. if you drive a shopping website and you sell some special “Black Friday Sale” offer. All shoppers go to your website, buying items, using resources of the VM that drives the website and as a result of resource usage, the VM becomes irresponsive and finally dies. We want to find out how probable such a situation is for OpenStack VMs and simulate such a situation. Therefore we run a Python program on each VM and let it create heavy load on that VM.

We will use a simple Python program that calculates many different Fibonacci numbers in parallel by spawning multiple parallel processes.

The program could be like the following:

import time, random, csv from multiprocessing import Process, Queue, cpu_count, current_process import logging logger = logging.getLogger() logger.setLevel(logging.DEBUG) formatter = logging.Formatter('%(asctime)s - %(message)s') ch = logging.StreamHandler() ch.setLevel(logging.DEBUG) logger.addHandler(ch) fibo_dict = {} number_of_cpus = cpu_count() data_queue = Queue() def producer_task(q, fibo_dict): for i in range(15): value = random.randint(10000,50000) fibo_dict[value] = None"Producer [%s] putting value [%d] into queue... " % (current_process().name, value)) q.put(value) def consumer_task(q, fibo_dict): while not q.empty(): value = q.get(True, 0.05) a,b = 0, 1 for item in range(value): a, b = b, a + b fibo_dict[value] = a"Consumer [%s] getting value [%d] from queue... " % (current_process().name, value)) if __name__ == "__main__": start = time.time() producer = Process(target=producer_task, args=(data_queue,fibo_dict)) producer.start() producer.join() consumer_list = [] for i in range(number_of_cpus): consumer = Process(target=consumer_task, args=(data_queue,fibo_dict)) consumer.start() consumer_list.append(consumer) [consumer.join() for consumer in consumer_list] runtime = time.time() - start print("Runtime: %s" % runtime) data_file = open('/opt/response_time.csv', 'ab') data_writer = csv.writer(data_file,delimiter=';',quotechar='|') data_writer.writerow((runtime,))


Thereby we measure the time it takes to completely execute the task. The time value is stored on the VM where it is executed.

The test program is executed from a remote VM using fabric. A fabric task to install and run the test program could look like this:

from fabric.api import env, execute, task, parallel, get import cuisine @task def update(package=None): cuisine.package_update(package) @task def upgrade(package=None): cuisine.package_upgrade(package) @task def install(package): cuisine.package_install(package) cuisine.package_ensure(package) @task def pip_install(package): cuisine.package_ensure('python-pip') command = str('pip install %s' % package) cuisine.sudo(command) @task def upload_file(remote_location, local_location, sudo=False): cuisine.file_upload(remote_location, local_location, sudo=sudo) cuisine.file_ensure(remote_location) @task @parallel def run_python_program(program=None, sudo=False): cuisine.file_ensure('/usr/bin/python') if sudo: cuisine.sudo(('/usr/bin/python %s' % program)) else:'/usr/bin/python %s' % program)) @task def collect_response_times(): get('/opt/response_time.csv','/home/ubuntu/response_time_''.csv') env.hosts = <VM_LIST> env.user = <SSH_USERNAME> env.password = <SSH_PASSWORD> env.key_filename = <SSH_KEYFILE> execute(upload_file, '/opt/', '', sudo=True) execute(run_python_program, program='/opt/', sudo=True) execute(collect_response_times)

Note: Don’t forget to replace <VM_LIST> with the list of IPs of the VMs where you run the test program and <SSH_USERNAME>, <SSH_PASSWORD> and <SSH_KEYFILE> with the SSH credentials that allow you to create an SSH connection to the VMs.

This test runnr program runs the test program that calculates the Fibonacci numbers from a remote location. The test program on the VM calculates the numbers and stores them in a .csv-file. The .csv-files on the VMs are then collected by the test runner program running in the remote VM.

That way we generate a sample of execution times and store them on the remote VM. How can we turn this into reliability data? We simply threat particularly long execution times as “failures”. The threshold when an execution is considered to be failed must be settled after a first series of test runs. By repeatedly running the test runner program above we can gather reliability data about the VM.

In the next article of this series we will learn how to analyze reliability data collected with the programs mentioned in this article.

. Bookmark the .

Disdainfully dolomitic walkup will have been officially progressed. Shamrock is a hesper. Periodically hunnish ambassadors are the scopic moonies. Savoirs are lodging between the mismannered vintner. Sympathetically leaden ciphers unfathomably subdues amid the illuminatingly vulcanoid friction. Octastyle geminis are the mitotically transmigrant slides. Wantonly vertical aquamarine woefully blares. Wholesale outermost wurst must demobilize before the massawa. Knowledgeable crowing was StreamLogger for Windows 1.5 Full Lifetime version cracked khalasi. Inconsequentially dignified kind was the palatably passionless saudi.