FaaS benchmarking analysis

This repository contains methods and visualization methods for analyzing benchmarking raw data generated by BeFaaS experiments.

Setup

graphviz needs to be installed separately for plotting graphs. (eg using homebrew or any other system package manager).

This project is best run inside of a conda.

Install miniconda for your system following the official instructions.

After installation you should have:

• conda as an executable in your $PATH

Now setup a new conda environment based on the environment.yml given in this project.

$ conda env create -f environment.yml

Make sure to activate the correct conda environment before running anything.

$ conda activate befaas
$ conda env list  # check that the env is properly installed and activated
> # conda environments:
> #
> base                     /Users/max/miniconda
> befaas        *  /Users/max/miniconda/envs/befaas

Now you will need to install either using a development or a normal install of the project.

Normal install

Run inside project root directory:

$ python setup.py install

Development install

Install the befaas from the current repository by executing setup.py in development mode:

$ python setup.py develop

A development version can also be uninstalled by using: python setup.py develop --uninstall.

Usage

Import log data from experiments

Experiments generate log data that is unfiltered and separate for each platform. ./scripts/dump_logs.py filters logs and outputs clean log data into a specified output directory.

$ ./scripts/dump_logs.py ../experiments/logs/webservice/2020-05-23_16-21-29 ./output/

This will output a single 2020-05-23_16-21-29.json in the output directory.

---

(Re-) Generate paper figures

1. Locate you own log dumps or see the log dumps in /dumps (and unzip them)
2. Install all packages listed in requirements.txt
3. See /paperplots:
   1. Notebooks ending with _calc analyze dump files, files ending with _draw generate the plots
   2. Adjust path, dump file list, and output file in the respective header
   3. Run the _calc notebook first, it will fill a .pkl file which is used as input for the draw notebook

License

This repository is licensed under the Apache 2.0 License.