Skip to content

Running Experiments

Jump to bottom
Bart Cox edited this page Sep 14, 2020 · 1 revision

Running experiments

To run an experiment with multiple variables, the following components are needed:

  • The binary (Exp_poisson_rv_nf)

  • bash script for setting the c-group (poisson_rv_nf_base.sh)

  • python script (poisson_rv_nf_multi.py): This script creates the set of CLI for the binary based on the permutations of the variables

  • yaml config file (for example: exp-python-poisson-rv-nf-multi-4w-age-gender-face.yaml): This is the configuration of the experiment. It lists the variations of the different variables.

How are these components related to each other:

Structure of a config file:

Example:

---
# Base configuration for the experiments for the poisson arrivals
network-path: networks
resources-path: ../resources
output-path: ../analysis/experiments/poisson_arrivals/rv-nf-masa-age-gender-face-4w
general-outputfile: generalOutput_poisson_arrivals.csv
mean-service-time:
  128M: 1096.65320827
  256M: 643.9415593100001
  4G: 368.53780722

  2G: 1291
  1G: 1322
  512M: 1367
rho: [1]
memory-constraints: [512M, 1G, 2G]
modes: [masa-e, bulk, deepeye]
networks: [[AgeNet, GenderNet, FaceNet]]
ait_multipliers: [0.6, 0.7, 0.8, 0.9, 0.925, 0.95, 0.975, 1]
n_workers: [4]
repetitions: 50
cmd-base: bin/Exp_poisson_arrival_rv_nf
build-folder: cmake-build-release

Components

  • network-path: relative path to the binary pointing to the network folder
  • resource-path: same as above but then for the folder containing input images
  • output-path: relative path pointing to a folder to write the measurements and all other csv files towards
  • general-outputfile: name of the general output file. This file hold the basic information over multiple execution. If the binary sees that this file already exists, it will append his ons data to the file.
  • mean-service-time: this is a map of service times by available memory. This way we can make sure the traffic is set according to the available memory. These values are found using a benchmark run and are for the composition of networks (an arrival) as a whole. This means for an arrival of {AgeNet, AgeNet} for example.
  • rho: Used to be the measure to change the traffic intensity. This is not actively used right now
  • build-folder: the folder where the executables are located
  • cmd-base: the path to the exact binary relative to the build-folder
  • repetitions: the number of arrivals in a single run.
  • modes: variable that lists the different algorithms to run
  • networks: variable that describes the different compositions of networks to run
  • ait_multipliers: variable that lists the different ait_multipliers to run. ait_multipliers stands for Inter-Arrival-Time multipliers. This is measure the change the traffic intensity. The product of the ait_multiplier and the mean-service-time is used for the actual inter-arrival-time in an experiment.
  • n_workers: variable that lists the different number of workers (threads) to run
  • memory-constraints: variable that lists the different memory constraints to run
Clone this wiki locally