Experiments

Introduction

Experiments are powerful tools for deriving insights from simulation models. A single experiment run typically consists of many individual simulation runs, with a large number typically occurring in parallel.

These simulation runs occur with subtle differences (including by varying initial states, agent populations, environmental conditions, and noise parameters).

Creating experiments

In HASH, experiments are defined in the experiments.json file. Currently supported are:

• linspace - vary a single parameter within a range
• arange - vary a parameter based on an increment
• values - manually enter values for a specific parameter
• monte-carlo - generate random numbers according to a distribution
• group - group together multiple experiment types into a single experiment
• multiparameter - create all possible parameter combinations from multiple experiments
• optimization - efficiently identify optimal parameter combinations that minimize or maximize desired metrics

Experiments can be created with the Experiments Wizard, or by manually defining them in the experiments.json file. You can access the wizard through the Experiments icon (beaker) in the runner controls, or from the menu bar along the top of hCore.

For example, this values experiment will run seven experiments, setting a different value of [0..6] in the radius field in each one.

{
    "Sweep values": {
        "type": "values",
        "field": "radius",
        "values": [0,1,2,3,4,5,6],
        "steps": 100
    }
}

To run an experiment, click the "Experiment Runner" button in the runner controls, denoted with a beaker icon. The option "Sweep Values" will be available in the selector.

Running this experiment will generate 7 new simulations, each with a slightly different globals.json. If we run the simulation, we can see exactly which parameters get changed in the sidebar:

To access the changed varied parameter from within the simulation, we simply access that parameter from context.globals():

const behavior = (state, context) => {
  const { radius } = context.globals();
};

Now, any behaviors that rely on the "radius" parameter from globals.json will use the corresponding value.

For more information on specific syntax, read more about Experiment Types.