Sensitivity Study

A sensitivity study answers two questions about a deck whose inputs are not known exactly: which inputs move the KPIs, and what range the KPIs should be quoted with given the inputs' ranges. It runs the deck many times with the uncertain inputs varied together, and the engine analyses the runs in one of two ways:

• Screening (Morris elementary effects) ranks the inputs by how much each one moves each KPI. It needs few runs, so use it first, to find the inputs that matter.
• Propagation (Monte Carlo with Latin hypercube sampling) gives each KPI's spread: its median and 5–95 % band (the error bar), and each input's share of that spread.

This page covers using the study in the workspace. The method itself (the Morris statistics, the sampling, the bootstrap intervals, the rank regression and the convergence test) is described in the engine's documentation: Sensitivity Analysis and Error Bars. The same study can be run from Python or the command line (Python & Command Line).

Open it from Sensitivity Study in the Navigator. You set the study up and follow its runs in the central panel. Its results appear on the right, in the Sensitivity results window.

Configure the KPIs first. The study watches KPIs of tracked components, so set up Cycle Design → KPI Configuration before you start (see Performance Metrics). To step one or two inputs over a grid and look at the curves, use a parametric study instead. To find the best design, use the optimiser.

Setting up a study

Inputs

Each row of Uncertain inputs is one parameter and its uncertainty. To add an input, pick a parameter in the Add an input… box. The list is the engine's parameter registry, the same one the parametric study uses: step times, flows, pressures, temperatures, bed and layer properties, isotherm and kinetic parameters, and your global variables. To vary several fields together, write them in terms of one global variable and make the variable the input.

A new input varies uniformly over ±10 % of its current value. A parameter whose current value is zero has no relative range, so type its bounds yourself. Choose the Distribution for each input:

Changing an input's distribution keeps its centre and spread where it can. For example, a uniform 80–120 becomes a triangular with its mode at 100, or a normal with mean 100 and SD 10.

Values are in the parameter's SI unit, shown in the Unit column. An incomplete or inconsistent row is outlined and the problem is named below it, for example lower must be below upper. Each input is varied independently of the others.

Outputs

Outputs is a grid of the tracked components against the KPIs. When you haven't changed it, the study watches the deck's default outputs: purity, recovery, mass productivity and specific energy of every tracked component. Tick or untick boxes to choose your own set. Use the defaults goes back to following the deck's tracked components. An output whose component is no longer tracked is kept and flagged, so that you can remove it.

How every run is made

Under Every run:

• Must reach cyclic steady state (on by default). A run counts only if it reaches cyclic steady state. A run that stops at the cycle limit first is a failed run. This needs a CSS check switched on in the deck. Turn it off only for a quick screening at a fixed number of cycles, where every run is equally far from steady state. Don't use that for error bars.
• Cycles replaces the deck's number of cycles for the study's runs. Leave it at 0 to keep the deck's.
• Seed fixes the random sampling, so the same set-up gives the same runs.

Under Study settings, you can give registry settings that are applied to every run before its sample values. Use them to make the CSS tolerance tighter than the deck's, so that the drift left at the end of each run doesn't read as sensitivity. For example, set css.state_tolerance to 5e-6 for the mass-balance CSS methods, or css.molar_relative_tolerance for the molar-profile method. Each setting is a registry path and a number in SI units. The cyclic steady state page explains the tolerances.

Sizes

• Screening (Morris). Trajectories (10 by default) and grid Levels (4, even). The screening runs trajectories × (inputs + 1) runs, so 10 trajectories over 3 inputs is 40 runs.
• Propagation (Monte Carlo). The First batch (100 runs). After it, Runs per added batch (50) are added until every output's percentiles have converged, or until the Run limit (400) is reached. A percentile has converged when its bootstrap interval is narrower than the Percentile tolerance (0.1) times the output's 5–95 % band. Bootstrap resamples (1000) sets how many resamples the intervals of both analyses use.

The engine checks the study as you edit it. Before any run, it confirms that every input is a tunable parameter of the deck, that every distribution is complete, that every output is a tracked component's KPI, and that a CSS check is on when runs must reach CSS. Its reason is shown under the buttons. The run buttons stay disabled until the check passes.

The set-up is saved with the deck. Inputs, outputs, run settings, seed and sizes are part of the deck file (see Saving & Loading), and every edit can be undone. The runs and results are not saved. Export them as CSV, described below, if you need to keep them.

Running

Click Run screening or Run propagation. Each button shows how many runs it will make. Each run is a copy of the deck with the inputs set to one sample's values. The runs go in parallel on a pool of background workers, as in the parametric study. Cancel stops the runs that are left; a cancelled study has no analysis. While a study runs, its log reports when each run finished, with its outputs, or failed, with the reason. When the study finishes, a status line gives the total, for example Screening finished: 40 runs, 2 failed.

The sample table below has one row per run. It shows the input values, the Monte Carlo batch, the cycle the run reached, its status and its outputs. A failed run is marked failed with the engine's reason, such as did not reach cyclic steady state in 100 cycles, and its outputs show —.

Failed runs are counted, never dropped. A failed run stays in the table. The analysis counts it as missing: the screening reports how many elementary effects are missing, and the propagation reports how many samples each output is missing. Many failures usually mean the input ranges reach where the deck doesn't solve, or that the runs need more cycles to reach CSS.

Reading the results

The Sensitivity results window shows the engine's analysis of the last study. A count of counted and failed runs is shown under every view.

Screening: the ranking

Choose an Output. The chart plots each input's μ* (the mean absolute effect, which ranks the inputs) against its σ (the spread of its effect). The whisker on each point is the bootstrap 95 % interval of μ*.

• An input far to the right moves the output most.
• An input near the origin can usually be fixed at its nominal value.
• An input with σ comparable to μ* (near the dashed σ = μ* line, or above it) has an effect that depends on the other inputs or is non-linear.

The table under the chart lists the inputs in rank order, with μ*, its interval, σ and μ. The sign of μ says whether the output rises or falls with the input, when the effect has one direction.

Use the ranking to choose which inputs to propagate. Overlapping intervals mean the screening can't tell those inputs apart. Add trajectories to separate them.

Propagation: error bars, histogram and input shares

A line at the top says whether the study converged. When it didn't, the line names the outputs that need more runs; raise the run limit and run the propagation again.

• Error bars. One row per output. The dark band is the 5–95 % band of the samples, which is the error bar to quote. The line is the median, and the light bands are the bootstrap intervals of the band's ends. Each row also gives the valid and missing sample counts, the range, and whether that output converged. Click a row to choose the output for the other two views.
• Histogram. The counted samples of one output, with lines at the 5 %, median and 95 % values. Click a run in the sample table to see where it falls; it is drawn as a solid line.
• Input shares. Each input's share of the output's spread, from rank regression. The table also gives the input's standardised rank regression coefficient (SRRC) and its rank correlation with the output, whose sign says which way the output moves. The fit's R² is shown above the table. Below 0.7 it is flagged: the inputs' ranks then explain little of the spread, because of interactions or effects that rise and fall, so don't rely on the shares.

The error bars cover input uncertainty only. They say how far the KPIs move when the inputs vary over the ranges you gave. They don't include model error: the model's own approximations, an isotherm that doesn't fit, or a mesh that is too coarse. Quote them as the spread due to the stated input uncertainty.

Exporting

Export samples (CSV) writes one row per run: its number, batch, status, input values (SI, with units in the header), outputs (empty for a failed run) and the reason a run failed. Export analysis (CSV) writes the engine's analysis as one row per output and input:

• for a screening, μ, μ*, μ*'s interval, σ, the number of effects and the output's missing count;
• for a propagation, the output's valid and missing counts, mean, SD, range, percentiles and their intervals, whether it converged and its R², and the input's rank correlation, SRRC and share.

The files are named after the stage and the deck, for example sensitivity-propagation-analysis-Config_A.csv.

A typical study

1. Track the components of interest and check that a CSS check is on.
2. Add the uncertain inputs and give them realistic ranges. The ±10 % default is only a starting point.
3. Add a study setting that tightens the CSS tolerance.
4. Run screening and look at the ranking for each output.
5. Remove the inputs whose μ* is negligible for every output.
6. Run propagation on the inputs that are left, and quote each KPI's median with its 5–95 % band.