netscenario

Code that talks over a network behaves differently on a good link and on a bad one: a video call over a mobile network that loses bursts of packets, a game on a link with 150 ms of delay, a protocol over a path that goes down for two seconds. netscenario makes this a normal, repeatable test. You describe a network (nodes, links, delay, loss, rate, changes over time), run traffic or your own program in it, measure every packet, and assert on the numbers.

The network is made of Linux network namespaces and the real kernel queues (netem, tbf, htb), so the packets of your program take a real path. Nothing outside the namespaces is touched, and on most machines nothing needs root.

Where to start

Everybody starts with Install and check the machine and Your first run. Then pick the path that is yours:

The chapters under Shapes of a network show, one by one, the networks you can describe: a single link, a chain, a star, a dumbbell, several paths, a link that moves.

What you can rely on

• A scenario is data. A short YAML file or a Python builder; the same scenario gives the same result structure everywhere.
• Nothing is approximated silently. If the machine cannot reproduce something faithfully, the run is refused with the feature, the backend and a way out (netscenario capabilities).
• A result carries its provenance. The seed, the kernel, the tool versions and the precision of the timers of the machine are in the result, and a recorded run can be replayed.

About this book

Every scenario in this book is read by the same reader as a run, and the short ones are run in the CI of every pull request, so what you see here works with the version of the program that the book was built from. The pages explain and show; the exact definition of every key, field and option lives in the reference documents.

Install and check the machine

netscenario runs on Linux. It needs the tools ip, tc, unshare, nsenter and prlimit (the packages iproute2 and util-linux) and the right to create network namespaces. Debian and Ubuntu 22.04 give an ordinary user that right; Ubuntu 24.04 needs one setting, which Running in a CI and the permissions page describe.

Install

For Python, which also gives you the command line:

pip install netscenario pytest-netscenario

The wheel carries the two programs it needs, so the Python package needs nothing else. For the command line without Python:

cargo install netscenario-cli netscenario-agent

Both programs have to be installed side by side, which is why both crates are named.

Ask the machine

doctor looks at the machine and says what it can do and, for everything that is missing, how to fix it:

netscenario doctor

The exit code is 0 when scenarios can run here and 3 when something has to be fixed first, so a script or a CI job can use it as the first step. The command makes its own namespaces for the probe and touches nothing of the host.

A second command says which features of a scenario this machine reproduces exactly:

netscenario capabilities

A feature that the machine cannot reproduce is not approximated: a scenario that uses it is refused before anything is built, and the message names the feature and what to change. This is the rule that makes a result worth reading.

From Python the same report is netscenario.doctor().

Where the files go

A run writes its result into a directory: --out, or netscenario-results/<name>-<time>/ in the current directory. Give it a place you can write to, because the programs in the namespaces write the raw records there too.

Your first run

The smallest scenario has two nodes, a link between them and one stream of packets from one to the other. Save it as minimal.yaml:

# The smallest scenario: two nodes, one link without any impairment, one stream.
version: 0
name: minimal
duration: 5s
topology:
  nodes: [client, server]
  links:
    - id: access
      between: [client, server]
workloads:
  - id: media
    kind: udp-cbr
    from: client
    to: server
    rate: 1mbit
    packet-size: 1200b

validate reads the file and checks it without building anything:

netscenario validate minimal.yaml

ok: minimal (minimal.yaml)
  2 nodes, 1 link, 1 workload, duration 5s
  needs: point-to-point, ipv4, udp-generators

run builds the two nodes in their own namespaces, joins them with the link, sends the stream for five seconds, and measures every packet:

netscenario run minimal.yaml --seed 1

running minimal for 5s ...
netscenario run: minimal
  backend kernel (user namespace, iproute2 6.15.0, reno cubic)
  seed 1

workload media  client -> server  1 mbit/s, 1200 byte packets
  sent 521  received 521  lost 0 (0.00 %)
  delay  min 0.0 ms  median 0.0 ms  p95 0.0 ms  p99 0.0 ms  max 0.1 ms
  jitter  mean 0.004 ms  last 0.003 ms  max 0.009 ms  (RFC 3550)
  throughput per second  1.01 mbit/s  998.4 kbit/s  998.4 kbit/s  998.4 kbit/s  998.4 kbit/s

result: netscenario-results/minimal-20261110T205757Z/result.json

The link has no impairment, so nothing is lost and the delay is the machine’s own. Make the link worse and run it again. Replace the link with one that has 40 ms of delay, a jitter of 5 ms and 1 % loss in the direction from the client to the server:

# file: worse.yaml
version: 0
name: worse
seed: 5
duration: 5s
topology:
  nodes: [client, server]
  links:
    - id: access
      between: [client, server]
      forward:
        delay: 40ms
        jitter: { amount: 5ms, distribution: normal }
        loss: { model: random, p: 1% }
workloads:
  - { id: media, kind: udp-cbr, from: client, to: server, rate: 1mbit, packet-size: 1200b }

The delay is now near 40 ms, a few packets are lost, and the same seed gives the same lost packets on the same machine. A run that has no seed draws one and prints it, so a surprising run can be repeated with --seed.

From Python

The same run from Python gives a result with typed fields, in SI units (seconds, bits per second, ratios from 0 to 1):

import netscenario as ns

result = ns.run(ns.Scenario.load("worse.yaml"))
media = result["media"]
print(media.sent, media.received, media.lost)
print(f"median delay {media.delay.p50 * 1000:.1f} ms")
assert 0.035 < media.delay.p50 < 0.060

What is in the result directory

result.json is the result: the provenance, the seed and, for each workload, the counts, the delay percentiles, the series over time and everything the other chapters mention. Next to it are the raw records of the run and scenario.json, the scenario as it ran, with which a run can be replayed. Results, reports and replay goes through them.

Your first test

pytest-netscenario makes a network scenario a normal test. Install it next to the package (pip install netscenario pytest-netscenario); pytest finds it by itself.

The scenario next to the test

A scenario is a file. Put it next to the test, here video-call.yaml: a video call over a Wi-Fi link that delays by 30 ms, with a jitter of 5 ms and 1 % loss from the caller to the callee.

# file: video-call.yaml
version: 0
name: video-call
seed: 11
duration: 5s
topology:
  nodes: [caller, callee]
  links:
    - id: wifi
      between: [caller, callee]
      forward:                     # caller -> callee
        delay: 30ms
        jitter: { amount: 5ms, distribution: normal }
        loss: { model: random, p: 1% }
        rate: 20mbit
      reverse:                     # callee -> caller
        delay: 30ms
        rate: 20mbit
workloads:
  - id: video
    kind: udp-cbr
    from: caller
    to: callee
    rate: 1500kbit
    packet-size: 1200b

The test

The fixture netscenario runs a scenario and gives you the result. Assertions are plain assert on numbers in SI units: seconds, bits per second, ratios from 0 to 1.

# test_video_call.py
from pytest_netscenario import Runner


def test_the_video_arrives_in_time(netscenario: Runner) -> None:
    result = netscenario.run("video-call.yaml")   # looked for next to this file
    video = result["video"]

    assert video.loss_ratio < 0.05                # 1 % is configured: allow noise, catch a broken link
    assert video.delay is not None
    assert video.delay.p99 < 0.100                # seconds

Run it with pytest. When a number is missed the test fails and the report shows the numbers of the run. The scenario file is looked for next to the test file, then in the directories of the ini option netscenario_scenarios, then in the root of the project.

If you do not need the runner, a marker runs the scenario for the test and gives you the result:

import pytest

import netscenario as ns


@pytest.mark.netscenario("video-call.yaml", seed=1)
def test_another_seed_with_the_marker(netscenario_result: ns.Result) -> None:
    assert netscenario_result["video"].sent > 0

Machines that cannot run it

A test that needs namespaces is skipped with the reason on a laptop that cannot make them, and fails when the variable CI is set (GitHub Actions sets it). A missing permission on the CI can never hide as a green run.

Building the scenario in code

A scenario does not have to be a file. The builder writes the same thing in Python, and everything it builds is checked by the same reader:

import netscenario as ns

scenario = (
    ns.Scenario.builder("video-call", duration="5s", seed=11)
    .nodes("caller", "callee")
    .link(
        "wifi",
        "caller",
        "callee",
        forward=ns.Impairments(delay="30ms", loss="1%"),
        reverse=ns.Impairments(delay="30ms"),
    )
    .udp_cbr("video", "caller", "callee", rate="1500kbit", packet_size="1200b")
    .build()
)
result = ns.run(scenario)
assert result["video"].sent > 0
print(result["video"].loss_ratio)

Parameters of a scenario file (params:) can be set from a test with params= on the marker, the fixture or ns.run; the next chapters show them at work.

Statistics and checks

One run is one sample. A claim such as “the loss stays below 5 %” is stable only if the end of the interval that matters is far from the threshold, so netscenario makes the set of runs a first-class thing.

Repetitions

repetitions=n runs the scenario n times (5 to 1000) with seeds derived from one seed, and gives a set with an interval for each metric:

from pytest_netscenario import Runner


def test_the_loss_is_low_with_a_confidence_interval(netscenario: Runner) -> None:
    runs = netscenario.run("video-call.yaml", repetitions=5)
    loss = runs.metric("video.loss_ratio")

    assert loss.ci95_high < 0.05                  # 1 % is configured: the claim is "below 5 %"
    assert runs.metric("video.delay.p99").ci95_high < 0.100

ci95_low and ci95_high are the ends of the 95 % interval around the mean (or the median, with statistic="median"); confidence= changes the level. A bound is not picked by eye: for a count, take four standard deviations of its model each side; for a delay, take its exact physical lower bound and leave room only upward.

Checks in the file

Instead of asserting in Python you can write what the scenario must show in the file itself, in the section expect. The core judges it, prints each check with its value and how far it is from the bound, and the same verdict is in the result and in every report:

# file: expect.yaml
version: 0
name: expect
seed: 21
duration: 4s
params:
  rtt: 40ms
topology:
  nodes: [client, server]
  links:
    - id: access
      between: [client, server]
      forward: { delay: "{rtt / 2}", rate: 10mbit, loss: { model: random, p: 0.5% } }
      reverse: { delay: "{rtt / 2}", rate: 10mbit }
workloads:
  - { id: media, kind: udp-cbr, from: client, to: server, rate: 2mbit, packet-size: 1200b }
expect:
  checks:
    - name: the stream arrives at its rate
      metric: media.throughput.mean
      value: ">= 1.9mbit"
    - name: the delay is that of the link
      metric: media.delay.p99
      value: "< {rtt / 2 + 10ms}"
    - name: the link does not lose more than it was told to
      metric: media.loss_ratio
      value: "< 2%"

netscenario run expect.yaml exits with 1 when a check does not hold, and the pytest plugin fails a test that ran such a file, with the checks that did not hold, also when its body asserts nothing. The parameter rtt has a default and can be set per run, which lets one file serve several tests:

def test_a_longer_path(netscenario):
    result = netscenario.run("expect.yaml", params={"rtt": "100ms"})
    assert result.expect is not None
    print(result.expect.explain())                # which checks hold

For a set of runs the bounds are on the statistics of the set (mean, median, min, max, ci95-low, ci95-high); the file scenarios/expect-set.yaml shows a claim on the worst of ten runs.

Judge only what the link guarantees

The delay of a stream over a link of 20 ms is never below 20 ms, and a test may say so exactly. The rest is noise and wants room: the machine wakes late, the CI is loaded, a burst of loss is not a mean. Noisy machines and timers says how to see how much room a machine needs.

Change the link while the test runs

Sometimes the moment of a change is the test’s: “when the app has connected, make the link worse”. A scenario file fixes its changes in advance (see Changes over time); a test can also change a link while the run goes.

The link has to be declared live when the run starts. The queues of a direction are built from every state it goes through and are not rebuilt later, so the run needs to know which links the code may change and builds those so that any state fits. A link that is not declared live is refused.

# file: stream.yaml
version: 0
name: live-stream
duration: 4s
seed: 7
topology:
  nodes: [client, server]
  links:
    - id: access
      between: [client, server]
      forward: { delay: 30ms, rate: 4mbit }
workloads:
  - { id: media, kind: udp-cbr, from: client, to: server, rate: 1mbit, packet-size: 1000b }

import netscenario as ns

with ns.start("stream.yaml", live=["access"], packets=True) as run:
    run.wait_until(1.5)                      # seconds into the measured phase
    steps = run.link("access").set(
        forward={"delay": "120ms", "loss": {"model": "random", "p": "20%"}}
    )
result = run.result()

(step,) = steps
print("the packet that met the old state:", step.packet)
assert step.live and step.packet is not None
assert result["media"].delay.max > 0.120

start returns at once with a Run. wait_until waits for a moment of the measured phase, run.link(...).set(...) changes a link (the keys are those of a set in the file), and down() and up() take it away and give it back. Leaving the with waits for the end of the run, and run.result() gives the result.

What the result records

A change from code is a step of the timeline, made between two packets: the sender is held before its next packet, the step is done, and the sender goes on. The step has live=True and packet, the number of the first packet that met the new state, so the result says exactly where the change landed.

The moment is the test’s, so two live runs differ, and the result says so (seed.not_reproducible_because). What it records is the scenario with the change as an event at the slot of that packet, and a replay of that directory does the change at the same packet and loses the same packets:

replay = ns.replay(result.out_dir)
assert replay.reproduced

With asyncio

The same run handle works from async code and never blocks the loop. Cancelling the task cancels the run and waits until the network is torn down, so no thread is left behind:

import asyncio

import netscenario as ns


async def main() -> None:
    async with await ns.start_async("stream.yaml", live=["access"]) as run:
        await run.wait_until(1.0)
        await run.link("access").down()
        await asyncio.sleep(0.5)
        await run.link("access").up()
    result = await run.result()
    print("lost:", result["media"].lost)


asyncio.run(main())

run.events() is an async iterator over the events of the run (when the network is built, when the workloads start, …). run_async(...) is the one-call form for a scenario that is only to be awaited. A change that the backend cannot do exactly (a state its queues do not fit) is refused to the caller as an error, before anything is touched, and the run goes on.

A program of your own in the network

A workload can also be your own program, started in a node: a server, a client, the service you are testing. It then sends real packets through the network, and you test what it does over a bad link instead of only what the generators measure.

command runs code, so it is written in the scenario file where everybody can see it. The example is a UDP echo server on one side and a client on the other, over a link with 30 ms of delay each way and 5 % loss each way:

# Your own program in a node: a UDP echo server on one side, a client on the other, over a link with 30 ms of delay each
# way and 5 % loss each way. `command` runs code, so it is written in the file where everybody can see it (SECURITY.md).
# The server is started first and the client when the server has a UDP socket open on port 4433; each program reports
# what it measured as lines of JSON on its standard output (`{"metric": ..., "value": ...}`), and the result has them.
version: 0
name: echo-over-a-bad-link
seed: 21
duration: 6s
topology:
  nodes: [client, server]
  links:
    - id: wan
      between: [client, server]
      both:
        delay: 30ms
        loss: { model: random, p: 5% }
workloads:
  - id: server
    kind: command
    node: server
    run: ["python3", "echo_server.py", "{server.addr}", "4433"]
    ready: { udp-port: 4433, within: 5s }
    shutdown: { signal: TERM, within: 2s }
  - id: client
    kind: command
    node: client
    run: ["python3", "echo_client.py", "{server.addr}", "4433", "200"]

Three things in it are worth knowing:

• {server.addr} is the address of a node, filled in when the network is built.
• ready says when the program is ready: the client is started when the server has a UDP socket open on port 4433, not after a guessed sleep.
• shutdown says how a program that does not end by itself is stopped (here SIGTERM, then the run waits up to two seconds).

A program reports what it measured as lines of JSON on its standard output, {"metric": "rtt", "value": 61.2}, and the result has them. The test reads the exit of each program and its metrics:

"""Your own program in a node, tested over a bad link (needs `pytest-netscenario`)."""

from pytest_netscenario import Runner


def test_the_echo_server_answers_over_a_lossy_link(netscenario: Runner) -> None:
    result = netscenario.run("echo.yaml")

    client = result.programs["client"]
    # The client ended by itself with 0; the server was stopped by the run (signal of `shutdown`)
    assert client.succeeded
    assert result.programs["server"].how_it_ended == "stopped"

    sent = client.metrics["sent"].last
    answered = client.metrics["answered"].last
    assert sent == 200
    # A packet is answered if neither it nor its answer is lost: 0.95 * 0.95 = 90.25 % of 200 is
    # 180, with a standard deviation of 4.2; four of them each way round are 17 packets
    assert 163 <= answered <= 198

    rtt = sorted(value for _, name, value in client.readings() if name == "rtt")
    assert len(rtt) == answered
    # The link takes 30 ms each way: no answer comes faster than 60 ms, and the median is near
    assert rtt[0] >= 60
    assert rtt[len(rtt) // 2] < 90

The bounds in that test are derived and written next to it: a packet is answered if neither it nor its answer is lost, so 90.25 % of 200 packets is about 180 with a standard deviation of 4.2, and no answer comes faster than the 60 ms the link takes both ways.

The programs run in the directory of the scenario file (change it with working_dir=), without a shell, with the environment you give them. Read Programs of your own for the limits a run sets on them, and the security notes before you run a scenario file you did not write: a command is code.

The scenario file

A scenario is a short YAML file. It says what the network is, what traffic goes through it and, if you want, what must come out. The same file runs from the command line, from Python and from the pytest plugin.

# file: annotated.yaml
version: 0                      # the format of the file
name: annotated                 # shown in results and reports
seed: 3                         # one seed for every random part; leave it out and a run draws one and prints it
duration: 10s                   # how long the workloads run
topology:
  nodes: [client, server]       # a node is a host (or, with `role: router`, a router)
  links:
    - id: access
      between: [client, server] # the first node is where "forward" starts
      both: { delay: 20ms, rate: 10mbit }
workloads:
  - { id: media, kind: udp-cbr, from: client, to: server, rate: 2mbit, packet-size: 1200b }

The parts

Values have units

A number always has its unit, so a wrong unit is an error and not a silent factor of a thousand: 20ms, 2s, 10mbit, 1500kbit, 1200b, 1%. An unknown key is refused with the line, the column and a hint, never ignored:

error: `both` cannot be combined with `forward` or `reverse`
 --> scenarios/invalid/both-and-forward.yaml:9:13
  |
8 |       between: [client, server]
9 |       both: { delay: 10ms }
  |             ^
  = path: topology.links[0].both
  = help: use `both` for the same values in both directions, or `forward` and `reverse` for different ones

Look before you run

netscenario validate annotated.yaml     # reads and checks, builds nothing
netscenario show annotated.yaml         # the nodes, the links, the routes and the path of each workload
netscenario expand annotated.yaml       # the long form, where a shortcut has become nodes and links

annotated: 2 nodes, 1 links, routing auto

nodes
  client host
  server host

links (forward is from the first node to the second)
  access client - server
      both     delay 20ms, rate 10mbit

routes (the smaller metric is used first; * is listed in the file)
  client to server via access next server metric 100
  server to client via access next client metric 100

paths of the workloads
  media: client -[access]-> server

show is how you see that the network is the one you meant before a run spends time on it.

An editor that knows the format

netscenario schema prints the JSON Schema of the format. Put this as the first line of a file and an editor with the YAML language server completes keys and checks values while you write:

# yaml-language-server: $schema=https://raw.githubusercontent.com/leonkrix/netscenario/main/docs/schema/scenario-v0.json

Parameters, templates and includes

A value can be an expression over a parameter ("{rtt / 2}"), a link that is needed twice can be a template, and a part of a file can be included from another file. scenarios/parameters.yaml, scenarios/templates.yaml and scenarios/include.yaml show each; the scenario format defines them. A file a scenario refers to (a trace, a table of delays) is read once when the scenario is loaded and lives in the recorded scenario, so a replay needs nothing from the original directory.

What a link can do

A link has two directions, forward (from the first node of between) and reverse, and each direction has its own impairments. both sets the same for both. Everything a direction does is a real kernel queue (netem, with tbf or htb for a rate), which is why a result can be trusted to be the network and not a model of it.

# file: impaired.yaml
version: 0
name: impaired
seed: 8
duration: 4s
topology:
  nodes: [client, server]
  links:
    - id: access
      between: [client, server]
      forward:
        delay: 40ms
        jitter: { amount: 5ms, distribution: normal }
        loss: { model: random, p: 1% }
        reorder: { share: 2%, gap: 5 }
        duplicate: 0.5%
        rate: 10mbit
      reverse:
        delay: 40ms
        rate: 20mbit
workloads:
  - { id: media, kind: udp-cbr, from: client, to: server, rate: 2mbit, packet-size: 1200b }

The exact keys, their forms and the models of loss are defined in the scenario format.

A queue makes a bottleneck

A rate limit alone drops what it cannot send. A queue in front of it holds packets, and its discipline decides what is dropped when it is full:

# file: bottleneck.yaml
version: 0
name: bottleneck
seed: 4
duration: 5s
topology:
  nodes: [client, server]
  links:
    - id: access
      between: [client, server]
      forward:
        delay: 10ms
        rate: 5mbit
        queue: { discipline: codel, limit: 100p }
workloads:
  - { id: media, kind: udp-cbr, from: client, to: server, rate: 8mbit, packet-size: 1200b }

The stream offers 8 mbit into 5 mbit, so more than a third is dropped; CoDel keeps the delay it adds near its target instead of letting the buffer fill. Change discipline to fifo and compare the delay percentiles.

A profile instead of numbers

A link can start from a profile: a named set of values from a source, such as lte-good. What you write next to it wins.

# file: profile.yaml
version: 0
name: profile
seed: 7
duration: 5s
topology:
  nodes: [phone, server]          # the first node of `between` is the terminal
  links:
    - id: access
      between: [phone, server]
      profile: lte-good
      forward: { loss: { model: random, p: 1% } }
workloads:
  - { id: media, kind: udp-cbr, from: phone, to: server, rate: 2mbit, packet-size: 1200b }

netscenario profiles lists them and netscenario profiles lte-good shows one with its values and the sources they come from. A profile sets only what a source states; the profile library lists what is there and what waits for a source.

What the machine cannot do

netscenario capabilities lists the features this machine reproduces exactly. A link that asks for one it cannot (for example a seed on a tc older than 6.6.0) is refused before anything is built, with the feature, the backend and the way out. That refusal is the design: a run that silently did something else would be worth nothing.

Changes over time

A link does not have to stay the same. The timeline lists events that happen at a moment of the measured phase: a rate that drops, an outage, a link that goes down and comes back. Each event is applied to one link, before the first packet its stream sends at or after the moment.

# file: drop-and-outage.yaml
version: 0
name: drop-and-outage
seed: 3
duration: 9s
topology:
  nodes: [client, server]
  links:
    - id: access
      between: [client, server]
      both: { delay: 20ms }
workloads:
  - { id: media, kind: udp-cbr, from: client, to: server, rate: 4mbit, packet-size: 1200b }
timeline:
  - { at: 2s, link: access, set: { both: { rate: 2mbit } } }      # a rate limit from the 2nd second
  - { at: 5s, link: access, outage: 1s }                           # nothing is delivered for a second
  - { at: 7s, link: access, set: { both: { rate: null } } }       # `null` takes the limit away again
measure:
  series: { bin: 500ms }

An event does exactly one of four things:

An event can repeat with every, until until or count times. A flapping link is two lines:

timeline:
  - { at: 5s, link: access, outage: 400ms, every: 10s, until: 55s }

How long did it take to recover?

Because the run records a series over time, it can answer the question that matters after an event: how long until the stream is back at its level. Each event that disturbs a stream has a recovery:

import netscenario as ns

result = ns.run(ns.Scenario.load("drop-and-outage.yaml"))
for recovery in result["media"].recoveries:
    print(recovery.kind, recovery.link, recovery.recovered, recovery.time)

The level of a bin is the share of the packets sent in it that arrived (the goodput for TCP). The resolution of the time to recover is one bin, set with measure.series.bin or --bin. The baseline, the window and the share that counts as recovered are options of the same section; the results reference defines them.

When the test decides

The moment of an event in the file is fixed in advance. If a test needs to make the change when something happens (when the app has connected), it changes the link while the run goes, and the result records that change as a step too: Change the link while the test runs.

What a result tells about the steps

Each step is in events of the result with the moment the file asked for (scheduled_ns), the moment the backend had done it (applied_ns), and for a step made at a packet, the number of that packet. applied_ns - scheduled_ns is how late the step was, which on a quiet machine is a fraction of a millisecond (Noisy machines and timers).

Traffic

A workload is something that sends packets through the network. All of them run at the same time, each from a node to a node, and each has its own numbers in the result.

UDP streams

The constant bit rate stream is the simplest: a packet of a fixed size at a fixed rate. Three more patterns are drawn from the seed, so they repeat:

# file: patterns.yaml
version: 0
name: patterns
seed: 7
duration: 6s
topology:
  nodes: [client, server]
  links:
    - id: access
      between: [client, server]
      both: { delay: 20ms }
workloads:
  - { id: steady, kind: udp-cbr, from: client, to: server, rate: 1mbit, packet-size: 1200b }
  - { id: requests, kind: udp-poisson, from: client, to: server, rate: 800kbit, packet-size: 400b }
  - id: sensor
    kind: udp-onoff
    from: client
    to: server
    rate: 400kbit
    packet-size: 200b
    on: 500ms
    off: 1500ms
    distribution: exponential
  - id: video
    kind: udp-bursty
    from: server
    to: client
    packet-size: 1200b
    burst-packets: 10
    gap: 200us
    pause: 100ms
    distribution: exponential

udp-cbr is a constant rate, udp-poisson has exponential gaps, udp-onoff sends in phases and pauses, and udp-bursty sends bursts of packets that follow each other closely. Every packet carries a sequence number and a send time, so the receiver can tell lost, reordered, duplicated and damaged apart, and measure the one-way delay exactly (both ends read the same clock).

For each stream the result has the counts, the delay percentiles, the jitter (RFC 3550), the loss runs (how the lost packets cluster), the reordering and the throughput over time. A recorded stream can be replayed from a file with kind: trace-replay (the moments and sizes of a real call, from a CSV file or a capture), and echo: true sends every packet back so you get the round trip.

TCP

TCP workloads use the kernel TCP of the machine. The congestion control is a key, and the result has the goodput, the retransmissions and the round trip of each connection:

# file: download.yaml
version: 0
name: download
seed: 5
duration: 8s
topology:
  nodes: [client, server]
  links:
    - id: access
      between: [client, server]
      both: { delay: 20ms, rate: 20mbit }
workloads:
  - { id: download, kind: tcp-bulk, from: server, to: client, bytes: 3mb, connections: 2, cc: reno }

tcp-bulk moves a number of bytes (or sends until stop). request-response is an API: a request of a size every so often over a number of connections, and the result has the time of each answer:

# file: api.yaml
version: 0
name: api
seed: 9
duration: 5s
topology:
  nodes: [client, server]
  links:
    - id: access
      between: [client, server]
      forward: { delay: 30ms, loss: { model: random, p: 0.5% } }
      reverse: { delay: 30ms }
workloads:
  - id: api
    kind: request-response
    from: client
    to: server
    request: 200b
    response: 4kb
    connections: 4
    interval: 20ms
    timeout: 5s

TCP is statistical: a connection’s goodput depends on the start-up, so judge a window after it and repeat the run. A dumbbell shows the classic fairness test.

Your own program

kind: command starts a program in a node (A program of your own in the network). It can run next to the generators, so the generators can load the link while your program is tested.

What a run measures

The section measure says what is recorded beyond the numbers: the width of the bins of the series, a pcap capture of the interfaces of a link at both ends, and a record of every packet (packets: true), which the tables of the Python layer read. The results reference lists the files.

Results, reports and replay

A run writes a directory. Its heart is result.json: everything the run measured, with the provenance that lets a reader judge it.

What is in a result

The numbers are in SI units in the Python layer (seconds, bits per second, ratios from 0 to 1). In result.json times are nanoseconds. Results reference defines every field.

In Python

The scenario of the first run:

# The smallest scenario: two nodes, one link without any impairment, one stream.
version: 0
name: minimal
duration: 5s
topology:
  nodes: [client, server]
  links:
    - id: access
      between: [client, server]
workloads:
  - id: media
    kind: udp-cbr
    from: client
    to: server
    rate: 1mbit
    packet-size: 1200b

Then:

import netscenario as ns

result = ns.run(ns.Scenario.load("minimal.yaml"), packets=True)
media = result["media"]

print(f"lost {media.lost} of {media.sent}")
print(f"p99 delay {media.delay.p99 * 1000:.2f} ms")
print("timers:", result.provenance.timers.sleeps[0].p99)

# One row for each packet, as a table that becomes a pandas or polars frame
packets = media.packets_table()
print(len(packets), packets.names)

The per-packet records, the series over time and the steps of the timeline are tables of plain Python values. to_pandas() and to_polars() turn them into frames; install netscenario[pandas] or netscenario[polars] for that. Nothing is imported unless you call it.

Reports

netscenario report <dir> turns a result directory into a report. It reads the files of the directory and nothing else, so a report can be made on another machine, later.

netscenario report out --format md       # Markdown, for the summary of a CI job
netscenario report out --format junit    # JUnit XML, for the test report of a CI
netscenario report out --format html     # one HTML file with the plots

The Markdown report of a run begins with where and with what it was made, then the checks of the file, then a table of the workloads. The HTML is one file, readable offline.

Replay

scenario.json in the directory is the scenario as it ran, with the files it read and the changes made from code. replay runs a recorded run again and compares the lost packets:

netscenario replay out

If the loss repeats from the seed, a replay loses exactly the same packets and the exit code is 0; if the packets differ, the exit code is 1 and the comparison says which. A run whose loss was not promised to repeat (a machine whose tc cannot apply a seed, an outage, a change made by a test) says so in seed.not_reproducible_because, and the replay compares what it can.

Sets of runs

--repetitions n runs a scenario n times with derived seeds and writes runset.json with an interval for each metric. A claim about a set (expect with repetitions) is judged on that interval: Statistics and checks.

Running in a CI

A scenario is a test like any other, so it belongs in the CI of the project it tests. This page is what the maintainer of that CI needs: the permission a run needs, the first step that tells you at once when a machine is wrong, the reports, and how to keep a flaky result out.

1. Allow the namespaces

A run needs the right to create network namespaces and the tools ip, tc, unshare, nsenter and prlimit. On GitHub Actions the hosted runners give it, with one setting on ubuntu-24.04:

- name: Allow unprivileged user namespaces
  run: |
    if sysctl -n kernel.apparmor_restrict_unprivileged_userns >/dev/null 2>&1; then
      sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
    fi

The step does nothing on images that do not have the setting. Permissions lists the other ways (root, two capabilities, a container) and what each costs.

2. Make doctor the first step

- name: Can this machine run scenarios?
  run: netscenario doctor

The exit code is 0 when scenarios can run and 3 when not, and the output says what to change. A failing doctor on the first step of a job saves a long log in which every test says “no namespace”.

The pytest plugin does the same by itself: on a machine that cannot run scenarios it fails the test when CI is set and skips it on a laptop, so a missing permission never turns into a green run.

3. Mind the tc version

The runner images ship an iproute2 older than 6.6.0, whose tc does not know the seed of netem. A scenario with a seed and loss is then refused, with a message that says so. Your choices are to install a newer iproute2 (the CI of this project builds 6.15.0 from the source and checks its checksum) or to leave out seed, in which case the run draws one, records it, and says that its loss is not reproducible.

4. Keep the reports

- name: Run the scenario
  run: netscenario run scenarios/lte-like.yaml --repetitions 10 --out results/lte
- name: Reports
  if: always()
  run: |
    netscenario report results/lte --format md >> "$GITHUB_STEP_SUMMARY"
    netscenario report results/lte --format junit --out results/junit.xml
    netscenario report results/lte/run-001 --format html --out results/report.html
- uses: actions/upload-artifact@v4
  if: always()
  with:
    name: netscenario-reports
    path: results/

The Markdown goes to the summary of the job, the HTML is kept as an artifact, and the JUnit file feeds the step that shows test reports. (Pin the action to a commit, as in the workflows of this repository.) The uploaded directory also holds the raw result, so a failing run can be replayed on a developer’s machine: download the artifact and run netscenario replay results/lte/run-001.

5. Exit codes

They are part of the interface:

6. Keep a flaky claim out

A claim about a measurement is stable only if its bound sits far from the value on a noisy machine. Three habits keep a CI green for the right reasons:

• Judge the mean of several runs, not one run: --repetitions 10 and a bound on the 95 % interval (Statistics and checks).
• Derive the bound from the model of the thing you test (a count: four standard deviations; a delay: its exact physical lower bound and room only upward) and write it next to the assertion.
• Know the machine. The result records the precision of the timers when the run began, and doctor --timers measures it on its own: Noisy machines and timers.

A run is a few seconds plus the setup of the network. Scenarios that only parse (netscenario validate) take milliseconds and make a good first job on every pull request.

Noisy machines and timers

A packet leaves at the slot the plan gives it only as exactly as the machine wakes a sleeping core, and a step of a timeline is done only as fast as tc answers. How late that is depends on the processor, its idle states, the timer slack of the process and the load. No figure can be promised, so netscenario measures the machine and reports it.

The report

netscenario doctor --timers

Timers
  ok    clock                  CLOCK_MONOTONIC from tsc: a reading costs 30 ns, the smallest step seen is 20 ns
  ok    sleep of 200 us        late by 54.0 us (median), 59.0 us (99th percentile), 64.7 us (largest) in 1000 sleeps
  ok    sleep of 1 ms          late by 54.5 us (median), 60.6 us (99th percentile), 84.6 us (largest) in 300 sleeps
  info  timer slack            50.0 us (the kernel may delay a timer this much to batch it with others); governor: performance
  info  load                   load average 10.30 of the last minute on 16 processors, 1 tasks runnable (this one included)
  ok    change of a qdisc      31.1 us median, 58.0 us (99th percentile), 70.4 us (largest) in 500 changes of a netem through the `tc` that lives for a run

It takes about two seconds and makes its own namespace for the last line. A late timer is a warning, never a failure: the exit code is the one of the rest of doctor, and a run is not refused because of a timer.

What a run records

Every run measures the same for about 40 ms just before it builds the network and records it in provenance.timers of the result. A result you get from somebody else’s CI says on what kind of machine it was made, and the Markdown and HTML reports have a line for it:

import netscenario as ns

report = ns.doctor(timers=True)
for sleep in report.timers.sleeps:
    print(f"sleep of {sleep.asked * 1e6:.0f} us: p99 {sleep.p99 * 1e6:.0f} us late")
assert report.ok

A run measures few sleeps, so its 99th percentile is close to the largest and each entry says how many there were.

The precision of an idle machine is one thing; what the run actually met is another, and the result has both. Each sender reports max_lateness, the most it was late for a packet, and each step in events has the moment it was due and the moment it was done (applied_ns - scheduled_ns).

Designing bounds for a noisy machine

• A lower bound is physical and exact: no packet arrives before the delay of the link. Keep it tight.
• An upper bound needs room: add the lateness the machine showed (max_lateness, or the 99th percentile above) and then some.
• Judge sets of runs with an interval rather than one run.
• On a loaded CI machine, a timer that is far later than on your laptop is a reason to widen the room, not to doubt the tool.

One link

The simplest network is two nodes and one link. Most real problems, a bad mobile connection or a long satellite hop, are one link, and most tests need no more.

A link has two directions that need not be alike. This one is a mobile access link: more delay and a lower rate going up, a higher rate going down. A stream goes each way, and the up stream is echoed so the round trip is measured:

# file: mobile.yaml
version: 0
name: mobile
seed: 42
duration: 5s
topology:
  nodes: [phone, server]
  links:
    - id: radio
      between: [phone, server]
      forward:                       # phone -> server, up
        delay: 40ms
        jitter: { amount: 5ms, distribution: normal }
        loss: { model: random, p: 1% }
        rate: 10mbit
      reverse:                       # server -> phone, down
        delay: 25ms
        rate: 20mbit
workloads:
  - { id: up, kind: udp-cbr, from: phone, to: server, rate: 1mbit, packet-size: 800b, echo: true }
  - { id: down, kind: udp-cbr, from: server, to: phone, rate: 4mbit, packet-size: 1200b }

import netscenario as ns

result = ns.run(ns.Scenario.load("mobile.yaml"))
up, down = result["up"], result["down"]

# The one-way delay of a stream is what the link says (both ends read the same clock).
# The down stream has no jitter, so no packet is faster than the delay of the link;
# the up stream has a jitter of 5 ms, so its delays are spread around 40 ms
assert down.delay.min >= 0.025
assert 0.030 < up.delay.p50 < 0.050

# The echo is the round trip: up and down together, and lost if either way lost it
print(f"round trip median {up.echo.round_trip.p50 * 1000:.0f} ms")

Three things this shape teaches:

• A bound on a delay has an exact lower end, where the link has no jitter. No packet arrives before the delay of the link (the down stream above), and a test may say so without any room; only the upper end needs room for the machine. With a jitter the delays are spread around the value (the up stream), and the bound is on the percentiles.
• Loss is per direction. The up stream loses 1 %; the down stream loses nothing. A stream does not see the loss of the other way unless it echoes.
• forward is from the first node of between. Put the terminal first and forward is up, the convention of the profile library.

When you do not know the numbers of your link, start from a profile (profile: lte-good, What a link can do) and override what you know.

A chain and a star

More than two nodes means routers. A node with role: router forwards packets; a host does not. The routes are computed by the shortest path, and you can list your own.

A chain

A client and a server with two routers between them, each hop with its own values. The chain shortcut writes the nodes and the links for you:

# file: chain.yaml
version: 0
name: chain
seed: 1
duration: 4s
topology:
  chain:
    nodes: [client, edge, core, server]
    links:
      - both: { delay: 5ms, rate: 50mbit }        # client - edge
      - both: { delay: 15ms, rate: 10mbit }       # edge - core
      - both: { delay: 5ms }                      # core - server
workloads:
  - { id: stream, kind: udp-cbr, from: client, to: server, rate: 2mbit, packet-size: 1000b }

The delay of the stream is the sum of the three hops, 25 ms, and its rate is held to the smallest rate on the path. netscenario show chain.yaml prints the nodes, the links, the routes and the path the stream takes; netscenario expand chain.yaml prints the long form.

A star

Three hosts around a hub with their own values, and a call from one of them to another through the hub:

# file: star.yaml
version: 0
name: star
seed: 2
duration: 4s
topology:
  star:
    hub: hub
    leaves: [phone, laptop, server]
    links:
      - both: { delay: 30ms, jitter: { amount: 5ms, distribution: normal }, rate: 20mbit }
      - both: { delay: 3ms, rate: 100mbit }
      - both: { delay: 1ms }
workloads:
  - { id: call, kind: udp-cbr, from: phone, to: server, rate: 1mbit, packet-size: 800b }

The call crosses two links, the phone’s and the server’s, so its delay is near 31 ms. The laptop is in the network but carries nothing; add a workload from it and both flows share the hub.

By hand

The shortcuts are only a way to write nodes and links. Any network of hosts and routers can be written in the long form, with routes to set a route by hand over the computed ones (routing: manual turns the computed ones off). The files scenarios/three-hops.yaml and scenarios/manual-routes.yaml show both, and scenarios/ipv6.yaml the same with IPv6 addresses.

A dumbbell

Two senders on the left, two receivers on the right, and one bottleneck between two routers that the senders share. It is the classic shape for the question “do two flows share a link fairly?”, and the shortcut makes it in a few lines:

# file: fairness.yaml
version: 0
name: fairness
seed: 11
duration: 9s
topology:
  dumbbell:
    left: [alice, bob]
    right: [carol, dave]
    access:
      both: { delay: 2ms, rate: 100mbit }
    bottleneck:
      forward: { delay: 5ms, rate: 10mbit, queue: { discipline: fifo, limit: 30p } }
      reverse: { delay: 5ms }
workloads:
  - { id: first, kind: tcp-bulk, from: alice, to: carol, cc: reno, stop: 8s }
  - { id: second, kind: tcp-bulk, from: bob, to: dave, cc: reno, stop: 8s }
measure:
  series: { bin: 1s }

The two flows start together, have the same round trip and send until stop. The bottleneck is 10 mbit with a queue of 30 packets, about two bandwidth-delay products. Read the goodput of each flow after the start-up and judge the share with the Jain index (1 is perfectly fair, 0.5 is one flow taking everything):

import netscenario as ns

result = ns.run(ns.Scenario.load("fairness.yaml"))
window = slice(2, 8)                         # leave the start-up out
a = result.tcp["first"].goodput.bits_per_second[window]
b = result.tcp["second"].goodput.bits_per_second[window]
share_a, share_b = sum(a) / len(a), sum(b) / len(b)
jain = (share_a + share_b) ** 2 / (2 * (share_a**2 + share_b**2))
print(f"{share_a / 1e6:.2f} and {share_b / 1e6:.2f} mbit, Jain index {jain:.2f}")
assert share_a + share_b > 8e6               # together they fill the bottleneck

What to take from it:

• Together the flows fill the bottleneck, and nothing more: the sum of the goodputs is the rate of the link less the headers.
• One run is one sample. The imbalance between two flows varies between runs, so a claim about fairness is made on several runs: --repetitions 6 and a bound on the mean of the index. The test of this release does exactly that, with the bounds worked out in the file.
• Change one thing and the shape changes. Give one access link 22 ms more delay and the flow with the shorter round trip takes most of the bottleneck (scenarios/tcp-fairness-unequal-rtt.yaml). Swap fifo for codel and the queue keeps its delay near a target (scenarios/tcp-queue.yaml).

Several paths

Between two nodes there may be more than one way: a fast one and a slow one, two ISPs, a cellular and a Wi-Fi link. multipath writes the paths, and routes with a weight says how connections are spread over them.

# file: multipath.yaml
version: 0
name: multipath
seed: 31
duration: 6s
topology:
  multipath:
    from: client
    to: server
    paths:
      - routers: [slow]
        link: { both: { delay: 5ms, rate: 4mbit, queue: { limit: 30p } } }
      - routers: [wide]
        link: { both: { delay: 5ms, rate: 8mbit, queue: { limit: 30p } } }
  routes:
    - { node: client, to: server, via: client-slow, weight: 1 }
    - { node: client, to: server, via: client-wide, weight: 2 }
workloads:
  - { id: flows, kind: tcp-bulk, from: client, to: server, cc: reno, connections: 16, stop: 5s }
measure:
  series: { bin: 1s }

Sixteen TCP connections are spread over the two paths by their ports (the kernel hashes them, and the backend asks it to look at the ports). Each connection stays on one path, and each path is full as long as it has a connection, so together they carry the sum of the rates, 12 mbit, which no single path can:

import netscenario as ns

result = ns.run(ns.Scenario.load("multipath.yaml"))
goodput = result.tcp["flows"].goodput.bits_per_second
steady = goodput[2:5]                        # after the start-up, before the stop
print(f"{sum(steady) / len(steady) / 1e6:.1f} mbit on average")

The weights 1 and 2 are the rates of the paths, so each path gets connections in proportion to what it can carry. With weights that do not match the rates, a path overflows while the other idles; scenarios/tcp-multipath-weights.yaml counts the share each path got with a capture.

Only the way from the client is listed here; the acknowledgements take the first path, which has room for them. A path can go down with the timeline (Changes over time) to see what the connections do when it does: scenarios/two-paths.yaml has a fast path that goes down and comes back.

A link that moves

A phone in a car, a satellite passing overhead, a signal that comes and goes: the link is not one set of numbers but a pattern over time. There are three ways to write it, from the simplest to the most exact.

States that repeat

A periodic pattern describes a link that is in view for a time, goes quiet for a handover, and is weak for a while, again and again. The link is written once with its values for the good state, and each state lists what differs:

# file: pass.yaml
version: 0
name: pass
seed: 4
duration: 9s
topology:
  nodes: [terminal, gateway]
  links:
    - id: hop
      between: [terminal, gateway]
      both: { delay: 35ms, rate: 20mbit, queue: { limit: 100p } }
      model:
        kind: periodic
        states:
          - { id: in-view, for: 3s }
          - { id: handover, for: 1s, outage: true }
          - { id: weak, for: 2s, set: { both: { rate: 4mbit, loss: { model: random, p: 1% } } } }
workloads:
  - { id: stream, kind: udp-cbr, from: terminal, to: gateway, rate: 3mbit, packet-size: 1200b }
measure:
  series: { bin: 500ms }

In the handover nothing is delivered, and in the weak state the link has 4 mbit and a little loss. A low orbit satellite is the same pattern with 40 s, 6 s and 14 s: scenarios/leo-pass.yaml. The same idea with random states is a Markov chain: three signal states, a mean time in each drawn from the seed, and the chances of the next one (scenarios/signal-states.yaml). The same seed gives the same states and the same lost packets.

A trace you recorded

A drive through a cell can be a CSV file of moments and values. Save this as cell-drive.csv:

time,rate,delay,loss
0s,18mbit,38ms,0%
2s,14mbit,40ms,0%
4s,6mbit,55ms,1%
6s,2mbit,80ms,3%
8s,18mbit,38ms,0%

and refer to it from the link:

# file: cell.yaml
version: 0
name: cell
seed: 5
duration: 10s
topology:
  nodes: [phone, server]
  links:
    - id: cell
      between: [phone, server]
      both: { queue: { limit: 200p } }
      model:
        kind: trace
        both: { file: cell-drive.csv }
workloads:
  - { id: stream, kind: udp-cbr, from: server, to: phone, rate: 6mbit, packet-size: 1200b }
measure:
  series: { bin: 1s }

The file is read once when the scenario is loaded and lives in the recorded scenario, so a replay needs nothing from the original directory. The trace begins again when it is over. A Mahimahi trace works the same way for a rate that follows a recorded sequence of delivery opportunities; the Kernel backend follows it in windows, which is only an approximation, so the scenario has to accept that with allow-limited (scenarios/mahimahi-walk.yaml). Nothing is approximated without you saying so.

Events that repeat

An outage every ten seconds, a rate that halves every twenty: every with until or count on a timeline event (Changes over time).

# file: flapping.yaml
version: 0
name: flapping
seed: 2
duration: 8s
topology:
  nodes: [client, server]
  links:
    - id: access
      between: [client, server]
      both: { delay: 20ms, rate: 10mbit }
workloads:
  - { id: stream, kind: udp-cbr, from: client, to: server, rate: 4mbit, packet-size: 1200b }
timeline:
  - { at: 2s, link: access, outage: 400ms, every: 3s, until: 7s }
measure:
  series: { bin: 250ms }

Two outages fall in the 8 seconds, and for each one the result has the time the stream took to recover. A test can assert that a stream is back at its level within a number of seconds, which is the question after a handover.

Reference documents

The pages of this book explain and show. The exact definition of every key, field, option and decision lives in documents next to the code, and they are linked here rather than copied, so there is one place to be right.

Using it

The scenario and the result

How it works and what it promises