A program of your own in the network

A workload can also be your own program, started in a node: a server, a client, the service you are testing. It then sends real packets through the network, and you test what it does over a bad link instead of only what the generators measure.

command runs code, so it is written in the scenario file where everybody can see it. The example is a UDP echo server on one side and a client on the other, over a link with 30 ms of delay each way and 5 % loss each way:

# Your own program in a node: a UDP echo server on one side, a client on the other, over a link with 30 ms of delay each
# way and 5 % loss each way. `command` runs code, so it is written in the file where everybody can see it (SECURITY.md).
# The server is started first and the client when the server has a UDP socket open on port 4433; each program reports
# what it measured as lines of JSON on its standard output (`{"metric": ..., "value": ...}`), and the result has them.
version: 0
name: echo-over-a-bad-link
seed: 21
duration: 6s
topology:
  nodes: [client, server]
  links:
    - id: wan
      between: [client, server]
      both:
        delay: 30ms
        loss: { model: random, p: 5% }
workloads:
  - id: server
    kind: command
    node: server
    run: ["python3", "echo_server.py", "{server.addr}", "4433"]
    ready: { udp-port: 4433, within: 5s }
    shutdown: { signal: TERM, within: 2s }
  - id: client
    kind: command
    node: client
    run: ["python3", "echo_client.py", "{server.addr}", "4433", "200"]

Three things in it are worth knowing:

• {server.addr} is the address of a node, filled in when the network is built.
• ready says when the program is ready: the client is started when the server has a UDP socket open on port 4433, not after a guessed sleep.
• shutdown says how a program that does not end by itself is stopped (here SIGTERM, then the run waits up to two seconds).

A program reports what it measured as lines of JSON on its standard output, {"metric": "rtt", "value": 61.2}, and the result has them. The test reads the exit of each program and its metrics:

"""Your own program in a node, tested over a bad link (needs `pytest-netscenario`)."""

from pytest_netscenario import Runner


def test_the_echo_server_answers_over_a_lossy_link(netscenario: Runner) -> None:
    result = netscenario.run("echo.yaml")

    client = result.programs["client"]
    # The client ended by itself with 0; the server was stopped by the run (signal of `shutdown`)
    assert client.succeeded
    assert result.programs["server"].how_it_ended == "stopped"

    sent = client.metrics["sent"].last
    answered = client.metrics["answered"].last
    assert sent == 200
    # A packet is answered if neither it nor its answer is lost: 0.95 * 0.95 = 90.25 % of 200 is
    # 180, with a standard deviation of 4.2; four of them each way round are 17 packets
    assert 163 <= answered <= 198

    rtt = sorted(value for _, name, value in client.readings() if name == "rtt")
    assert len(rtt) == answered
    # The link takes 30 ms each way: no answer comes faster than 60 ms, and the median is near
    assert rtt[0] >= 60
    assert rtt[len(rtt) // 2] < 90

The bounds in that test are derived and written next to it: a packet is answered if neither it nor its answer is lost, so 90.25 % of 200 packets is about 180 with a standard deviation of 4.2, and no answer comes faster than the 60 ms the link takes both ways.

The programs run in the directory of the scenario file (change it with working_dir=), without a shell, with the environment you give them. Read Programs of your own for the limits a run sets on them, and the security notes before you run a scenario file you did not write: a command is code.