Skip to content

Difficulty & Evaluation Splits

NetForge exposes difficulty as reproducible, named tiers rather than ad-hoc config dicts, and ships a frozen held-out evaluation seed suite so that difficulty and the train/eval split are comparable across runs, machines, and papers.

All of this lives in netforge_rl.environment.presets.

Presets

from netforge_rl.environment import make_env, make_config, DIFFICULTY_PRESETS

env = make_env('hard', scenario_type='ot_stuxnet', seed=0)   # build + reset
cfg = make_config('easy')                                    # just the config dict

Each tier sets the environment's difficulty knobs:

Knob easy medium hard Effect
max_active_hosts 6 15 100 (uncapped) size of the live network
dhcp_interval 0 (off) 80 40 ticks between DHCP address reshuffles
topology_churn_rate 0.0 0.01 0.02 hosts leaving/arriving per tick
topology_migration_rate 0.0 0.0 0.01 hosts changing subnet
topology_arrival_rate 0.0 0.0 0.005 brand-new hosts appearing
log_latency 0 2 4 ticks of SIEM telemetry delay
max_ticks 200 200 200 episode horizon

easy is a small, static, fully observable network; hard is a larger, non-stationary network with a lagging SOC feed. Overrides always win, so you can pin a scenario or extend the horizon without forking a preset:

cfg = make_config('medium', scenario_type='apt_espionage', max_ticks=400, log_latency=6)

The knobs, explained

  • max_active_hosts — caps the live topology. The procedural generator naturally produces 15–30 active hosts (the remaining slots are 169.254.0.0/16 padding that keeps the observation shape constant); the cap trims below that.
  • dhcp_interval — every N ticks, host IP assignments reshuffle, forcing agents to track identity through address churn. 0 disables it.
  • topology_churn_rate / migration_rate / arrival_rate — drive the TopologyEventEngine, adding non-stationarity: hosts disappear, change subnet, or appear.
  • log_latency — delays SIEM log visibility by N ticks, modelling a lagging SOC feed. 0 is immediate (the historical behaviour). See Reproducibility for how this interacts with determinism.

Held-out evaluation split

Report on topologies your agent never trained on. evaluation=True shifts topology generation into a disjoint seed range (offset 1000), and EVAL_SEEDS is a frozen 20-seed suite:

from netforge_rl.environment import make_env, EVAL_SEEDS

for seed in EVAL_SEEDS:                       # 9001..9020, frozen
    env = make_env('medium', evaluation=True, seed=seed)
    ...

The same base seed produces a different network under evaluation=True than under training, which is what makes the generalization gap meaningful:

train = make_env('medium', evaluation=False, seed=9001)
held  = make_env('medium', evaluation=True,  seed=9001)
assert set(train.global_state.all_hosts) != set(held.global_state.all_hosts)

The benchmark runner reports the train-vs-held-out generalization gap directly:

python -m benchmarks.run_benchmark --name my_agent --gap --seeds 20

Curriculum learning

For automatic difficulty progression during training, use the CurriculumWrapper, which advances through novice → intermediate → expert phases based on a rolling reward window. See Curriculum Learning.