Baselines¶

NetForge ships scripted and learned baselines so you have reference points before training your own agents. They live in netforge_rl.baselines.

Policies¶

Policy	Team	Behaviour
`RandomPolicy`	any	Uniform over `MultiDiscrete([32, 100])`
`HeuristicBluePolicy`	blue	Isolate the first compromised host; otherwise analyse
`HeuristicRedPolicy`	red	Fire an exploit at a random host (no recon — a weak lower bound)
`KillChainRedPolicy`	red	Full recon → exploit → pivot kill-chain; actually compromises hosts
`jax_ppo`	any	JAX PPO/IPPO reference implementation (`baselines/jax_ppo.py`)

Why the kill-chain matters¶

ExploitRemoteService requires a prior DiscoverNetworkServices on the target and a routable path. HeuristicRedPolicy skips recon, so its exploits always fail the prerequisite check and it compromises nothing — a misleadingly quiet benchmark. KillChainRedPolicy port-scans a reachable, vulnerable host, exploits it, and lets footholds expand DMZ → Corporate → Secure as routing opens up. Against a defending blue policy it compromises ~2–3 hosts per episode and drives SLA down, producing a genuine red/blue contest.

from netforge_rl.baselines.policies import KillChainRedPolicy, HeuristicBluePolicy

Evidence: kill-chain vs. everything else¶

build_leaderboard (python -m benchmarks.build_leaderboard --episodes 5 --max-steps 150, see Run) runs every scripted policy across the scenarios. Compromise counts, active hosts only, 5 episodes × 150 ticks:

Policy	ransomware	apt_espionage	ot_stuxnet	iot_grid
`killchain-red`	~2.8	~2.8	~3.0	~1.8–2.6
`heuristic-red` / `random` / `heuristic-blue`	0.0	0.0	0.0	0.0

See Run for the full sweep/scoring commands and the composite-score formula.

Learned baselines¶

NetForge ships a JIT-fused JAX IPPO trainer (netforge_rl/baselines/jax_ppo.py) whose entire rollout — environment step, observation, policy, GAE — runs on-device via lax.scan. benchmarks/train_curve.py trains it, records the learning curve, and checkpoints the policy:

python -m benchmarks.train_curve --name blue_ransomware --iters 40
# -> results/ippo_curve_blue_ransomware.json  (reward + loss curves)
# -> results/ippo_curve_blue_ransomware.png   (chart)
# -> results/ippo_blue_ransomware.npz         (checkpoint; load with load_params)

A committed 40-iteration run on ransomware (245,760 env-steps, ~60 s on CPU) shows the blue policy learning to defend: mean reward 0.06 → 0.71. Checkpoints reload with jax_ppo.load_params and can be dropped straight into the self-play tournament or a capability card.

The RLlib R-MAPPO bridge (benchmarks/rllib_rmappo.py) is available for CPU/GPU multi-agent training with Ray.