Skip to content

Dynamic & Non-Stationary Topologies

NetForge RL implements non-stationary network topologies to force reinforcement learning agents to learn robust, generalizable policies rather than overfitting to static routing tables. Real-world enterprise networks are dynamic—devices connect, disconnect, and migrate between subnets constantly.

This behavior is governed by the TopologyEventEngine.

The Topology Event Engine

The TopologyEventEngine sits within the core environment loop and emits topology state-change events mid-episode. It governs three primary dynamics:

  1. Churn (Node Status): Nodes randomly transition between online and offline (isolated) states, simulating devices going to sleep or being rebooted.
  2. Migration (Node Movement): Active nodes may randomly migrate from one subnet to another (e.g., a corporate laptop moving from a WiFi VLAN to a wired connection).
  3. Arrival (BYOD Injection): Zero-padded "phantom" nodes are dynamically instantiated into active subnets as new "Bring Your Own Device" (BYOD) endpoints with randomized OS profiles and running services.

Topology Events

When the engine alters the state, it emits a TopologyEvent to the event queue. These events are parsed by the simulation to invalidate stale agent knowledge.

  • host_offline: A previously active host is marked as isolated.
  • host_online: An isolated host is brought back online.
  • host_migrate: A host changes its IP address and subnet_cidr. The old IP is wiped from the GlobalNetworkState and agent knowledge bases.
  • host_arrive: A new BYOD endpoint joins a subnet with a newly leased IP address.

Configuration

The rates for these events are controlled via environment initialization parameters (which are also dynamically scaled by the Curriculum Wrapper):

cfg = {
    'topology_churn_rate': 0.02,     # 2% chance per tick for node status flip
    'topology_migration_rate': 0.01, # 1% chance per tick for subnet migration
    'topology_arrival_rate': 0.005,  # 0.5% chance per tick for new BYOD
}
env = NetForgeRLEnv(cfg)