Agent Configuration

The FBOSS agent is configured by a single JSON file, /etc/coop/agent.conf by default. This page explains the file's format, how the agent loads it (including a silent fallback that catches people out), how defaultCommandLineArgs turn config entries into command-line flags, the main SwitchConfig keys with their defaults, where the other services' config files live, and how configs are generated for real platforms.

Where the Config Lives

Service Default path Flag to override
Agent (wedge_agent, fboss_sw_agent, fboss_hw_agent) /etc/coop/agent.conf --config
qsfp_service /etc/coop/qsfp.conf --qsfp_config
led_service /etc/coop/led.conf --led_config
fsdb /etc/coop/fsdb/current --fsdb_config
Platform services (sensor_service, fan_service, platform_manager, ...) Built into the binary per platform --config_file

When the config is managed with fboss2-dev, /etc/coop is a local Git repository and agent.conf is usually a symlink to cli/agent.conf. See fboss2-dev Config Sessions.

File Format: AgentConfig

The file is the JSON form of the Thrift struct AgentConfig from fboss/agent/agent_config.thrift:

Key Type Default Meaning
defaultCommandLineArgs map of string to string {} Default values for the agent's command-line flags (gflags). See below.
sw SwitchConfig The switch configuration: ports, VLANs, interfaces, routes, ACLs, QoS, and so on. Defined in fboss/agent/switch_config.thrift.
platform PlatformConfig Hardware bring-up settings: the ASIC configuration (chip), per-switch-index chip configs and settings, and the switchIndexToSwitchId map. Defined in fboss/agent/platform_config.thrift.
thriftApiToRateLimitInQps map of string to double {} Per-Thrift-API rate limits. The agent runs rate limiting in shadow mode by default (--thrift_rate_limit_shadow_mode=true).

A skeleton looks like this:

{
  "defaultCommandLineArgs": {
    "multi_switch": "true",
    "enable_lldp": "true",
    "tun_intf": "false"
  },
  "sw": {
    "version": 0,
    "ports": [],
    "vlans": [],
    "vlanPorts": [],
    "defaultVlan": 4094,
    "interfaces": []
  },
  "platform": {
    "chip": {}
  }
}

The fboss-sim runtime's config (fboss-sim/docker/runtime/mono.conf) is a complete working example; its split variant is created simply by setting defaultCommandLineArgs.multi_switch to "true".

The SwitchConfig Fallback

The agent parses the file with Thrift's SimpleJSONSerializer, which is "extremely permissive about unexpected keys and never throws" (in the words of the comment in AgentConfig.cpp). To decide whether the file was really an AgentConfig, the agent checks whether the parsed sw member differs from a default SwitchConfig. If it does not, it prints

Not valid AgentConfig, fallback to parsing as SwitchConfig...

to stderr and parses the whole file as a bare SwitchConfig. This is why old files such as fboss/agent/configs/sample1.json, which have ports and vlans at the top level and no sw wrapper, still load.

Consequences worth knowing:

  • A typo in a key name is ignored silently rather than rejected.
  • If your file has a sw key whose content is entirely default, the agent treats the file as a SwitchConfig, and your defaultCommandLineArgs and platform sections are lost.
  • A bare SwitchConfig file has no platform section, which real hardware needs.

If the file cannot be read at all, start-up fails with unable to read <path>.

defaultCommandLineArgs

Before full flag parsing, the agent reads the config (honoring --config on the command line) and, for each entry in defaultCommandLineArgs, sets that gflag's default value. Each override is logged to stderr:

Overriding default flag from config: multi_switch=true

Because the config only changes defaults, a flag given explicitly on the command line still wins. Values are strings even for booleans and numbers. The same mechanism is used by the software agent, the hardware agent (HwAgentMain.cpp), and the pre-start step. qsfp_service has an equivalent defaultCommandLineArgs map in its own config.

Commonly set entries (taken from the configs in the repository):

Flag Meaning
multi_switch Run in split mode (fboss_sw_agent + hardware agents). Default false.
enable_lldp Run LLDP in the agent. Default true.
tun_intf Create TUN interfaces so other processes can send and receive through switch ports. Default true. The simulator sets it to false.
can_warm_boot Whether the agent should prepare for warm boot on exit.
enable_replayer Record SAI calls for the SAI replayer.
publish_state_to_fsdb, publish_stats_to_fsdb Publish switch state or stats to FSDB. Both default false.

All flags are listed on Service Flags and Ports.

Main SwitchConfig Keys

SwitchConfig has over 60 fields; these are the ones nearly every config uses. Defaults are from switch_config.thrift.

Key Type Default Meaning
version i64 0 Config version number.
ports list of Port One entry per front-panel or fabric port.
vlans list of Vlan [] VLANs: id, name, routable, ipAddresses, DHCP relay settings.
vlanPorts list of VlanPort [] VLAN membership: vlanID, logicalPort, spanningTreeState, emitTags.
defaultVlan i32 VLAN for untagged traffic.
interfaces list of Interface [] L3 interfaces: intfID, vlanID, routerID (default 0), mac, ipAddresses, mtu, ndp, type.
arpTimeoutSeconds i32 60 ARP/NDP entry timeout.
arpRefreshSeconds i32 20 Refresh interval for neighbor entries.
arpAgerInterval i32 5 How often the neighbor ager runs.
maxNeighborProbes i32 300 Probes before a neighbor is considered unreachable.
staleEntryInterval i32 10 Interval for stale neighbor handling.
staticRoutesWithNhops, staticRoutesToNull, staticRoutesToCPU lists [] Static routes.
acls, aclTableGroups lists ACL entries and table groups.
clientIdToAdminDistance map see below Admin distance per route client.
aggregatePorts, lacp Link aggregation.
qosPolicies, portQueueConfigs, defaultPortQueues, cpuQueues QoS and queuing.
cpuTrafficPolicy, dataPlaneTrafficPolicy Control-plane policing and data-plane policies.
loadBalancers list [] ECMP and LAG hashing.
mirrors, sFlowCollectors lists [] Mirroring and sFlow.
switchSettings SwitchSettings Switch-wide settings (L2 learning mode, switch type and ID, ...).
dsfNodes map {} Distributed scheduled fabric (VOQ) node definitions.
hostname optional string Switch hostname.

The default clientIdToAdminDistance map is: interface routes (client 2), link-local routes (3), and remote interface routes (4) get 0; static routes (1) get 1; TE agent (800) gets 2; Open/R (786) gets 10; BGP (0) gets 20; static internal (700) gets 255.

Port Keys

Key Default Meaning
logicalID Port ID, as defined by the platform mapping.
state DISABLED (1) Admin state: DISABLED = 1, ENABLED = 2 (DOWN = 0 is deprecated).
speed DEFAULT Port speed.
profileID PROFILE_DEFAULT Port profile (speed, lanes, FEC, media), must exist in the platform mapping.
name, description Port name such as eth1/1/1 and free text.
ingressVlan VLAN for untagged ingress traffic.
routable Whether the port participates in L3.
minFrameSize, maxFrameSize 64, 9412 Frame size limits (DEFAULT_PORT_MTU is 9412).
loopbackMode NONE MAC or PHY loopback, used heavily by hardware tests.
pause, pfc NO_PAUSE Flow control.
portType INTERFACE_PORT Interface, fabric, recycle, management, ...
drainState UNDRAINED Drained ports carry no traffic.
expectedLLDPValues {} LLDP values checked against neighbors.

Getting a Config for Your Platform

The JSON in agent.conf is not meant to be written by hand. The README notes that Meta uses tooling to generate it. In the open source repo you have three sources:

  1. Test configs: fboss/oss/hw_test_configs/<platform>.agent.materialized_JSON (for example fuji.agent.materialized_JSON, montblanc.agent.materialized_JSON), link test configs in fboss/oss/link_test_configs/, and QSFP test configs in fboss/oss/qsfp_test_configs/. They are packaged into /opt/fboss/share/ (see Packaging and Running on a Switch).
  2. Default runtime configs: fboss/oss/scripts/run_configs/default_configs/<platform>/ holds agent.conf, qsfp.conf, led.conf, and num_hw_agents for some platforms (for example minipack3, m4061nhp).
  3. Config generation: fboss2 config gen agent generates agent configs from the platform's sources in fboss/configs/platforms/<vendor>/<platform>/forwarding_stack/. Its options are --fboss-root (required), --platform, --profile, --asic-config-variant, --asic-config-file, --asic-config-type (key_value, json, or yaml), --reference-config-file, --output-dir, and --verify. --verify checks that the checked-in generated configs are up to date and reports missing, stale, and obsolete files; it cannot be combined with --profile, and --asic-config-variant cannot be combined with --asic-config-file.

Running Config vs Startup Config

The startup config is the file on disk. The running config is what the agent applied, saved as /var/facebook/fboss/running-agent.conf, and shown by fboss2 show config running agent. They can differ when something changed state directly through Thrift, or when you used runtime fboss2 set ... commands, which are never written back to the file.

To apply a changed file, use fboss2-dev config reload (or config session commit, which reloads or restarts only what is needed), or restart the agent.

Best Practices

  • Start from the packaged test config for your platform and change as little as possible.
  • Keep defaultCommandLineArgs for run-mode flags (multi_switch, logging, warm boot) so systemd units can stay generic.
  • After editing by hand, validate the JSON with a JSON tool, because the agent will not complain about unknown keys.
  • Prefer fboss2-dev config sessions on a live switch: they keep history and support rollback.

See Also