Agent Configuration
The FBOSS agent is configured by a single JSON file, /etc/coop/agent.conf by default. This page explains the file's format, how the agent loads it (including a silent fallback that catches people out), how defaultCommandLineArgs turn config entries into command-line flags, the main SwitchConfig keys with their defaults, where the other services' config files live, and how configs are generated for real platforms.
Where the Config Lives
| Service | Default path | Flag to override |
|---|---|---|
Agent (wedge_agent, fboss_sw_agent, fboss_hw_agent) |
/etc/coop/agent.conf |
--config |
qsfp_service |
/etc/coop/qsfp.conf |
--qsfp_config |
led_service |
/etc/coop/led.conf |
--led_config |
fsdb |
/etc/coop/fsdb/current |
--fsdb_config |
Platform services (sensor_service, fan_service, platform_manager, ...) |
Built into the binary per platform | --config_file |
When the config is managed with fboss2-dev, /etc/coop is a local Git repository and agent.conf is usually a symlink to cli/agent.conf. See fboss2-dev Config Sessions.
File Format: AgentConfig
The file is the JSON form of the Thrift struct AgentConfig from fboss/agent/agent_config.thrift:
| Key | Type | Default | Meaning |
|---|---|---|---|
defaultCommandLineArgs |
map of string to string | {} |
Default values for the agent's command-line flags (gflags). See below. |
sw |
SwitchConfig |
The switch configuration: ports, VLANs, interfaces, routes, ACLs, QoS, and so on. Defined in fboss/agent/switch_config.thrift. |
|
platform |
PlatformConfig |
Hardware bring-up settings: the ASIC configuration (chip), per-switch-index chip configs and settings, and the switchIndexToSwitchId map. Defined in fboss/agent/platform_config.thrift. |
|
thriftApiToRateLimitInQps |
map of string to double | {} |
Per-Thrift-API rate limits. The agent runs rate limiting in shadow mode by default (--thrift_rate_limit_shadow_mode=true). |
A skeleton looks like this:
{
"defaultCommandLineArgs": {
"multi_switch": "true",
"enable_lldp": "true",
"tun_intf": "false"
},
"sw": {
"version": 0,
"ports": [],
"vlans": [],
"vlanPorts": [],
"defaultVlan": 4094,
"interfaces": []
},
"platform": {
"chip": {}
}
}
The fboss-sim runtime's config (fboss-sim/docker/runtime/mono.conf) is a complete working example; its split variant is created simply by setting defaultCommandLineArgs.multi_switch to "true".
The SwitchConfig Fallback
The agent parses the file with Thrift's SimpleJSONSerializer, which is "extremely permissive about unexpected keys and never throws" (in the words of the comment in AgentConfig.cpp). To decide whether the file was really an AgentConfig, the agent checks whether the parsed sw member differs from a default SwitchConfig. If it does not, it prints
Not valid AgentConfig, fallback to parsing as SwitchConfig...
to stderr and parses the whole file as a bare SwitchConfig. This is why old files such as fboss/agent/configs/sample1.json, which have ports and vlans at the top level and no sw wrapper, still load.
Consequences worth knowing:
- A typo in a key name is ignored silently rather than rejected.
- If your file has a
swkey whose content is entirely default, the agent treats the file as aSwitchConfig, and yourdefaultCommandLineArgsandplatformsections are lost. - A bare
SwitchConfigfile has noplatformsection, which real hardware needs.
If the file cannot be read at all, start-up fails with unable to read <path>.
defaultCommandLineArgs
Before full flag parsing, the agent reads the config (honoring --config on the command line) and, for each entry in defaultCommandLineArgs, sets that gflag's default value. Each override is logged to stderr:
Overriding default flag from config: multi_switch=true
Because the config only changes defaults, a flag given explicitly on the command line still wins. Values are strings even for booleans and numbers. The same mechanism is used by the software agent, the hardware agent (HwAgentMain.cpp), and the pre-start step. qsfp_service has an equivalent defaultCommandLineArgs map in its own config.
Commonly set entries (taken from the configs in the repository):
| Flag | Meaning |
|---|---|
multi_switch |
Run in split mode (fboss_sw_agent + hardware agents). Default false. |
enable_lldp |
Run LLDP in the agent. Default true. |
tun_intf |
Create TUN interfaces so other processes can send and receive through switch ports. Default true. The simulator sets it to false. |
can_warm_boot |
Whether the agent should prepare for warm boot on exit. |
enable_replayer |
Record SAI calls for the SAI replayer. |
publish_state_to_fsdb, publish_stats_to_fsdb |
Publish switch state or stats to FSDB. Both default false. |
All flags are listed on Service Flags and Ports.
Main SwitchConfig Keys
SwitchConfig has over 60 fields; these are the ones nearly every config uses. Defaults are from switch_config.thrift.
| Key | Type | Default | Meaning |
|---|---|---|---|
version |
i64 | 0 |
Config version number. |
ports |
list of Port |
One entry per front-panel or fabric port. | |
vlans |
list of Vlan |
[] |
VLANs: id, name, routable, ipAddresses, DHCP relay settings. |
vlanPorts |
list of VlanPort |
[] |
VLAN membership: vlanID, logicalPort, spanningTreeState, emitTags. |
defaultVlan |
i32 | VLAN for untagged traffic. | |
interfaces |
list of Interface |
[] |
L3 interfaces: intfID, vlanID, routerID (default 0), mac, ipAddresses, mtu, ndp, type. |
arpTimeoutSeconds |
i32 | 60 |
ARP/NDP entry timeout. |
arpRefreshSeconds |
i32 | 20 |
Refresh interval for neighbor entries. |
arpAgerInterval |
i32 | 5 |
How often the neighbor ager runs. |
maxNeighborProbes |
i32 | 300 |
Probes before a neighbor is considered unreachable. |
staleEntryInterval |
i32 | 10 |
Interval for stale neighbor handling. |
staticRoutesWithNhops, staticRoutesToNull, staticRoutesToCPU |
lists | [] |
Static routes. |
acls, aclTableGroups |
lists | ACL entries and table groups. | |
clientIdToAdminDistance |
map | see below | Admin distance per route client. |
aggregatePorts, lacp |
Link aggregation. | ||
qosPolicies, portQueueConfigs, defaultPortQueues, cpuQueues |
QoS and queuing. | ||
cpuTrafficPolicy, dataPlaneTrafficPolicy |
Control-plane policing and data-plane policies. | ||
loadBalancers |
list | [] |
ECMP and LAG hashing. |
mirrors, sFlowCollectors |
lists | [] |
Mirroring and sFlow. |
switchSettings |
SwitchSettings |
Switch-wide settings (L2 learning mode, switch type and ID, ...). | |
dsfNodes |
map | {} |
Distributed scheduled fabric (VOQ) node definitions. |
hostname |
optional string | Switch hostname. |
The default clientIdToAdminDistance map is: interface routes (client 2), link-local routes (3), and remote interface routes (4) get 0; static routes (1) get 1; TE agent (800) gets 2; Open/R (786) gets 10; BGP (0) gets 20; static internal (700) gets 255.
Port Keys
| Key | Default | Meaning |
|---|---|---|
logicalID |
Port ID, as defined by the platform mapping. | |
state |
DISABLED (1) |
Admin state: DISABLED = 1, ENABLED = 2 (DOWN = 0 is deprecated). |
speed |
DEFAULT |
Port speed. |
profileID |
PROFILE_DEFAULT |
Port profile (speed, lanes, FEC, media), must exist in the platform mapping. |
name, description |
Port name such as eth1/1/1 and free text. |
|
ingressVlan |
VLAN for untagged ingress traffic. | |
routable |
Whether the port participates in L3. | |
minFrameSize, maxFrameSize |
64, 9412 |
Frame size limits (DEFAULT_PORT_MTU is 9412). |
loopbackMode |
NONE |
MAC or PHY loopback, used heavily by hardware tests. |
pause, pfc |
NO_PAUSE |
Flow control. |
portType |
INTERFACE_PORT |
Interface, fabric, recycle, management, ... |
drainState |
UNDRAINED |
Drained ports carry no traffic. |
expectedLLDPValues |
{} |
LLDP values checked against neighbors. |
Getting a Config for Your Platform
The JSON in agent.conf is not meant to be written by hand. The README notes that Meta uses tooling to generate it. In the open source repo you have three sources:
- Test configs:
fboss/oss/hw_test_configs/<platform>.agent.materialized_JSON(for examplefuji.agent.materialized_JSON,montblanc.agent.materialized_JSON), link test configs infboss/oss/link_test_configs/, and QSFP test configs infboss/oss/qsfp_test_configs/. They are packaged into/opt/fboss/share/(see Packaging and Running on a Switch). - Default runtime configs:
fboss/oss/scripts/run_configs/default_configs/<platform>/holdsagent.conf,qsfp.conf,led.conf, andnum_hw_agentsfor some platforms (for exampleminipack3,m4061nhp). - Config generation:
fboss2 config gen agentgenerates agent configs from the platform's sources infboss/configs/platforms/<vendor>/<platform>/forwarding_stack/. Its options are--fboss-root(required),--platform,--profile,--asic-config-variant,--asic-config-file,--asic-config-type(key_value,json, oryaml),--reference-config-file,--output-dir, and--verify.--verifychecks that the checked-in generated configs are up to date and reports missing, stale, and obsolete files; it cannot be combined with--profile, and--asic-config-variantcannot be combined with--asic-config-file.
Running Config vs Startup Config
The startup config is the file on disk. The running config is what the agent applied, saved as /var/facebook/fboss/running-agent.conf, and shown by fboss2 show config running agent. They can differ when something changed state directly through Thrift, or when you used runtime fboss2 set ... commands, which are never written back to the file.
To apply a changed file, use fboss2-dev config reload (or config session commit, which reloads or restarts only what is needed), or restart the agent.
Best Practices
- Start from the packaged test config for your platform and change as little as possible.
- Keep
defaultCommandLineArgsfor run-mode flags (multi_switch, logging, warm boot) so systemd units can stay generic. - After editing by hand, validate the JSON with a JSON tool, because the agent will not complain about unknown keys.
- Prefer
fboss2-dev configsessions on a live switch: they keep history and support rollback.