i226-V Link Drops: Fix 2.5GbE Flapping on OPNsense

The symptom is distinctive. A 2.5GbE port on a fanless N100 appliance links, passes traffic, then drops for a second or two and comes back. Sometimes it renegotiates down to 1 Gbps and stays there. On a WAN port it looks like an ISP fault; on a LAN trunk it looks like a switch fault. It is neither often enough that it deserves its own checklist.

An important framing before anything else. Intel’s I225 family shipped with widely reported link-stability problems across several silicon steppings, and the I226 that replaced it has not entirely escaped the reputation. Reports of intermittent drops and speed renegotiation on i226-V ports are common in owner forums and vendor support threads. What follows separates what is documented by the platform vendors, which is most of the fixes, from what is community-reported, which is the underlying fault pattern. Treat the ladder below as an ordered diagnostic sequence rather than a set of tweaks to apply at once.

Step 0: confirm the port is actually an i226

This costs thirty seconds and it changes everything that follows.

ifconfig -a | grep -E '^(igc|re|em|igb)'

On FreeBSD-based platforms, Intel I225 and I226 controllers are driven by igc(4), which the FreeBSD manual page lists as covering the I225 and I226 families, including the I225-V, I225-IT, I225-LM, I226-V, I226-IT, I226-LM and I226-K, along with the earlier I220-V and I221-V. If the interfaces are named igc0 through igc3, the board has Intel controllers.

If they come up as re0, re1 and so on, the appliance has Realtek RTL8125 controllers instead. That is a different problem with a different history, and the advice below about ASPM and flow control does not map cleanly onto it. It also means the listing was not describing what shipped.

Then confirm the negotiated media:

ifconfig igc0 | grep -E 'media|status'

igc(4) supports full duplex only at 1000 and 2500 Mbps, and half duplex only at 10 and 100 Mbps. A port that has settled on 1000baseT <full-duplex> when 2.5GbE was expected has already renegotiated once, which is itself the symptom.

Step 1: eliminate the cable and the far end

This is not filler. 2.5GBASE-T is far more sensitive to marginal cabling than gigabit, and it is the single most common cause of a link that works at 1 Gbps and flaps at 2.5.

• Swap the patch cable for a known-good short Cat 6 or Cat 6a. Old Cat 5e runs will often carry gigabit indefinitely and fail intermittently at 2.5.
• Move the link to a different port on the switch, and to a different port on the appliance. If the fault follows the appliance port, that narrows it; if it follows the switch port, the appliance is exonerated.
• Check what the far end is. Some consumer switches and most ISP-supplied ONTs implement 2.5GbE conservatively, and a mismatch in auto-negotiation behaviour between two devices produces exactly this symptom.
• Look for a run through a wall or a keystone patch panel. Punchdown terminations that pass gigabit fine are a frequent 2.5GbE failure point.

If the flapping disappears with a short cable directly between two devices, the problem is the cable plant and no driver tunable will fix it.

Step 2: force the link speed

If the port is stable at 1 Gbps and unstable at 2.5, pinning the speed is a legitimate production answer and not just a diagnostic.

On either platform this is set per interface in the GUI: Interfaces → [interface] → Speed and duplex, choosing 1000baseT full-duplex instead of the default autoselect. It can also be set at the console with ifconfig igc0 media 1000baseT mediaopt full-duplex, though a GUI setting is the one that survives a reboot.

Two reasons to do this deliberately rather than as a shrug:

• A WAN link from an ISP that provisions 1 Gbps gains nothing from a 2.5GbE negotiation, so forcing it costs nothing and removes a whole failure class.
• On a LAN link where 2.5 Gbps is genuinely needed, forcing to 1 Gbps is a diagnosis, not a fix. If it stabilises, the fault is in the 2.5GbE negotiation path, which points at cabling, the far end, or power management.

Step 3: disable PCIe ASPM in the BIOS

Active State Power Management lets a PCIe device drop into a low-power state between bursts of activity. On generic N100 appliance boards it is often enabled by default, and it is a commonly cited BIOS-level suspect in owner reports of intermittent 2.5GbE link loss. Neither platform vendor documents it as a cause, so treat it as a cheap thing to rule out rather than a known fix.

Look under Chipset, PCH, or Power Management in the appliance BIOS for ASPM, PCI Express Native Power Management, Native ASPM, or per-root-port L1 Substates. Set them to Disabled.

While in the BIOS, two neighbouring settings are worth the same treatment on a firewall:

• Energy Efficient Ethernet (EEE), sometimes shown as Green Ethernet or EEE LPI. It parks the PHY during idle periods and its wake behaviour is a plausible contributor to a link that drops when traffic is bursty rather than steady.
• Wake on LAN, which on some boards keeps parts of the controller in a state that interacts badly with ASPM. A firewall does not need it.

None of these settings costs measurable throughput. On a device drawing single-digit watts, the power saved is not worth an unstable WAN.

Step 4: driver-level tunables

Only reach for these after the cable, the far end and the BIOS have been ruled out. Netgate’s hardware tuning documentation is the reference for all of them, and it applies to OPNsense as well because both platforms sit on FreeBSD.

Flow control. Netgate documents a per-interface tunable specifically for igc(4):

dev.igc.<id>.fc="0"

where <id> matches the interface number, so igc3 is configured under dev.igc.3. This is entered under System → Advanced → System Tunables on pfSense, or the equivalent tunables page on OPNsense. Disabling flow control is worth trying when the drops correlate with sustained transfers rather than with idle periods, because pause frames from a congested switch are one of the ways a link can appear to stall.

MSI-X. Netgate notes that some cards behave better with MSI, MSI-X or classic interrupts, and that both can be disabled with loader tunables:

hw.pci.enable_msix="0"
hw.pci.enable_msi="0"

Disabling only MSI-X nudges the card toward MSI; disabling both forces classic interrupts. These are loader tunables, so a reboot is required. Treat this as a late step: it costs interrupt efficiency and should be reverted if it does not help.

Network memory buffers. Netgate’s warning here names a different Intel driver: “Certain intel igb cards, especially multi-port cards, can easily exhaust mbufs and cause kernel panics.” The same page notes more generally that mbuf exhaustion can result in “expected interfaces not being initialized and made available by the operating system,” which is why it is worth checking on a four-port igc board even though the warning was written about igb. Check first:

netstat -m

If any pool is near its limit, raise the cluster count as a loader tunable:

kern.ipc.nmbclusters="1000000"

Netgate calls one million a safe starting point on 64-bit systems with several gigabytes of RAM, consuming roughly 2.3 GB of physical memory if fully allocated. On an 8 GB N100 build that is acceptable; on a 4 GB box it is another argument for having bought 8 GB.

Leave the offloads alone. Netgate is explicit that hardware TCP Segmentation Offload and Large Receive Offload “default to checked (disabled) for good reason,” because “nearly all hardware/drivers have issues with these settings, and they can lead to throughput issues.” If a previous troubleshooting session enabled them, that is a candidate cause rather than a fix. Netmap-based inline IPS also requires them off.

Step 5: rule out the things that are not the NIC

Three non-NIC causes produce link symptoms on this class of hardware.

Thermal throttling is not link loss, but it looks like a network fault. A fanless chassis in a closed cabinet degrades throughput under sustained load without crashing. If the complaint is “slow after a few minutes” rather than “link light goes out”, check temperature before touching the NIC.

Power delivery. These appliances ship with inexpensive external bricks. A marginal 12 V supply under load is a plausible source of intermittent faults across several subsystems at once. If more than one port misbehaves simultaneously, suspect the brick.

PPPoE WAN links. A PPPoE connection that stalls is usually not a link fault at all. FreeBSD’s traditional netgraph PPPoE path pins that traffic to a single queue, and Netgate documents net.isr.dispatch=deferred as the mitigation while noting that the newer if_pppoe kernel backend “is much faster and more efficient and may eliminate the need for additional tuning.” Check which backend the platform is using before treating a PPPoE ceiling as hardware trouble.

When to stop and return the box

If, after a known-good short cable directly between the appliance and a second known-good 2.5GbE device, with ASPM and EEE disabled in the BIOS and flow control off, a specific port still drops link while the other ports on the same board are stable, the reasonable conclusion is a fault in that port. Intel publishes the I226-V as a 2.5G-per-port controller at 1.3 W; a port that cannot hold a link at its rated speed against a clean cable plant is not behaving to specification.

Both platform vendors steer buyers toward Intel controllers for a reason. OPNsense’s hardware documentation states that Intel network interface cards “are reliable, fast and not error-prone” and that “Intel chipset NICs deliver higher throughput at a reduced CPU load.” That is the baseline expectation, and a board that does not meet it is a warranty conversation rather than a tuning exercise.

Order of operations, condensed

1. Confirm igc, not re. Read the negotiated media.
2. Swap the cable; move both ends; test direct between two devices.
3. Force 1000baseT full-duplex and observe.
4. Disable PCIe ASPM, EEE and Wake on LAN in the BIOS.
5. Disable flow control with dev.igc.<id>.fc="0".
6. Check netstat -m; raise kern.ipc.nmbclusters if pressed.
7. Try disabling MSI-X, then MSI, and revert if it does not help.
8. Rule out thermals, the power brick and PPPoE.
9. Conclude a port fault only after all of the above.

Change one thing at a time and give each change enough time to reproduce the fault. A flap that occurs twice a day cannot be cleared by a five-minute test.

Related reading

• Intel N100 firewall build guide: NICs, throughput, power
• N100 firewall setup: OPNsense install and first boot
• N100 vs N305 vs N5105: firewall CPU comparison
• Appliance throughput and power estimator