How DHCP nerfed my NIC

Bought a new 10GbE-capable NIC using the intel 550 chipset. I had purchased one previously but that was for when I had an Eero Pro unit with a 10Gbit port on the unit.

Now that I’ve “downgraded” my network to a 2.5gbit switch, I needed a unit that could do the intermediate steps between gigabit and 10GbE. And yes, I’ve heard much moaning about how the 2.5 and 5GbE are just hobbled steps on the way to full 10GbE – doesn’t matter, it’s what we have and we need it to work at line speed.

Problem is, the DHCP auto-negotiation speeds only allow 10GbE and 1GbE; nothing else in-between will autonegotiate. Turns out this is by design!

 * 2.5G and 5G autonegotiation speeds on X550
 * are disabled by default due to reported
 * interoperability issues with some switches.

How do I fix it?

First, accept that you’re gonna have to let go of DHCP for a minute. If you can’t do without DHCP on your device, that is… for a future post! Yeah! >_>

My unit is an X550-T2 unit, meaning it has two ports: ix0 and ix1, so I have to double-up my commands here. The active port that I selected is ix1 if you want the absolute must-do lines.

First we have to add some overrides to the loader.conf because the ix driver does not allow changes to advertised speeds after boot.

[/boot/loader.conf]

if_ix_load="YES"
hw.ix.0.advertise_speed=0x10
hw.ix.1.advertise_speed=0x10
dev.ix.0.advertise_speed=0x10
dev.ix.1.advertise_speed=0x10

The 0x10 is special because the (as said before) those speeds are disabled by default:

/************************************************************************
 * ixgbe_get_default_advertise - Get default advertised speed settings
 *
 *   Formatted for sysctl usage.
 *   Flags:
 *     0x1 - advertise 100 Mb
 *     0x2 - advertise 1G
 *     0x4 - advertise 10G
 *     0x8 - advertise 10 Mb (yes, Mb)
 *     0x10 - advertise 2.5G (disabled by default)
 *     0x20 - advertise 5G (disabled by default)
 ************************************************************************/

Then we reboot. This should have let DHCP autonegotiate to a 2.5Gbit line speed, but it didn’t. I went through a few more options and finally said “I must tell the machine what to do.”

Thus I changed the following lines in my rc.conf:

[/etc/rc.conf]

#ifconfig_ix1="DHCP"
ifconfig_ix1="inet xxx.xxx.xxx.xxx netmask 0xffffff00 media 2500Base-T"
defaultrouter="xxx.xxx.xxx.xxx"

Now it works! And it’s pretty snappy. Don’t forget to add the defaultrouter line or you won’t be able to reach anything.