Some servers allow you to share an Ethernet interface between the host and the management controller (BMC, IPMI, iLO, iDRAC, etc.)
I always found this strange and have been wondering how it works. Today I learned about NC-SI, the network controller sideband interface, and it's even more cursed than expected.
Fortunately the spec is freely available, so let's have a quick look at it: https://www.dmtf.org/sites/default/files/standards/documents/DSP0222_1.1.1.pdf
[thread]
Essentially they connect the servers NIC to the BMC via a special sideband interface that's based on RMII.
RMII is a physical interface that's commonly used to connect a NIC to a PHY. (You may know SFP modules, they are using the faster SGMII on the back.).
To make this work they have to implement all sorts of queuing, filtering, etc. It's almost like they implement an Ethernet switch in the NIC, but not really. Due to this simple design the host can't reach its own BMC and vice versa.
@vidister How strongly does this rely on the host (a) not doing something silly like flooding the interface with outgoing packets (b) not messing up the NICs configuration in some standard way (c) not doing things that are unlikely to happen by accident but if they happen will mess up the BMCs connection?
