FreeBSD problems. Getting desperate
Here's the same question I asked on matrix. Perhaps someone here is familiar with FreeBSD networking.
I have spent close to 2 weeks struggling with my bridge and vlan configuration when used with bhyve. I really cannot figure this out. Long story short: I have an interface (igc0), and then a virtual interface which exposes vlan 100 (interface name is ext). I then use vm-bhyve to create a switch called dmz that is mapped to ext. When I start the VM and ping my router on the vlan, I can see the ARP request reaching the router, and it replies, however, if I do a tcpdump on the ext interface, I don't see the ARP reply.
Adding an ip address to ext allows me to ping the router from the host, so the ext interface does work.
@loke The bridge is designed to grab all traffic including ARP it seems reading the comments in this bug report:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240106https://genneko.github.io/playing-with-bsd/networking/freebsd-vlan/#do-not-bridge-the-parent-interface-of-the-vlans
Also found some interesting general info here: https://genneko.github.io/playing-with-bsd/networking/freebsd-vlan/#do-not-bridge-the-parent-interface-of-the-vlans
@modrobert thanks, this seems aligned with what I see.
The question then is, how can I solve this? I need to have some VM's that can see the vlans, but some that needs access to the main network.
@loke It's hard without some kind of drawing of your network, but after reading the the info/comments I linked, it seems like it can be solved by the way it is configured, by not bridging the parent interface of the vlans, and someone wrote that it matters if you give the vlans an IP address or not. Also this:
"NOTE: To route packets between the bridges (VLANs), make sure to assign IP addresses to the bridges as described in the previous section, not their member interfaces such as em0.10."
@modrobert Thank you for the information you provided. It was indeed a problem where you cannot create a bridge on an interface and also another bridge on a VLAN on the same interface.
I worked around the problem by creating a new subnet that is routed to my main network, and then created a new VLAN that exposes this network. That way VM's that should live on the main network can use this VLAN and no VM's will then have to use the primary interface.
It's ugly, but at least it works. Surely this must be a bug in FreeBSD.
@modrobert I've outlined my configuration in this thread. Including output of ifconfig.
https://forums.freebsd.org/threads/vm-bhyve-and-vlan.86712/
It's really quite a simple configuration, and I'm surprised it's so difficult to get it to work correctly