Hyper-V Alert: Beware of the Broadcom Nextreme II

Ever since I first installed my Dell PowerEdge 2900 with Virtual Server I have been experiencing issues with the Broadcom Nextreme II Gigabit adapter also known as the BCM5708C. After my latest disaster at a client with Hyper-V RTM I think it’s time to send out an alert.

In my case I the two hosts I was installing with Hyper-V ended with corrupted network stacks on Virtual Machines and hosts that initially worked flawlessly and all of a sudden caused strange error messages and in the end forced me to completely reinstall from scratch.

Here is my story:

I installed two clean HP ML370G5 Servers with Windows Server 2008 and Hyper-V. At first all installations went smooth. Installed Windows Server 2008 on both hosts. Installed Hyper-V (first RC1 later with RTM). Installed the first VM with Windows Server 2008 and IIS on one host and  another VM on the other host.
 
Both VMs initially work fine. Then I start setting up a new VM on the first host. All of a sudden I notice that the first VM on the same host is no longer responding to the network. I have a look at the eventlog and notice that ASP.Net crashed with an unhandled exception (event ID 1309). I reboot the VM. After the reboot, the system tray shows a disconnected network icon. In the Eventlog I see that the DHCP Client service could not start due to an Access Denied event (ID 1004). Two services fail to start on the VMs:

  • Base Filtering Engine
  • DHCP Client

Now comes the real surprise:

I can no longer start any other VM on the host!

When I start a VM that contains a NIC (virtual or synthetic) I get the following message: "No available MAC address for ‘Network Adapter’". The message occurs before the BIOS startup of the VM.
 
Now I am stuck. I cannot start any VM with a Network Adapter on this host and my first VM with IIS is broken. I tried to take the VHD to another Hyper-V server, but I was never able to recover network connectivity.
When I started creating VMs on the other host I got the same errors there.
 
Both hosts are HP Proliant ML370G5 with dual Broadcom 5708C Nextreme II GigE network adapters connected to a 100 MB half duplex switch without VLANs. I did clean installs of Windows Server 2008 with no extra drivers or software from HP installed. After the first panic, I gave the situation some thought. That’s when I started to seriously suspect the Broadcom adapters of screwing up  my game. Especially because they messed up for me before on  my Dell server with the same NICs in Hyper-V Beta and RC0.

So I did another test:

  1. Clean install of Windows Server 2008 on one of the hosts
  2. Install Hyper-V RTM
  3. Create one virtual network connected to a Broadcom NIC
  4. Create the first VM with a synthetic NIC and boot from the Windows Server 2008 ISO
  5. Create another VM with a synthetic NIC

When I start the second VM I get the message "No available MAC address…". The message is logged in the Hyper-V-Worker Admin log with Event ID 12564, followed by event Ids 12010 and 12030.

In order to proof my theory that my issues were caused by the Broadcom Nextreme II adapters I did the following:

  1. I obtained an extra network adapter with a different chipset and added it to both servers 
  2. Created a Virtual Network connected to the new NIC
  3. Created and started multiple VMs

This went smooth without issues. I can now run Hyper-V and I’m a happy camper!

The Broadcom Nextreme II is built-in in servers from both Dell and HP. I don’t know if other vendors are using the chipset in their systems and what other systems from Dell and HP contain them besides the Dell PowerEdge 2900/2950 and HP Proliant ML370 G5.

Advertisements
This entry was posted in Hyper-V. Bookmark the permalink.

One Response to Hyper-V Alert: Beware of the Broadcom Nextreme II

  1. I had a similar situation, where Hyper-V wouldn’t install properly (the networking component wouldn’t start). There seems to be a “known” issue with Hyper-V and the Broadcom cards.

    Firstly, I downloaded and installed the latest BIOS update from DELL for the NICs. I then downloaded and installed the latest drivers.

    In the advanced NIC settings, I set the following:-

    Ethernet@WireSpeed – Enabled
    Flow Control – Disabled
    Interrupt Moderation – Enabled
    Jumbo Packet – 9000
    Large Send Offload Version 1 (IPV4) – Disabled
    (Same with IPv6)
    Locally Administered Address – Not Present
    Number of RSS Queues – Auto
    Pause On Exhausted Host Ring – Disabled
    Priority & VLAN – Priority & VLAN enabled
    Receive Buffers – 0
    receive Side Scaling – Disabled
    Speed and Duplex – Auto Negotiation
    TCP Connection Offload (IPv4) – Disabled
    (Same with IPv6)
    TCP/UDP Checksum Offload (IPv4) – RX & Tx Enabled
    (Same with IPv6)
    Transmit Buffers – 0
    Virtual Machine Queues – Enabled
    VLAN ID – 0
    VMQ Lookahead Split – Enabled
    Wake On Magic Packet – Enabled
    Wake On Pattern Match – Enabled

    Driver downloaded from Dell Broadcom Corporation 29/9/11 Version 7.0.11.0 – Broadcom BCM5709C NetXtreme II GigE

    Hyper-V installed correctly and all networking components now work.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s