Intel ICH8/ICH9/ICH10 chipsets using e1000e driver (including Lenovo T400/T500): Don't use 2.6.27 RC kernels!!!

kernelsensei brought this to attention in the Linux T400/T500 thread, and I thought it was definitely worth a sticky until it's ironed out:

 

Very much worth sticking, thanks.

 

To clean this up a little bit, the issue is that the current e1000e driver provided with the 2.6.27 Linux kernel release candidates can cause corruption of the network controller EEPROM and non-volatile memory (NVM), on some systems. If you have the e1000e hardware and you have been running it under a 2.6.27 release candidate (RC) kernel with the in kernel driver, you can verify whether or not you have the corrupted EEPROM/NVM by scanning your dmesg output. The easiest way to do this is to issue the command from a terminal;

Code:

dmesg |less

Next hit the / key and type in e1000e, and the less utility will scroll forward to the first instance of the e1000e driver message output. If you see the message output there similar to the following;

Code:

e1000e: Intel(R) PRO/1000 Network Driver - 0.4.1.7
e1000e: Copyright (c) 1999-2008 Intel Corporation.
e1000e 0000:00:19.0: PCI INT A -> GSI 20 (level, low) -> IRQ 20
e1000e 0000:00:19.0: setting latency timer to 64
0000:00:19.0: 0000:00:19.0: The NVM Checksum Is Not Valid
e1000e 0000:00:19.0: PCI INT A disabled
e1000e: probe of 0000:00:19.0 failed with error -5

(the key message being the "NVM Checksum is Not Valid"), then you have the problem. Some have noted the problem occurs when rebooting, after a system hang and some when rebooting after suspending the system. Others have noted that when they initially had EEPROM/NVM corruption problems with 2.6.27RC, by downgrading their kernel, it somehow corrected the problem. Other noted something similar by booting into Windows. Lastly, some folks have noted that updating the BIOS may fix this memory corruption problem. You can read their experiences in the Ubuntu bug tracking page below.

The bad news is that some users who have experienced this problem, have had to replace hardware to restore the network port. Of course with laptops, or other systems which have the network chip built into the motherboard, if the EEPROM/NVM cannot be restored, then the entire system board needs to be replaced. The consensus seems to be that if you develop a problem, the use of Intel's IABUTIL.EXE is strongly discouraged as a method of trying to correct the problem. Phoronix has a brief post about this issue here.

So beginning from the top we have;

1) The official kernel bug tracking page

2) An Ubuntu mail list announcement, and the official Ubuntu bug tracking page

3) An openSUSE news post, and a related mail thread

4) A Redhat bug tracking page

5) A Gentoo bug tracking page

6) And the previously mentioned Mandriva blog post describing this problem

The Linux developers in the -mm patched kernel tree, have discussed this issue in this thread. There was even a suggestion that the bug was fixed with this patch. However an Intel engineer posting last in the thread here, suggests that the issue requires further investigation. (At least the right engineers are working on a fix.) The good news is that three patches were submitted to the kernel development team by the Intel engineers recently, and if they solve the problem, we can expect to see them in the final release of kernel 2.6.27. For the curious they are here; 1, 2, and 3.

For now, most distros are masking the current 2.6.27RC e1000e driver going forward, so it's not installed by default, until this issue can be addressed. However, you need to check to make sure you are not using the problem e1000e driver.

So the first thing you should do is make sure whether or not your system uses the e1000e driver. It does not have to be a module loaded in your system, it can also be compiled as a driver into your kernel, so just checking lsmod might not be enough. Check out this driver page, and see if your system contains any of the mentioned network hardware. In an ironic twist you can read from this announcement post at lwn.net, that;

(This has already led to some previously interesting snafus regarding this changeover, as described in this announcement, including breaking Linus Torvald's machine.)

Finally, if you are running the appropriate Intel ethernet hardware, including ICH8 and ICH9 onboard LAN, server 5000 platform onboard LAN (es2), and 82571/2/3 chipset based adapters and variants, then you have two choices. The first as kernelsensei has pointed out, you can use the 2.6.27RC kernels, provided you do not use the e1000e driver. (This would mean that you would not have a working wired ethernet port too.) The previous instructions for removing the driver module, only work if your driver is configured as a module. My recommendation would be for you to recompile your kernel without the e1000e driver support, module or built in. Of course you should be familiar with compiling kernels on your distro in order to do this. BTW, the driver in question using the make menuconfig option, is at Device Drivers ---> Network device support ---> Ethernet (1000 Mbit) ---> Intel(R) PRO/1000 PCI-Express Gigabit Ethernet support. Neither * or m, and the driver is removed from your kernel, once compiled and installed. You can also check the status of your kernel configuration by simply;

Code:

sudo grep CONFIG_E1000E /usr/src/linux/.config

If you see the kernel option as y or m, then it's enabled as built in, (y), or as a module (m).

An easier solution might be to make sure you are not running any of the 2.6.27 release candidate kernels. Ubuntu introduces it with their Intrepid Ibex release, and other distros make the RC versions of 2.6.27 available in other ways. If you have the applicable hardware and you are running 2.6.27RC kernel, you should downgrade your system until the 2.6.27 kernel is finalized.

Finally, if you have experienced this problem. Don't panic. Try downgrading the kernel you are running, booting into Windows, or updating the BIOS and see if your wired ethernet port is restored. Others of you who are using the 2.6.27RC kernels, should also not panic if you find your ethernet port NOT working. As I mentioned the Linux distros are masking the driver, so you probably don't have the driver loaded anymore. Downgrade your kernel and see if your wired ethernet port is working correctly.

BTW, freiric posted this issue to the forums first, in the Linux on Latitude E6400 thread in this post. Kudos to both freiric and kernelsensei, for providing the heads up. The only thing I would suggest is changing the title of the thread to something like;

Good Luck..

 

How would I know if I have the Intel ICH8/ICH9/ICH10 chipset or the e1000e driver?

 

Software like CPU-Z on Windows or the built-in lspci command in Linux can tell you. But you needn't worry anymore; as of 2.6.27-4, the e1000e driver was blacklisted to prevent damage, and then in 2.6.27-6 the bug was corrected and the driver re-enabled. The Ubuntu Intrepid Beta and all subsequent daily builds are safe, then; I can't speak for other distributions, but I can imagine that something this critical was reacted to with equal swiftness by most distribution vendors.

 

I use the e1000 driver. Is this an issue for me? Could someone point me to the place that explains the difference between the e1000 and the e1000e driver? Thanks.

 

Aye. Thanks for the heads up.

 

I've been using the e1000e drivers with Intrepid on a W500 with no problems so far. I thought it's been fixed.

 

It has been fixed. I'm henceforth locking this thread and un-sticking it