Release notes Xframe 10GbE Server / Storage Adapters Neterion source code release for production Linux Xframe device drivers Release Version 2.0.14.5152, 03/24/06 Contents ======= - 1. Requirements - 2. Installation - 3. What's new in this release? - 4. Available Downloads (User Guides, Additional Features) - 5. Known Issues - 6. Disclaimer 1. Requirements: -------------------------------- Linux Redhat or SuSE, 2.4.21 and 2.6.x (Other distributions may not be extensively tested) Xframe I is PCI-X-1.0 compliant and will operate in any PCI slot. Xframe II is PCI-X-2.0 compliant and will operate in any 3.3v PCI slot. Using PCI-X 266 is highly recommended. At least 512MB memory is advised. However, 2Gb is recommended in order to achieve optimal performance. 2. Installation: -------------------------------- Build Instructions For 2.6 kernel make sure you have installed the Linux kernel source rpm. To verify whether you have kernel source directories inside /usr/src do "find /usr/src|grep ". Use "uname -r" to get kernel version. For 2.4 kernels make sure that there exists a symbolic link file by name "/usr/src/linux-2.4" pointing to correct kernel source directory. Run "ls -l" and verify whether "/usr/src/linux-2.4" is pointing. If it is pointing to wrong directory please delete the link file and create new file with command "ln -s /usr/src/linux-2.4.xxxx /usr/src/linux-2.4" Red Hat users, please find references to the latest driver updates in http://rhn.redhat.com/errata/RHSA-2006-0132.html Go to the directory where Neterion drivers reside and build the driver as follows # make Or you can choose to make install the driver to automatically copy it to lib/modules/2.x.x.x/driver/net directory. # make install Driver Load/Unload Load On 2.4 kernel # insmod s2io.o On 2.6 kernel # insmod s2io.ko Configure IP address # ifconfig e.g. # ifconfig eth1 20.1.1.15 can be found by executing "ifconfig -a". S2io cards typically will have HWaddr starting with "00:0C:FC" or "00:11:25" Unload # ifconfig down # rmmod s2io Modify MTU Down the interface # ifconfig down Modify MTU # ifconfig mtu Bring up the interface # ifconfig up Enable Features Two-buffer mode When installing the driver for 2buff mode please use the following command: insmod s2io.ko rx_ring_mode=2 NAPI support NAPI is supported only in 2.6 kernel. To enable NAPI 1. Go to directory where kernel source tree installed usually will be inside "/usr/src/linux-2.6.xxx" or "/usr/src/kernel/2.6.xxxx" 2. Do "make menuconfig" and enable "Use Rx Polling (NAPI) (EXPERIMENTAL)". This option will be displayed under following menu option. "Device Drivers ---> Networking support ---> Ethernet (10000 Mbit)" 3. Build the driver as per build instruction section. 3. What's new in this release?: ------------------------------------------------------ Release 2.0.14.5152, 03/24/06 Features: * Added timestamp support to LRO Bug Fixes: * Remove RoHs specific changes as hardware fix is available * Updated Readme_lx.txt with new util command options * Modified order of writes to special registers (mdio_control, dtx_control, etc) * Merge driver tune, README.tune with linux driver release and Readme_lx.txt * Remove legacy code for old transponder * Increment macro NETLINK_S2IO since Netlink id 17 taken by Device Mapper NETLINK_DM in Suse Beta 7 * Fixed link-up issue for finisar transponders * Simplified debugging tools Previous Release 2.0.13.4939, 03/07/06 Features: * CX4 adapters support * UDP fragmentation offload for Xframe II. * Large receive offload(LRO) * 3-buffer receive mode * Receive traffic steering(RTH and SPDM) * Receive side performance improvements (particularly for MTU=1500) * Support for PCI-E Xframe adapter Bug fixes: * Fixed indicate_max_pkts failure at boundary conditions. * Fixed MSI-X binding problem in the presence of multiple Xframe II adapters. * Added README.tune and files required for tuning/debugging driver. * Fix for gcc4.0 32 bit platform compilation errors. * Used repeated subtraction instead of 64-bit divide to load on 2.4 kernel. * Fixed bug in multicast handling code when setting multiple multicast addresses. * Fixed bus speed configuration for PCI-E Xframe adapter * Fixed nttcp losing connection with 2 buffer mode enabled * Added loadable parameter to configure max pkts to be aggregated * Fixed rmmod hang issue due to interrupt (ctrl-C) * Driver to check parameters' boundry conditions before loading * Corrected 2.6.14.4 kernel compilation error * insmod from shell script * Handle link interupt as per user guide for Xframe II * Fixed warning message while loading module in 2.4 kernel * Removed forced disabling of ERO(Enable relaxed ordering). * Corrected Crash/hang with multi-ring * Corrected ethtool online link test failures * Added a conditional check in case of wait during interrupt context * Fixed rmmod crash on 32 bit system when AMD_FIX enabled * Added graceful shutdown in stop_nic() * Sleep delay fix * Replaced readq with readl to flush posted writes in s2io_poll * Organized messages displayed on driver load * Correctly identify rx buffer level * Fixed compilation warnings on IA64 system * fixed DMA map and unmap done with different sizes in 1-bug mode * Added support to display xpak alarm and warnings in ethtool -S * Changed to use common code to free txdl * Resolved compilation errors with LRO enabled * Rolled back the per-packet Rx replenish and bumped up level of "Ring full" msg * Continue with initalization if min required MSI-X vectors met. * Fixed offline diagnostics (ethtool-t) failure * Changes to validate module parameters for correct value. 4. Available Downloads: ------------------------------------------- For latest available Neterion drivers or further support please contact your network adapter provider or check the Neterion support downloads site at http://www.neterion.com/support/xframe_customers.html . 5. Known Issues: ---------------------------------- The TSO is disabled by default in SuSE9. It may be manually enabled using ethtool command. e.g. ethtool -K ethX tso on By default, the mapping from MSI/MSI-X to hyper transport interrupts is disabled on AMD 8132 bridge. This is indicated by a value of 0xA8000008 at PCI config space offset 0xF4. Changing this to 0xA8010008 will enable this translation (iff qword at 0xF8 is 0x0000FEE00000). This can be done with the command: #setpci -d 1022:7458 f6=01 Common Problems: Load Failures: The following operations will fail due to a known bug in the kernel: 1) Multiple load/unload of driver with MSI-X. 2) Driver load in MSI-X mode followed by load in MSI mode. 3) Driver load in MSI mode followed by load in MSI-X mode. Symbolic link: Before debugging any compilation related problems, ensure that the following symbolic link/directory points to the correct kernel source tree. For 2.4, /usr/src/linux-2.4 For 2.6, /usr/src/linux-2.6 Version Mismatch: One of the common problems encountered on trying to load the driver is related to a version mismatch between current kernel and the kernel for which the driver was built. Typical error message is as follows: Using /lib/modules/2.x.y-SMP/kernel/drivers/net/s2io.o /lib/modules/2.x.y-SMP/kernel/drivers/net/s2io.o: kernel-module version mismatch /lib/modules/2.x.y-SMP/kernel/drivers/net/s2io.o was compiled for kernel version 2.x.y while this kernel is version 2.x.y-SMP. To fix this problem, ensure the kernel is enabled for SMP by following these steps. a. Go to directory where kernel sources are located. (eg. /usr/src/linux-2.4.19.SuSE) b. Execute "make menuconfig". This shows up a menu. Under this, select "Processor type and features". Ensure that "Symmetri multi-processing suppor" is enabled. Save this configuration and exit. c. This will compile the kernel with SMP support. General suggestions: 1. Set MTU to 9000 2. Run sysctl script(sysctl -p sysctl_xframe9000.conf) 3. If CPU bottleneck(usage close to 100%) is being hit, you can play around with utilization parameters by using driver_tune script.This may help bring down the CPU utilization and improve throughput. Suggestions to improve Transmit performance: 1. Experiment with the PCI max split transactions and MMRBC value in pcix_command (use setpci command). Max split transaction a value of 2 has been found to be optimal for Opterons and 3 for Itanium. It could be different for your h/w. set MMRBC set value to be 4k** for Example you can set #setpci -d 17d5:* 62=1d For Opteron platform #setpci -d 17d5:* 62=3d For Itanium platform 3. Ensure Transmit Checksum offload is ON, use "ethtool -K" to set/verify this parameter. 4. If you are using 2.6 kernel(check using command "uname -r"), additional performance benefit can be obtained by turning on TSO (TCP segmentation offload). Use "ethtool -k" to set this parameter. ie "ethtool -K ethX tso on" Suggestions to improve Receive performance: 1. Ensure PCI latency timer is set to 248. #setpci -d 17d5:* 0d=f8 2. Try using 2-buffer mode. This gives good performance boost on Receive side on certain platforms(eg. SGI Altix). 3. Ensure Receive Checksum offload is ON. Use "ethtool -K ethX" command to set/verify the rx checksum offload option. 4. If MTU is set to 1500(standard frame size), receive performance can be improved by setting the TCP window size. Run sysctl script(sysctl -p sysctl_xframe1500.conf) to set this parameter. 5. Enable NAPI feature to bring down the CPU utilization. ** For AMD opteron platforms, MMRBC=1K and Max_splits=2 are recommended as safe parameters. For more information, please review the AMD8131 errata at http://www.amd.com/us-en/Processors/TechnicalResources/0,,30_182_739_9004,00.html 6. Disclaimer: -------------------------------- The information in this manual is furnished for informational use only, is subject to change without notice, and should not be construed as a commitment by Neterion Corporation. Neterion Corporation assumes no responsibility or liability for any errors or inaccuracies that may appear in this document or any software that may be provided in association with this document. Except as permitted by such license, no part of this document may be reproduced, stored in a retrieval system, or transmitted in any form or by any means without the express written consent of Neterion Corporation. -------------------------------------------------------------- * Other product and corporate names may be trademarks of other companies and are used only for explanation and to the owner's benefit, without intent to infringe.