NOTICE: The Processors Wiki will End-of-Life in December of 2020. It is recommended to download any files or other content you may need that are hosted on processors.wiki.ti.com. The site is now set to read only.

Enabling Multiple Ethernet Ports on TI81xx Devices for Substation Automation Applications

From Texas Instruments Wiki
Jump to: navigation, search

Introduction[edit]

The revamping of power grids across the world requires smart devices with the capability to monitor multiple networks. These devices are used within electrical substations where the ability to handle multiple communication links between equipment and the central office is a key requirement. These systems require application processors with multiple Ethernet connections for fault-tolerance purposes.

The DM816x/C6A816x class of application processors include two Gigabit Ethernet controllers. However, substation automation applications require more than two Ethernet ports.

This article describes how a a PCIe-to-PCI bridge can be used to expand the number of Ethernet ports on the C6A816x EVM. The two Gigabit Ethernet ports on the C6A816x device operate in parallel with three PCI 10/100 network interface cards connected through the bridge.

Performance benchmarks are also presented to give an idea of the bandwidth achieved through each Ethernet port as well as the loading on the C6A816x ARM core.

NOTE: The hardware and sotware setup instructions on this wiki are specific to DM816x/C6A816x devices.  However, they can be extended to DM814x/C6A814x devices since they share a common hardware architecture and software base.  For the purposes of this wiki, the term "TI81xx" encompasses both DM816x/C6A816x and DM814x/C6A814x devices.

MultiEthernet HW Setup.PNG
MultiEthernet HW.PNG


Prerequisites[edit]

This guide assumes that the development environment is setup, and that UBL and U-boot has been flashed on the C6A816x EVM. If not, please refer to the documentation that is part of the PSP release for more information.

Hardware[edit]

Development Environment[edit]

  • Connect to the serial port, power on the board, and verify that the UBL and U-boot prints from serial port terminal
  • Verify that the C6A816x EZSDK is installed, and that the kernel source code is version 2.6.37 or higher. If it is not, go to the EZSDK software page and download the latest software.
  • Verify that the TFTP server is configured in your host Linux machine


Configure Kernel options and modify drivers[edit]

  • Setup environment variables
$ source <EZSDK directory>/linux-devkit/environment-setup
$ cd <EZSDK directory>/psp/linux-#.#.#.#
  • Setup default configuration by executing the following command
$ make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- ti8168_evm_defconfig
  • Modify the kernel by entering into the interactive configuration with the following command
$ make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- menuconfig
  • Configure kernel to enable Realtek 8139 driver
1. Highlight "Device Drivers" and hit Enter
2. Highlight "Network device support" and hit Enter
3. Highlight "Ethernet (10 or 100Mbit)", hit "Y", and hit Enter
4. Highlight "EISA, VLB, PCI, and on board controllers" and hit "Y"
5. Highlight "RealTek RTL-8129/8130/8139 PCI Fast Ethernet Adapter support" and hit "M"
6. Highlight "Use PIO instead of MMIO" and hit "N"
7. Save configuration
  • Since the PCIe peripheral maximum remote read request size is 256 bytes, the following will modify the PCI driver to restrict XIO2001 read request size to 256 bytes. By default, the read request size of XIO2001 is 512 bytes.
$ cd <EZSDK directory>/psp/linux-#.#.#.#
$ sudo gedit drivers/pci/quirks.c
At the bottom of the file, add the following:
static void __devinit quirk_ti_xio2001_read_req_size(struct pci_dev *dev)
{ WARN_ON(pcie_set_readrq(dev, 256) < 0); }
DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_TI, PCI_DEVICE_ID_TI_XIO2001, quirk_ti_xio2001_read_req_size);
Save quirks.c
$ sudo gedit include/linux/pci_ids.h
Add the following under line 786:
#define PCI_DEVICE_ID_TI_XIO2001	0x8240
Save pci_ids.h
  • Recompile kernel
$ make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- uImage
  • Recompile modules
$ make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- modules
  • Copy compiled binaries to the TFTP folder
$ cp arch/arm/boot/uImage /tftpboot
$ cp drivers/net/*.ko /tftpboot


Load Kernel and Drivers[edit]

  • Flash the kernel to the desired boot mode by following the instructions found in the U-Boot User Guide, which is available in the PSP release.
  • Transfer the driver modules to the running C6A816x file system. You can use TFTP to get the files on the C6A816x EVM as follows. Note that in the example, we assume TFTP server 172.24.190.43 is running with the concerned files copied into the server's TFTP folder.
TI8168_EVM# cd /home/root
TI8168_EVM# tftp -g -r mii.ko 172.24.190.43
TI8168_EVM# tftp -g -r 8139too.ko 172.24.190.43
  • Note that the above commands may vary depending upon the filesystem and the tftp utility being used. Also, you can use different means of transferring the above files into the C6A816x filesystem as convenient. Assuming the files are transferred successfully, and you have navigated to the directory containing the above files, use the following commands to load the network drivers.
TI8168_EVM# cd /home/root
TI8168_EVM# ls              // Verify that you see 8139too.ko and mii.ko
  • Execute the following commands:
TI8168_EVM# insmod mii.ko
TI8168_EVM# insmod 8139too.ko
  • If everything went well, you should be able to see all Ethernet ports now.
TI8168_EVM# ifconfig -a



Performance and Benchmarks[edit]

The following benchmarks were collected on a Multiple Ethernet setup that has three 10/100 NIC cards and two Gigabit Ethernet ports.  Each Ethernet port directly connects to a Host PC with a crossover cable.  In the test environment, a total of 5 Host PCs were used. 


Benchmark[edit]

Network
Configuration
CPU Loading
(ARM Only)
Throughput
1 10/100 NIC(1)
ARM Load = 18%

NIC_1 = 94.9 Mbits/s

2 10/100 NIC(1) ARM Load = 32%

NIC_1 = 44.8 Mbits/s
NIC_2 = 57.6 Mbits/s

3 10/100 NIC(1) ARM Load = 51%

NIC_1 = 41.9 Mbits/s
NIC_2 = 38.6 Mbits/s
NIC_3 = 38.8 Mbits/s

1 GigE
ARM Load = 95%

GigE_1 = 591 Mbits/sec

2 GigE
ARM Load = 99%

GigE_1 = 273 Mbits/sec
GigE_2 = 275 Mbits/sec

1 GigE
2 10/100 NIC
ARM Load = 99%

GigE_1 = 196 Mbits/s
NIC_1 = 32.6 Mbits/s
NIC_2 = 32.5 Mbits/s
NIC_3 = 34.7 Mbits/s

2 GigE
3 10/100 NIC
ARM Load = 99%

GigE_1 = 117 Mbits/s
GigE_2 = 121 Mbits/s
NIC_1 = 29.1 Mbits/s
NIC_2 = 30.4 Mbits/s
NIC_3 = 28.8 Mbits/s

(1) In cases where the ARM load is less than 99-100%, the window size can be tweaked to get higher performance.  


Notes

  • All ARM load numbers are approximations/averages of numbers read using top command on Linux.
  • The performance numbers were captured using the iperf v2.0.4 tool in unidirectional mode. Usage details are mentioned below:
  • On the C6A816x DUT iperf is invoked in server mode with a window size of 212K.  NOTE: Linux default window size of 212K.
  • On PC Hosts invoked iperf in the client mode with a window size of 128K (unless otherwise noted).  
  • The transfers are measured over a duration of 60Secs
  • Cross cable is used to measure performance
  • C6A816x are running at an ARM speed of 987MHz (unless otherwise noted) and DDR2 speed of 398MHz.
  • For all testing, root filesystem mounted from SD card.
  • Gigabit ports speed is set to 1000Mbps; NIC speed is set to 100Mbps.
E2e.jpg {{
  1. switchcategory:MultiCore=
  • For technical support on MultiCore devices, please post your questions in the C6000 MultiCore Forum
  • For questions related to the BIOS MultiCore SDK (MCSDK), please use the BIOS Forum

Please post only comments related to the article Running Multiple Ethernets on C6A816x here.

Keystone=
  • For technical support on MultiCore devices, please post your questions in the C6000 MultiCore Forum
  • For questions related to the BIOS MultiCore SDK (MCSDK), please use the BIOS Forum

Please post only comments related to the article Running Multiple Ethernets on C6A816x here.

C2000=For technical support on the C2000 please post your questions on The C2000 Forum. Please post only comments about the article Running Multiple Ethernets on C6A816x here. DaVinci=For technical support on DaVincoplease post your questions on The DaVinci Forum. Please post only comments about the article Running Multiple Ethernets on C6A816x here. MSP430=For technical support on MSP430 please post your questions on The MSP430 Forum. Please post only comments about the article Running Multiple Ethernets on C6A816x here. OMAP35x=For technical support on OMAP please post your questions on The OMAP Forum. Please post only comments about the article Running Multiple Ethernets on C6A816x here. OMAPL1=For technical support on OMAP please post your questions on The OMAP Forum. Please post only comments about the article Running Multiple Ethernets on C6A816x here. MAVRK=For technical support on MAVRK please post your questions on The MAVRK Toolbox Forum. Please post only comments about the article Running Multiple Ethernets on C6A816x here. For technical support please post your questions at http://e2e.ti.com. Please post only comments about the article Running Multiple Ethernets on C6A816x here.

}}

Hyperlink blue.png Links

Amplifiers & Linear
Audio
Broadband RF/IF & Digital Radio
Clocks & Timers
Data Converters

DLP & MEMS
High-Reliability
Interface
Logic
Power Management

Processors

Switches & Multiplexers
Temperature Sensors & Control ICs
Wireless Connectivity