标题: kernel-utils套件学习之三-irqbalance(IRQBalance真的有效么) (查看:552 回复:0)
liquan8909
学徒网管



UID 740509
精华 0
积分 1
帖子 96
MST币 0 点    
BST币 1 点    赚取
阅读权限 60
注册 2008-6-4
状态 离线
 
[ 使用道具 ]  
发表于 2008-6-10 23:33  [ 资料 ]  [ 博客 ]  [ 短消息 ]  [ 加为好友
爱琴思邮件系统iGENUS V5.0
kernel-utils套件学习之三-irqbalance(IRQBalance真的有效么)
本帖发表在我是网管论坛,帖子地址:http://bbs.54master.com/255110,1,1
看到名字:IRQ这是个linux newibe都知道的东西,倒退几年关于IRQ的问题一直是困扰newibe们的难题,至少现在仍是。
什么是IRQ?[size=-1]Interrupt Request的简写,翻译中文为“中断请求”。通过中断,外设可以取得CPU的处理时间。
linux会给每个设备分配一个中断号,常见的有
Device          I/O address       IRQ           DMA
ttyS0 (COM1)      3f8             4             NA[a]
ttyS1 (COM2)      2f8             3             NA
ttyS2 (COM3)      3e8             4             NA
ttyS3 (COM4)      2e8             3             NA
lp0 (LPT1)        378-37f         7             NA
lp1 (LPT2)        278-27f         5             NA
fd0, fd1          3f0-3f7         6             2
fd2, fd3          370-377         10            3

当然还有很多,分配固定,那么就会有冲突。我们常见的问题是中断号冲突导致设备不可用,网卡经常遇到此种情形。
大家可以看下我本本的输出:
lee@lee-laptop:/proc/irq$ lspci -bv
00:00.0 Host bridge: Intel Corporation 82852/82855 GM/GME/PM/GMV Processor to I/O Controller (rev 02)
        Subsystem: Hewlett-Packard Company Unknown device 3080
        Flags: bus master, fast devsel, latency 0
        Memory at <unassigned> (32-bit, prefetchable)
        Capabilities: <access denied>

00:00.1 System peripheral: Intel Corporation 82852/82855 GM/GME/PM/GMV Processor to I/O Controller (rev 02)
        Subsystem: Hewlett-Packard Company Unknown device 3080
        Flags: bus master, fast devsel, latency 0

00:00.3 System peripheral: Intel Corporation 82852/82855 GM/GME/PM/GMV Processor to I/O Controller (rev 02)
        Subsystem: Hewlett-Packard Company Unknown device 3080
        Flags: bus master, fast devsel, latency 0

00:02.0 VGA compatible controller: Intel Corporation 82852/855GM Integrated Graphics Device (rev 02) (prog-if 00 [VGA])
        Subsystem: Hewlett-Packard Company Unknown device 3080
        Flags: bus master, fast devsel, latency 0, IRQ 10
        Memory at e8000000 (32-bit, prefetchable)
        Memory at e0000000 (32-bit, non-prefetchable)
        I/O ports at 1800
        Capabilities: <access denied>

00:02.1 Display controller: Intel Corporation 82852/855GM Integrated Graphics Device (rev 02)
        Subsystem: Hewlett-Packard Company Unknown device 3080
        Flags: bus master, fast devsel, latency 0
        Memory at f0000000 (32-bit, prefetchable)
        Memory at e0080000 (32-bit, non-prefetchable)
        Capabilities: <access denied>

00:1d.0 USB Controller: Intel Corporation 82801DB/DBL/DBM (ICH4/ICH4-L/ICH4-M) USB UHCI Controller #1 (rev 03) (prog-if 00 [UHCI])
        Subsystem: Hewlett-Packard Company Unknown device 3080
        Flags: bus master, medium devsel, latency 0, IRQ 10
        I/O ports at 1820

00:1d.1 USB Controller: Intel Corporation 82801DB/DBL/DBM (ICH4/ICH4-L/ICH4-M) USB UHCI Controller #2 (rev 03) (prog-if 00 [UHCI])
        Subsystem: Hewlett-Packard Company Unknown device 3080
        Flags: bus master, medium devsel, latency 0, IRQ 3
        I/O ports at 1840

00:1d.2 USB Controller: Intel Corporation 82801DB/DBL/DBM (ICH4/ICH4-L/ICH4-M) USB UHCI Controller #3 (rev 03) (prog-if 00 [UHCI])
        Subsystem: Hewlett-Packard Company Unknown device 3080
        Flags: bus master, medium devsel, latency 0, IRQ 4
        I/O ports at 1860

00:1d.7 USB Controller: Intel Corporation 82801DB/DBM (ICH4/ICH4-M) USB2 EHCI Controller (rev 03) (prog-if 20 [EHCI])
        Subsystem: Hewlett-Packard Company Unknown device 3080
        Flags: bus master, medium devsel, latency 0, IRQ 3
        Memory at e0100000 (32-bit, non-prefetchable)
        Capabilities: <access denied>

00:1e.0 PCI bridge: Intel Corporation 82801 Mobile PCI Bridge (rev 83) (prog-if 00 [Normal decode])
        Flags: bus master, fast devsel, latency 0
        Bus: primary=00, secondary=02, subordinate=06, sec-latency=64
        I/O behind bridge: 00003000-00003fff
        Memory behind bridge: e0200000-e02fffff
        Prefetchable memory behind bridge: 30000000-33ffffff

00:1f.0 ISA bridge: Intel Corporation 82801DBM (ICH4-M) LPC Interface Bridge (rev 03)
        Flags: bus master, medium devsel, latency 0

00:1f.1 IDE interface: Intel Corporation 82801DBM (ICH4-M) IDE Controller (rev 03) (prog-if 8a [Master SecP PriP])
        Subsystem: Hewlett-Packard Company Unknown device 3080
        Flags: bus master, medium devsel, latency 0, IRQ 255
        I/O ports at <unassigned>
        I/O ports at <unassigned>
        I/O ports at <unassigned>
        I/O ports at <unassigned>
        I/O ports at 1810
        Memory at 34000000 (32-bit, non-prefetchable)

00:1f.3 SMBus: Intel Corporation 82801DB/DBL/DBM (ICH4/ICH4-L/ICH4-M) SMBus Controller (rev 03)
        Subsystem: Hewlett-Packard Company Unknown device 3080
        Flags: medium devsel, IRQ 5
        I/O ports at 1880

00:1f.5 Multimedia audio controller: Intel Corporation 82801DB/DBL/DBM (ICH4/ICH4-L/ICH4-M) AC'97 Audio Controller (rev 03)
        Subsystem: Hewlett-Packard Company Unknown device 3080
        Flags: bus master, medium devsel, latency 0, IRQ 5
        I/O ports at 1c00
        I/O ports at 18c0
        Memory at e0100c00 (32-bit, non-prefetchable)
        Memory at e0100800 (32-bit, non-prefetchable)
        Capabilities: <access denied>

00:1f.6 Modem: Intel Corporation 82801DB/DBL/DBM (ICH4/ICH4-L/ICH4-M) AC'97 Modem Controller (rev 03) (prog-if 00 [Generic])
        Subsystem: Hewlett-Packard Company Unknown device 3080
        Flags: medium devsel, IRQ 5
        I/O ports at 2400
        I/O ports at 2000
        Capabilities: <access denied>

02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ (rev 10)
        Subsystem: Hewlett-Packard Company Unknown device 3080
        Flags: bus master, medium devsel, latency 64, IRQ 10
        I/O ports at 3000
        Memory at e0207800 (32-bit, non-prefetchable)
        Capabilities: <access denied>

02:06.0 Network controller: Intel Corporation PRO/Wireless 2200BG Network Connection (rev 05)
        Subsystem: Hewlett-Packard Company Unknown device 12f6
        Flags: bus master, medium devsel, latency 64, IRQ 4
        Memory at e0206000 (32-bit, non-prefetchable)
        Capabilities: <access denied>

02:09.0 CardBus bridge: Texas Instruments PCIxx21/x515 Cardbus Controller
        Subsystem: Hewlett-Packard Company Unknown device 3080
        Flags: bus master, medium devsel, latency 168, IRQ 255
        Memory at e0209000 (32-bit, non-prefetchable)
        Bus: primary=02, secondary=03, subordinate=06, sec-latency=176
        Memory window 0: 30000000-33fff000 (prefetchable)
        Memory window 1: 38000000-3bfff000
        I/O window 0: 00003400-000034ff
        I/O window 1: 00003800-000038ff
        16-bit legacy interface ports at 0001

02:09.2 FireWire (IEEE 1394): Texas Instruments OHCI Compliant IEEE 1394 Host Controller (prog-if 10 [OHCI])
        Subsystem: Hewlett-Packard Company Unknown device 3080
        Flags: bus master, medium devsel, latency 64, IRQ 4
        Memory at e0207000 (32-bit, non-prefetchable)
        Memory at e0200000 (32-bit, non-prefetchable)
        Capabilities: <access denied>

02:09.3 Mass storage controller: Texas Instruments PCIxx21 Integrated FlashMedia Controller
        Subsystem: Hewlett-Packard Company Unknown device 3080
        Flags: bus master, medium devsel, latency 64, IRQ 11
        Memory at e0204000 (32-bit, non-prefetchable)
        Capabilities: <access denied>

02:09.4 Generic system peripheral [0805]: Texas Instruments PCI6411/6421/6611/6621/7411/7421/7611/7621 Secure Digital Controller
        Subsystem: Hewlett-Packard Company Unknown device 3080
        Flags: bus master, medium devsel, latency 64, IRQ 11
        Memory at e0208400 (32-bit, non-prefetchable)
        Memory at e0208000 (32-bit, non-prefetchable)
        Memory at e0207c00 (32-bit, non-prefetchable)
        Capabilities: <access denied>
红色的字体显示的是IRQ同为10的输出。
当然也可以直接在/proc中查到:
lee@lee-laptop:~$ sudo cat /proc/interrupts
[sudo] password for lee:
           CPU0      
  0:    9269905   IO-APIC-edge      timer
  1:       8989   IO-APIC-edge      i8042
  8:          1   IO-APIC-edge      rtc
  9:      19537   IO-APIC-fasteoi   acpi
12:     121305   IO-APIC-edge      i8042
14:     296468   IO-APIC-edge      libata
15:     118272   IO-APIC-edge      libata
16:          1   IO-APIC-fasteoi   sdhci:slot0, sdhci:slot1, sdhci:slot2, yenta, tifm_7xx1
17:     502348   IO-APIC-fasteoi   Intel 82801DB-ICH4
18:    1491470   IO-APIC-fasteoi   uhci_hcd:usb1, eth0, i915@pci:0000:00:02.0
19:     149576   IO-APIC-fasteoi   uhci_hcd:usb2
20:          1   IO-APIC-fasteoi   uhci_hcd:usb3, ipw2200
21:          2   IO-APIC-fasteoi   ehci_hcd:usb4
22:          3   IO-APIC-fasteoi   ohci1394
NMI:          0
LOC:    2222598
ERR:          0
MIS:          0

而且irq或DMA有一个特性,那就是不能在用户模式下禁用。。You cannot use IRQs or DMA directly from a user−mode process. You need to write a kernel driver; see The
Linux Kernel Hacker's Guide for details and the kernel source code for examples.
You can disable interrupts from within a user−mode program, though it can be dangerous (even kernel
drivers do it for as short a time as possible). After calling iopl(3), you can disable interrupts simply by
calling asm("cli");, and re−enable them with asm("sti");.

如果我们手动的去分配这些东西,是不是太复杂了?性能有保证吗?
幸亏我们有irqbalance.先去读一下他的文档吧,知道它是什么?为什么要有它?怎么使用?http://irqbalance.org/documentation.php



通过这次的介绍,对kernel-utils有了更进一步的了解!但是对自己的挑战也越来越多!对于真正硬件底层的东西,缺乏的东西还真是太多了。
还有一个感慨就是,开源是无数细小的东西积累起来的!


CPU的频率之争到目前为止应该暂且告一段落,下一个竞争的目标是什么呢?大家应该有所觉察:多核CPU。可是操作系统和应用程序为多核CPU准备好了么?

这段时间测试一个SMP系统的网络吞吐量,发现大多数情况下只有一个CPU负载较重,另外一个CPU的负载几乎为0,查看他们对中断的响应情况,前者几乎响应了所有的网络中断请求。

因为Linux内核在处理网络数据包的过程中引入了软中断,所以硬中断只是将skb挂在响应此中断的CPU的软中断软网络设备(softnet)上,然后唤醒此CPU的软中断处理例程处理skb,所以响应此中断的CPU要处理所有网络数据包,负载当然很大。

IRQBalance能够在多个CPU之间均衡分配硬件中断,试图达到SMP系统CPU负载均衡的目的,启用此守护进程之后,IRQ的分配确实趋向平衡。仔细考虑,任何一个时间都只是有一个CPU在响应中断,那么IRQBalance所能做到的也无非就是在多个CPU之间均衡负载,这对于提升性能几乎是于事无补的。可实际的测试结果表明,IRQ的均衡确实提高了网络的吞吐量,一定是我漏掉了什么条件。再思之,对于网络包的接收中断是分硬中断和软中断两步处理的,其中硬中断的处理速度较软中断要迅速得多,我上面的考量只是对于硬中断有效,当网络负载比较大的情况下,软网络设备上肯定会积压很多数据包需要处理,而这时如果将数据包通过IRQ的均衡平均分配到多个CPU的软中断设备上,其后续的软中断处理的压力也会均分,网络吞吐量能不增加么?如果网卡的中断过于频繁,致使CPU大部分时间都在响应其硬中断而无暇他顾,此时中断机制相比轮询就是弊大于利了,一些人也许会考虑启用Linux的NAPI,这确实是个不错的主意,实际也能提高性能,不过这点儿提升只是杯水车薪而已,采用SMP系统或者是多核CPU加上IRQBalance才能大幅度提高网络的吞吐量。既然两者都能提升性能,为何不兼用之?这还得从NAPI的原理说起,目前的NAPI实现为:当网络流量特别大时,网卡将被关联到某个CPU的软中断上下文中,然后关闭其硬件中断,由接收软中断用轮询的方式从网卡取走数据包并处理之。如果SMP系统只有一块网卡流量较大,而它由于NAPI的缘故被关联到某一个CPU,并关闭了硬中断,这个时候发生的情况就合单CPU无异了,SMP系统的优势并没得到充分发挥。所以,在某种情况下两者是鱼和熊掌的关系--不可兼得,具体的取舍还要综合各方面因素具体分析。但是如果网卡首先成为了系统的瓶颈,那么两者也就都不能奏效了,你可以选择购买更高速率的网卡,或者是采用网卡bonding技术。

讨论到这里,基本就可以收场了。可当我读IRQBalance的文档时发现它所考虑的问题要比我多得多,比如说它的另外一个目的--节省功耗、均衡时的Cache命中率,以及更深层次的系统结构等。

不得不感叹:事物是普遍联系的。

[ 点这里复制网址,推荐给你QQ/MSN上的好友们! ]
本站声明:以上内容由网友 liquan8909 提供,与54master立场无关!
[ 顶部 ]
域名、虚拟主机、服务器租用/托管一站式IT服务 V5.0
  
 



当前时区 GMT+8, 现在时间是 2008-12-3 13:44

Powered by Discuz! 5.5.0  © 2001-2007 Comsenz Inc.
Processed in 0.066646 second(s), 12 queries , Gzip enabled

清除 Cookies - 关于我是网管 - 联系我是网管 - 广告服务 - 诚聘版主 - 无图版 - WAP -