看到名字:IRQ这是个linux newibe都知道的东西,倒退几年关于IRQ的问题一直是困扰newibe们的难题,至少现在仍是。
什么是IRQ?[size=-1]Interrupt Request的简写,翻译中文为“中断请求”。通过中断,外设可以取得CPU的处理时间。
linux会给每个设备分配一个中断号,常见的有
Device I/O address IRQ DMA
ttyS0 (COM1) 3f8 4 NA[a]
ttyS1 (COM2) 2f8 3 NA
ttyS2 (COM3) 3e8 4 NA
ttyS3 (COM4) 2e8 3 NA
lp0 (LPT1) 378-37f 7 NA
lp1 (LPT2) 278-27f 5 NA
fd0, fd1 3f0-3f7 6 2
fd2, fd3 370-377 10 3
当然还有很多,分配固定,那么就会有冲突。我们常见的问题是中断号冲突导致设备不可用,网卡经常遇到此种情形。
大家可以看下我本本的输出:
lee@lee-laptop:/proc/irq$ lspci -bv
00:00.0 Host bridge: Intel Corporation 82852/82855 GM/GME/PM/GMV Processor to I/O Controller (rev 02)
Subsystem: Hewlett-Packard Company Unknown device 3080
Flags: bus master, fast devsel, latency 0
Memory at <unassigned> (32-bit, prefetchable)
Capabilities: <access denied>
00:00.1 System peripheral: Intel Corporation 82852/82855 GM/GME/PM/GMV Processor to I/O Controller (rev 02)
Subsystem: Hewlett-Packard Company Unknown device 3080
Flags: bus master, fast devsel, latency 0
00:00.3 System peripheral: Intel Corporation 82852/82855 GM/GME/PM/GMV Processor to I/O Controller (rev 02)
Subsystem: Hewlett-Packard Company Unknown device 3080
Flags: bus master, fast devsel, latency 0
00:02.0 VGA compatible controller: Intel Corporation 82852/855GM Integrated Graphics Device (rev 02) (prog-if 00 [VGA])
Subsystem: Hewlett-Packard Company Unknown device 3080
Flags: bus master, fast devsel, latency 0, IRQ 10
Memory at e8000000 (32-bit, prefetchable)
Memory at e0000000 (32-bit, non-prefetchable)
I/O ports at 1800
Capabilities: <access denied>
00:02.1 Display controller: Intel Corporation 82852/855GM Integrated Graphics Device (rev 02)
Subsystem: Hewlett-Packard Company Unknown device 3080
Flags: bus master, fast devsel, latency 0
Memory at f0000000 (32-bit, prefetchable)
Memory at e0080000 (32-bit, non-prefetchable)
Capabilities: <access denied>
00:1d.0 USB Controller: Intel Corporation 82801DB/DBL/DBM (ICH4/ICH4-L/ICH4-M) USB UHCI Controller #1 (rev 03) (prog-if 00 [UHCI])
Subsystem: Hewlett-Packard Company Unknown device 3080
Flags: bus master, medium devsel, latency 0, IRQ 10
I/O ports at 1820
00:1d.1 USB Controller: Intel Corporation 82801DB/DBL/DBM (ICH4/ICH4-L/ICH4-M) USB UHCI Controller #2 (rev 03) (prog-if 00 [UHCI])
Subsystem: Hewlett-Packard Company Unknown device 3080
Flags: bus master, medium devsel, latency 0, IRQ 3
I/O ports at 1840
00:1d.2 USB Controller: Intel Corporation 82801DB/DBL/DBM (ICH4/ICH4-L/ICH4-M) USB UHCI Controller #3 (rev 03) (prog-if 00 [UHCI])
Subsystem: Hewlett-Packard Company Unknown device 3080
Flags: bus master, medium devsel, latency 0, IRQ 4
I/O ports at 1860
00:1d.7 USB Controller: Intel Corporation 82801DB/DBM (ICH4/ICH4-M) USB2 EHCI Controller (rev 03) (prog-if 20 [EHCI])
Subsystem: Hewlett-Packard Company Unknown device 3080
Flags: bus master, medium devsel, latency 0, IRQ 3
Memory at e0100000 (32-bit, non-prefetchable)
Capabilities: <access denied>
00:1e.0 PCI bridge: Intel Corporation 82801 Mobile PCI Bridge (rev 83) (prog-if 00 [Normal decode])
Flags: bus master, fast devsel, latency 0
Bus: primary=00, secondary=02, subordinate=06, sec-latency=64
I/O behind bridge: 00003000-00003fff
Memory behind bridge: e0200000-e02fffff
Prefetchable memory behind bridge: 30000000-33ffffff
00:1f.0 ISA bridge: Intel Corporation 82801DBM (ICH4-M) LPC Interface Bridge (rev 03)
Flags: bus master, medium devsel, latency 0
00:1f.1 IDE interface: Intel Corporation 82801DBM (ICH4-M) IDE Controller (rev 03) (prog-if 8a [Master SecP PriP])
Subsystem: Hewlett-Packard Company Unknown device 3080
Flags: bus master, medium devsel, latency 0, IRQ 255
I/O ports at <unassigned>
I/O ports at <unassigned>
I/O ports at <unassigned>
I/O ports at <unassigned>
I/O ports at 1810
Memory at 34000000 (32-bit, non-prefetchable)
00:1f.3 SMBus: Intel Corporation 82801DB/DBL/DBM (ICH4/ICH4-L/ICH4-M) SMBus Controller (rev 03)
Subsystem: Hewlett-Packard Company Unknown device 3080
Flags: medium devsel, IRQ 5
I/O ports at 1880
00:1f.5 Multimedia audio controller: Intel Corporation 82801DB/DBL/DBM (ICH4/ICH4-L/ICH4-M) AC'97 Audio Controller (rev 03)
Subsystem: Hewlett-Packard Company Unknown device 3080
Flags: bus master, medium devsel, latency 0, IRQ 5
I/O ports at 1c00
I/O ports at 18c0
Memory at e0100c00 (32-bit, non-prefetchable)
Memory at e0100800 (32-bit, non-prefetchable)
Capabilities: <access denied>
00:1f.6 Modem: Intel Corporation 82801DB/DBL/DBM (ICH4/ICH4-L/ICH4-M) AC'97 Modem Controller (rev 03) (prog-if 00 [Generic])
Subsystem: Hewlett-Packard Company Unknown device 3080
Flags: medium devsel, IRQ 5
I/O ports at 2400
I/O ports at 2000
Capabilities: <access denied>
02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ (rev 10)
Subsystem: Hewlett-Packard Company Unknown device 3080
Flags: bus master, medium devsel, latency 64, IRQ 10
I/O ports at 3000
Memory at e0207800 (32-bit, non-prefetchable)
Capabilities: <access denied>
02:06.0 Network controller: Intel Corporation PRO/Wireless 2200BG Network Connection (rev 05)
Subsystem: Hewlett-Packard Company Unknown device 12f6
Flags: bus master, medium devsel, latency 64, IRQ 4
Memory at e0206000 (32-bit, non-prefetchable)
Capabilities: <access denied>
02:09.0 CardBus bridge: Texas Instruments PCIxx21/x515 Cardbus Controller
Subsystem: Hewlett-Packard Company Unknown device 3080
Flags: bus master, medium devsel, latency 168, IRQ 255
Memory at e0209000 (32-bit, non-prefetchable)
Bus: primary=02, secondary=03, subordinate=06, sec-latency=176
Memory window 0: 30000000-33fff000 (prefetchable)
Memory window 1: 38000000-3bfff000
I/O window 0: 00003400-000034ff
I/O window 1: 00003800-000038ff
16-bit legacy interface ports at 0001
02:09.2 FireWire (IEEE 1394): Texas Instruments OHCI Compliant IEEE 1394 Host Controller (prog-if 10 [OHCI])
Subsystem: Hewlett-Packard Company Unknown device 3080
Flags: bus master, medium devsel, latency 64, IRQ 4
Memory at e0207000 (32-bit, non-prefetchable)
Memory at e0200000 (32-bit, non-prefetchable)
Capabilities: <access denied>
02:09.3 Mass storage controller: Texas Instruments PCIxx21 Integrated FlashMedia Controller
Subsystem: Hewlett-Packard Company Unknown device 3080
Flags: bus master, medium devsel, latency 64, IRQ 11
Memory at e0204000 (32-bit, non-prefetchable)
Capabilities: <access denied>
02:09.4 Generic system peripheral [0805]: Texas Instruments PCI6411/6421/6611/6621/7411/7421/7611/7621 Secure Digital Controller
Subsystem: Hewlett-Packard Company Unknown device 3080
Flags: bus master, medium devsel, latency 64, IRQ 11
Memory at e0208400 (32-bit, non-prefetchable)
Memory at e0208000 (32-bit, non-prefetchable)
Memory at e0207c00 (32-bit, non-prefetchable)
Capabilities: <access denied>
红色的字体显示的是IRQ同为10的输出。
当然也可以直接在/proc中查到:
lee@lee-laptop:~$ sudo cat /proc/interrupts
[sudo] password for lee:
CPU0
0: 9269905 IO-APIC-edge timer
1: 8989 IO-APIC-edge i8042
8: 1 IO-APIC-edge rtc
9: 19537 IO-APIC-fasteoi acpi
12: 121305 IO-APIC-edge i8042
14: 296468 IO-APIC-edge libata
15: 118272 IO-APIC-edge libata
16: 1 IO-APIC-fasteoi sdhci:slot0, sdhci:slot1, sdhci:slot2, yenta, tifm_7xx1
17: 502348 IO-APIC-fasteoi Intel 82801DB-ICH4
18: 1491470 IO-APIC-fasteoi uhci_hcd:usb1, eth0, i915@pci:0000:00:02.0
19: 149576 IO-APIC-fasteoi uhci_hcd:usb2
20: 1 IO-APIC-fasteoi uhci_hcd:usb3, ipw2200
21: 2 IO-APIC-fasteoi ehci_hcd:usb4
22: 3 IO-APIC-fasteoi ohci1394
NMI: 0
LOC: 2222598
ERR: 0
MIS: 0
而且irq或DMA有一个特性,那就是不能在用户模式下禁用。。You cannot use IRQs or DMA directly from a user−mode process. You need to write a kernel driver; see The
Linux Kernel Hacker's Guide for details and the kernel source code for examples.
You can disable interrupts from within a user−mode program, though it can be dangerous (even kernel
drivers do it for as short a time as possible). After calling iopl(3), you can disable interrupts simply by
calling asm("cli");, and re−enable them with asm("sti");.
如果我们手动的去分配这些东西,是不是太复杂了?性能有保证吗?
幸亏我们有irqbalance.先去读一下他的文档吧,知道它是什么?为什么要有它?怎么使用?
http://irqbalance.org/documentation.php

通过这次的介绍,对kernel-utils有了更进一步的了解!但是对自己的挑战也越来越多!对于真正硬件底层的东西,缺乏的东西还真是太多了。
还有一个感慨就是,开源是无数细小的东西积累起来的!
CPU的频率之争到目前为止应该暂且告一段落,下一个竞争的目标是什么呢?大家应该有所觉察:多核CPU。可是操作系统和应用程序为多核CPU准备好了么?
这段时间测试一个SMP系统的网络吞吐量,发现大多数情况下只有一个CPU负载较重,另外一个CPU的负载几乎为0,查看他们对中断的响应情况,前者几乎响应了所有的网络中断请求。
因为Linux内核在处理网络数据包的过程中引入了软中断,所以硬中断只是将skb挂在响应此中断的CPU的软中断软网络设备(softnet)上,然后唤醒此CPU的软中断处理例程处理skb,所以响应此中断的CPU要处理所有网络数据包,负载当然很大。
IRQBalance能够在多个CPU之间均衡分配硬件中断,试图达到SMP系统CPU负载均衡的目的,启用此守护进程之后,IRQ的分配确实趋向平衡。仔细考虑,任何一个时间都只是有一个CPU在响应中断,那么IRQBalance所能做到的也无非就是在多个CPU之间均衡负载,这对于提升性能几乎是于事无补的。可实际的测试结果表明,IRQ的均衡确实提高了网络的吞吐量,一定是我漏掉了什么条件。再思之,对于网络包的接收中断是分硬中断和软中断两步处理的,其中硬中断的处理速度较软中断要迅速得多,我上面的考量只是对于硬中断有效,当网络负载比较大的情况下,软网络设备上肯定会积压很多数据包需要处理,而这时如果将数据包通过IRQ的均衡平均分配到多个CPU的软中断设备上,其后续的软中断处理的压力也会均分,网络吞吐量能不增加么?如果网卡的中断过于频繁,致使CPU大部分时间都在响应其硬中断而无暇他顾,此时中断机制相比轮询就是弊大于利了,一些人也许会考虑启用Linux的NAPI,这确实是个不错的主意,实际也能提高性能,不过这点儿提升只是杯水车薪而已,采用SMP系统或者是多核CPU加上IRQBalance才能大幅度提高网络的吞吐量。既然两者都能提升性能,为何不兼用之?这还得从NAPI的原理说起,目前的NAPI实现为:
当网络流量特别大时,网卡将被关联到某个CPU的软中断上下文中,然后关闭其硬件中断,由接收软中断用轮询的方式从网卡取走数据包并处理之。如果SMP系统只有一块网卡流量较大,而它由于NAPI的缘故被关联到某一个CPU,并关闭了硬中断,这个时候发生的情况就合单CPU无异了,SMP系统的优势并没得到充分发挥。所以,在某种情况下两者是鱼和熊掌的关系--不可兼得,具体的取舍还要综合各方面因素具体分析。但是如果网卡首先成为了系统的瓶颈,那么两者也就都不能奏效了,你可以选择购买更高速率的网卡,或者是采用网卡bonding技术。
讨论到这里,基本就可以收场了。可当我读IRQBalance的
文档时发现它所考虑的问题要比我多得多,比如说它的另外一个目的--节省功耗、均衡时的Cache命中率,以及更深层次的系统结构等。
不得不感叹:事物是普遍联系的。