Linux IP 转发不起作用

Question

我遇到了这个奇怪的问题。相同的设置之前有效，但突然停止。 两台机器 [Internet] 网关 核心

我正在尝试通过网关路由 Core。核心：

root@core:~# ifconfig eth1
eth1      Link encap:Ethernet  HWaddr 76:61:6b:7a:65:af
          inet addr:10.0.0.2  Bcast:10.0.0.255  Mask:255.255.255.0
          inet6 addr: fe80::7461:6bff:fe7a:65af/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:38423 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3814 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:1959037 (1.9 MB)  TX bytes:501771 (501.7 KB)

root@core:~# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.0.0.1        0.0.0.0         UG    0      0        0 eth1
10.0.0.0        0.0.0.0         255.255.255.0   U     0      0        0 eth1

root@core:~# ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=48 time=10.6 ms
^C
--- 8.8.8.8 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 10.644/10.644/10.644/0.000 ms

网关：

root@gateway:~# cat /proc/sys/net/ipv4/ip_forward
1
root@gateway:~# ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 3e:50:8a:be:b9:80
          inet addr:83.222.241.213  Bcast:83.222.241.255  Mask:255.255.255.0
          inet6 addr: fe80::3c50:8aff:febe:b980/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:4536 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4197 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:485439 (485.4 KB)  TX bytes:798131 (798.1 KB)

root@gateway:~# ifconfig eth1
eth1      Link encap:Ethernet  HWaddr 42:50:8a:be:b9:80
          inet addr:10.0.0.1  Bcast:10.0.0.255  Mask:255.255.255.0
          inet6 addr: fe80::4050:8aff:febe:b980/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1985 errors:0 dropped:0 overruns:0 frame:0
          TX packets:13169 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:256280 (256.2 KB)  TX bytes:701930 (701.9 KB)

root@gateway:~# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         83.222.241.1    0.0.0.0         UG    0      0        0 eth0
10.0.0.0        0.0.0.0         255.255.255.0   U     0      0        0 eth1
83.222.241.0    0.0.0.0         255.255.255.0   U     0      0        0 eth0
root@gateway:~# iptables-save
# Generated by iptables-save v1.4.21 on Thu Oct 23 23:13:32 2014
*nat
:PREROUTING ACCEPT [3:180]
:INPUT ACCEPT [3:180]
:OUTPUT ACCEPT [173:10388]
:POSTROUTING ACCEPT [170:10200]
-A POSTROUTING -o eth0 -j MASQUERADE
COMMIT
# Completed on Thu Oct 23 23:13:32 2014
# Generated by iptables-save v1.4.21 on Thu Oct 23 23:13:32 2014
*filter
:INPUT ACCEPT [581:49229]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [819:73373]
-A FORWARD -i eth1 -j ACCEPT
COMMIT
# Completed on Thu Oct 23 23:13:32 2014
# Generated by iptables-save v1.4.21 on Thu Oct 23 23:13:32 2014
*mangle
:PREROUTING ACCEPT [581:49229]
:INPUT ACCEPT [581:49229]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [822:73737]
:POSTROUTING ACCEPT [822:73737]
COMMIT
# Completed on Thu Oct 23 23:13:32 2014
# Generated by iptables-save v1.4.21 on Thu Oct 23 23:13:32 2014
*raw
:PREROUTING ACCEPT [581:49229]
:OUTPUT ACCEPT [822:73737]
COMMIT
# Completed on Thu Oct 23 23:13:32 2014

现在在我运行的网关上运行tcpdump -i any -n -v udp and port 53：

root@gateway:~# dig test.com @8.8.8.8 &>/dev/null
23:16:33.426336 IP (tos 0x0, ttl 64, id 16201, offset 0, flags [none], proto UDP (17), length 65)
    83.222.241.213.41376 > 8.8.8.8.53: 39515+ [1au] A? test.com. (37)
23:16:33.436145 IP (tos 0x0, ttl 49, id 26701, offset 0, flags [none], proto UDP (17), length 81)
    8.8.8.8.53 > 83.222.241.213.41376: 39515 1/0/1 test.com. A 50.23.225.49 (53)

如果我在 Core 上运行 dig（从网关捕获）：

23:17:55.801448 IP (tos 0x0, ttl 64, id 50634, offset 0, flags [none], proto UDP (17), length 65)
    10.0.0.2.55008 > 8.8.8.8.53: 6910+ [1au] A? test.com. (37)
23:17:55.801539 IP (tos 0x0, ttl 63, id 50634, offset 0, flags [none], proto UDP (17), length 65)
    83.222.241.213.55008 > 8.8.8.8.53: 6910+ [1au] A? test.com. (37)
23:18:00.801477 IP (tos 0x0, ttl 64, id 50635, offset 0, flags [none], proto UDP (17), length 65)
    10.0.0.2.55008 > 8.8.8.8.53: 6910+ [1au] A? test.com. (37)
23:18:00.801559 IP (tos 0x0, ttl 63, id 50635, offset 0, flags [none], proto UDP (17), length 65)
    83.222.241.213.55008 > 8.8.8.8.53: 6910+ [1au] A? test.com. (37)
23:18:05.801862 IP (tos 0x0, ttl 64, id 50636, offset 0, flags [none], proto UDP (17), length 65)
    10.0.0.2.55008 > 8.8.8.8.53: 6910+ [1au] A? test.com. (37)
23:18:05.801932 IP (tos 0x0, ttl 63, id 50636, offset 0, flags [none], proto UDP (17), length 65)
    83.222.241.213.55008 > 8.8.8.8.53: 6910+ [1au] A? test.com. (37)

所以不知何故我的 ping 被发送了，但我的 UDP 数据包没有？ 我猜 ip_forward 不会以某种方式重写源地址？ 有什么想法吗？

Answer 1

所以问题出在 QEMU 中的 VirtIO 网卡。

当两个 VirtIO 虚拟机在同一台物理机器上运行时，NAT 后面的虚拟机出于某种奇怪的原因开始发送校验和错误的数据包。关闭校验和检查可以解决问题：

ethtool -K eth1 tx off tso off ufo off gso off

并确保它在重新启动后仍然存在：

echo "ethtool -K eth1 tx off tso off ufo off gso off" >> /etc/rc.local

感谢我的 VPS 支持。