https://groups.google.com/forum/#!topic/public-dns-discuss/p1o62SJElck public-dns-discuss › |
|
DNS Rate Limiting ICMP (8.8.8.8 and 8.8.4.4) 7 posts by 5 authors
|
| Brian Mcmichael |
9/9/16 |
Hi All,
We are a larger company with several DIA's. We noticed over the past week that saw ping to these two DNS servers were losing pings out one provider, but not the other. Calling the provider they indicated that Google is starting to deploy a rate-limit on ICMP, possibly for DDOS control. Is this true and will the larger community notice this with ping?
Thanks, Brian
| Brian Mcmichael |
9/9/16 |
Other recipients: 1pint.g...@gmail.com
Same situation. We had to route those blocks out our ATT link as a temporary solution.
Thanks
Brian McMichael
Network Architect
On Sep 9, 2016, at 3:54 PM, 1pint.g...@gmail.com wrote:
- Quote :
- We are a FTTH company with 2 Internet feeds from level 3 and Cogent. We are seeing the same issues as you but I don't believe this is due to rate-limiting. We are only experiencing packet loss across the Level 3 connection and not the Cogent. We also have customers that have been reporting slow response to anything google such as google search, youtube, etc. Also we have a customer using Google VPN for work and they are unable to hold up a connection.This leads me to believe they have a problem between Level 3 and Google. We have a ticket open with Level 3 but cannot find anywhere to open a ticket with Google.
- show quoted text -
| Alex Mercado |
9/12/16 |
Other recipients: bamm...@gmail.com
We are a FTTH company with 2 Internet feeds from level 3 and Cogent. We are seeing the same issues as you but I don't believe this is due to rate-limiting. We are only experiencing packet loss across the Level 3 connection and not the Cogent. We also have customers that have been reporting slow response to anything google such as google search, youtube, etc. Also we have a customer using Google VPN for work and they are unable to hold up a connection.This leads me to believe they have a problem between Level 3 and Google. We have a ticket open with Level 3 but cannot find anywhere to open a ticket with Google.
On Friday, September 9, 2016 at 1:37:16 PM UTC-5, bamm...@gmail.com wrote:
- show quoted text - | phillip....@level3.com |
9/15/16 |
Other recipients: bamm...@gmail.com, 1pint.g...@gmail.com
This is what I provided to Google and as you see at the time I was also seeing the issue through Cogent's lookingglass
Please provide any additional information below.Cogent
Test Router Location Hostname / IP Address
8.8.8.8
Go!
PING 8.8.8.8 (8.8.8.
56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=2 ttl=54 time=26.2 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=54 time=26.2 ms
64 bytes from 8.8.8.8: icmp_seq=4 ttl=54 time=26.1 ms
--- 8.8.8.8 ping statistics ---
5 packets transmitted, 3 received, 40% packet loss, time 5005ms <---- 40% packet loss
rtt min/avg/max/mdev = 26.152/26.196/26.231/0.189 ms
traceroute to 8.8.8.8 (8.8.8.
, 30 hops max, 60 byte packets
1 multi-use.cogentco.com (66.250.250.81) 1.193 ms 1.231 ms
2 te0-6-0-16.ccr21.dfw01.atlas.cogentco.com (154.54.44.193) 0.795 ms 0.809 ms
3 be2764.ccr41.dfw03.atlas.cogentco.com (154.54.47.214) 0.695 ms 0.799 ms
4 tata.dfw03.atlas.cogentco.com (154.54.12.106) 0.590 ms 0.560 ms
5 72.14.219.124 (72.14.219.124) 26.321 ms 209.85.172.106 (209.85.172.106) 26.254 ms
6 108.170.240.65 (108.170.240.65) 26.099 ms 108.170.240.193 (108.170.240.193) 26.153 ms
7 72.14.238.45 (72.14.238.45) 26.139 ms 72.14.234.151 (72.14.234.151) 26.109 ms
8 google-public-dns-a.google.com (8.8.8.
26.218 ms 26.083 ms
Level 3
ping 8.8.8.8 count 50
PING 8.8.8.8 56 data bytes
64 bytes from 8.8.8.8: icmp_seq=3 ttl=63 time=0.360ms.
64 bytes from 8.8.8.8: icmp_seq=4 ttl=63 time=0.360ms.
64 bytes from 8.8.8.8: icmp_seq=6 ttl=63 time=0.356ms.
Request timed out. icmp_seq=1.
64 bytes from 8.8.8.8: icmp_seq=7 ttl=63 time=0.348ms.
Request timed out. icmp_seq=2.
64 bytes from 8.8.8.8: icmp_seq=8 ttl=63 time=0.364ms.
64 bytes from 8.8.8.8: icmp_seq=9 ttl=63 time=0.340ms.
Request timed out. icmp_seq=5.
64 bytes from 8.8.8.8: icmp_seq=11 ttl=63 time=0.348ms.
64 bytes from 8.8.8.8: icmp_seq=12 ttl=63 time=0.340ms.
64 bytes from 8.8.8.8: icmp_seq=15 ttl=63 time=0.360ms.
Request timed out. icmp_seq=10.
64 bytes from 8.8.8.8: icmp_seq=17 ttl=63 time=0.348ms.
64 bytes from 8.8.8.8: icmp_seq=18 ttl=63 time=0.352ms.
Request timed out. icmp_seq=13.
Request timed out. icmp_seq=14.
64 bytes from 8.8.8.8: icmp_seq=20 ttl=63 time=0.352ms.
Request timed out. icmp_seq=16.
64 bytes from 8.8.8.8: icmp_seq=24 ttl=63 time=0.376ms.
Request timed out. icmp_seq=19.
64 bytes from 8.8.8.8: icmp_seq=26 ttl=63 time=0.348ms.
Request timed out. icmp_seq=21.
64 bytes from 8.8.8.8: icmp_seq=27 ttl=63 time=0.356ms.
Request timed out. icmp_seq=22.
Request timed out. icmp_seq=23.
Request timed out. icmp_seq=25.
64 bytes from 8.8.8.8: icmp_seq=32 ttl=63 time=0.344ms.
64 bytes from 8.8.8.8: icmp_seq=33 ttl=63 time=0.360ms.
Request timed out. icmp_seq=28.
64 bytes from 8.8.8.8: icmp_seq=34 ttl=63 time=0.352ms.
Request timed out. icmp_seq=29.
64 bytes from 8.8.8.8: icmp_seq=35 ttl=63 time=0.352ms.
Request timed out. icmp_seq=30.
Request timed out. icmp_seq=31.
64 bytes from 8.8.8.8: icmp_seq=40 ttl=63 time=0.356ms.
64 bytes from 8.8.8.8: icmp_seq=41 ttl=63 time=0.348ms.
Request timed out. icmp_seq=36.
Request timed out. icmp_seq=37.
Request timed out. icmp_seq=38.
^C
ping aborted by user
---- 8.8.8.8 PING Statistics ----
43 packets transmitted, 21 packets received, 51.16% packet loss
round-trip min = 0.340ms, avg = 0.353ms, max = 0.376ms, stddev = 0.000ms
traceroute 8.8.8.8
traceroute to 8.8.8.8, 30 hops max, 40 byte packets
1 0.0.0.0 * * *
2 0.0.0.0 * * *
3 Google-level3-100G.Dallas1.Level3.net (4.68.72.138) 4.82 ms 4.81 ms 4.86 ms
4 108.170.240.129 (108.170.240.129) 5.01 ms
4 108.170.240.1 (108.170.240.1) 4.96 ms 5.00 ms
5 72.14.234.107 (72.14.234.107) 5.06 ms
5 72.14.234.149 (72.14.234.149) 5.07 ms
5 64.233.175.101 (64.233.175.101) 5.28 ms
6 google-public-dns-a.google.com (8.8.8.
4.96 ms 4.94 ms 4.98 ms
- show quoted text -- show quoted text - | Alex Dupuy |
9/15/16 |
TL;DR: At the risk of repeating myself: Google Public DNS is a Domain Name System service, not an ICMP network testing service.
If you want to measure the quality of your DNS service from Google Public DNS, you should use a dnsping tool (https://github.com/farrokhi/dnsdiag [Python] or https://sourceforge.net/projects/dnsping/ [C#]) to send real DNS queries and check for responses. Note that while traceroute -U sends UDP/53 packets, they are
not DNS queries, and traceroute -U is
not a substitute for dnsping.
If dnsping shows significant levels of unanswered queries (and especially if ping and traceroute do not show any drops), you should check whether your IP address is generating more than 100 queries per second (the default per-IP address QPS limit for Google Public DNS). If you are legitimately generating more than 100 QPS and need to increase your QPS limit, you can request an increase through the Google Public DNS issue tracker.
While Google does not block ICMP or random UDP to 8.8.8.8 or other Google Public DNS IP addresses, there
are rate limits on ICMP error replies from Google networking equipment, and ICMP error replies may be de-prioritized within Google networks and could be more likely to be dropped.
There are sometimes network anomalies (I hesitate to call them denial of service attacks, since ICMP is a laughable DoS vector) that send large volumes of ICMP queries to Google IP addresses, exceeding the rather high rate limiting thresholds, and suppressing some ICMP replies. This can cause ICMP-based network monitoring to report "dropped" packets, even when no packets are being dropped (Google is merely choosing not to respond to all ICMP echo requests). For much of the past two weeks ICMP traffic to Google in the DFW area has been affected by such an anomaly.
For this reason, occasional ping and traceroute drops are not evidence
by themselves of a problem with your connectivity to Google Public DNS, and do not reflect the quality of service that your DNS queries are receiving. If there is a real networking problem (high latencies or large numbers of drops) traceroute may be useful to diagnose and understand the source or nature of the problem, but evidence of
DNS failures is needed to demonstrate the existence of a problem.
Furthermore, if you
are having a networking (not DNS-specific) issue with your connection to Google, any DNS engineers you may reach through this forum or the issue tracker are not able to do anything about it (we
may forward reports to networking teams in the case of complete connectivity loss causing SERVFAIL for domains, but that is not guaranteed). You should contact the Google networking teams through the ISP portal at isp.google.com or by escalating through your upstream ISP if you do not have a peering or GGC relationship with Google yourself.