In 2003 University of Wisconsin Madison ("UoWM") began to suffer a massive influx of
NTP traffic directed at its NTP server. This was not an intentional
attack. I think it's interesting.
Symptoms
On 14th May 2003 UoWM's network started receiving a rapidly escalating deluge
of IP traffic of the order initially 40,000 packets per second. The University
staff responded pretty rapidly and identified sufficient characteristics in the
traffic to block it in their ISP's border routers (1). It's characteristics were:
This looks very much like a
distributed denial of service attack, for example
implemented by a
virus.
Traffic continued to escalate to the point that it exceeded 250,000 packets per second
consistently dropped by WiscNet's border router. (1) explains the procedure that
UoWM staff employed to determine the source, which makes good reading in itself. The
outcome was that the packets came exclusively from certain models of Netgear
domestic router. The router polls the hard coded address of UoWM's NTP server once
every second until it gets a response. Since at least 250,000 other guys are doing
the same thing, it has little chance of getting such a response. Thus the behaviour
builds up into a massive deluge of NTP traffic, with little prospect of ever declining.
Resolution
Initially UoWM's ISP, WiscNet filtered out the traffic. So the Internet core has been carrying
all this stuff, but generally it never got answered. Netgear ultimately produced firmware patches
which fixed the problem. However, the consumers who bought the items have to do this
upgrade on their own initiative.
Analysis
Netgear estimated around 700,000 affected devices were sold and this is consistent
with the measurements taken by WiscNet and UoWM; UoWM estimate the half-life of
these devices in the problematic state as five years (1). Is this the longest and
most comprehensive denial of service attack the world has ever seen? Even Windows
and Outlook viruses have shorter half lives than this - the average consumer is
surely more likely to run Windows Update than to do something obscure like
download firmware onto a router.
Bugs in passive-looking routers have historically been produced in business
quantities, and given to enterprise IT staff, ISPs and telcos. Retail leverages
massive economy of scale to produce huge numbers of
units and give them at low cost to people who don't understand them at all.
This means bugs get distributed much more widely and more rapidly, and are far
less likely to get fixed in a hurry.
Asides
I own one of the affected routers and it works just fine for me. Needless
to say I upgraded the flash and reconfigured the NTP server
address in it at an early stage. I
don't personally have a problem with Netgear or Nortel.
(1) is the definitive write up on this topic, but is much too long and graphical
for e2.
References
1. "Flawed Routers Flood University of Wisconsin Internet Time Server",
Dave Plonka, August 21, 2003 - University of Wisconsin-Madison
2. RFC 1305 "Network Time Protocol Version 3 - Specification, Implementation and
Analysis" David L. Mills, March 1992
3. RFC 2030 "Simple Network Time Protocol (SNTP) Version 4 for IPv4, IPv6 and OSI"
D. Mills, October 1996
4. Internet Assigned Numbers Authority (IANA) TCP and UDP Port Numbers registry