The last few days have been quite
fun... my poor webserver is looking a bit ragged:
Monday 25th February 2002, approx 1100hrs GMT
Transfer Statistics: 83 hits per hour, approx 0.73MB/h
Percentage of usual transfer rate: 100%
Total hits today: 2,000
While
spodding on
Uberworld, a friend posts a URL of a directory containing .jpg images of all the
Playboy Playmates from 1960 to 2000. It occurs to me that it would be cool to have something that would let you work out who was
Playmate Of The Month when you were born, so I hack a quick PHP script to do just that, and put it on my website.
1300hrs GMT
I post the URL to the
uknot mailing list, cos it's the kind of thing they dig. There's a bit of discussion but it's not a very big hit, so I forget about it.
Monday 25th February 2002, approx 2300hrs GMT
Transfer Statistics: 640 hits per hour, approx 2MB/h
Percentage of usual transfer rate: 771%
Total hits today: 15,395
Whilst I'm in the pub, an article I submitted to
Slashdot is posted, and my website is linked to from the front page. Because it's not the focus of the article I don't get the full
Slashdot Effect, but there's a noticeable increase in hit rate.
Tuesday 26th February 2002, approx 0930hrs GMT
Transfer Statistics: 1,092 hits per hour, approx 3.4MB/h
Percentage of usual transfer rate: 2,400%
Total hits today: 26,219
I get into work and notice the increase in the hit rate, but I don't know about the dramatic increase currently under way (log analysis is done automatically at 2300 each night). In hindsight, it seems clear that the subscribers to
uknot have taken my link and posted it on
bulletin boards,
newsgroups,
blogs and other
mailing lists. So whilst I'm at work, the hit rate is rapidly increasing.
Wednesday 27 February 2002
Transfer Statistics: 3,992 hits per hour, approx 3.4MB/h
Percentage of usual transfer rate: 1,150%
Total hits today: 95,827
I'm busy all day Wednesday morning and assume that the hit rate hasn't changed much since yesterday, but at lunchtime I manually invoke
webalizer to see what's going on, and get a bit of a shock. The link to my page has spread around the world, in the referrer logs you can see sites in
.ru,
.de and
.nl. But I'm out in
London with friends in the evening so after 5.30 I don't know what's going on.
Thursday 28 February 2002, approx 1000hrs GMT
Transfer Statistics: 10,416 hits per hour, approx 32MB/h
Percentage of usual transfer rate: 1,240%
Total hits today: 249,326
I log in first thing and get a shock! Now I start to worry, because my website is on a cheap rented box and has fixed bandwidth requirements. So I ask my host about the transfer situation, and it turns out that in the last 48 hours my one page has used up 3/4 of a month's worth of bandwidth. Time to worry!
1300hrs GMT
My host gets an e-mail from the hosting provider complaining about bandwidth usage. At times I've been using up to 10% of their total network capacity, and other hosts on their network are becoming inaccessible. Time to do something...
1430hrs GMT
Phew... we now have two mirror sites with DNS-level
load balancing (using
round-robin) to share the load - and I've dropped in an
Amazon banner to try to offset the cost of extra bandwidth we'll have to come up with. America has now woken up and the hit rate seems as strong as ever - let's see what tomorrow brings!
State of play at 2115hrs GMT:
Total hits above expected: 455,761
Total amount of data transferred: 1.4GB
I'm not going to advertise my website here, in fact you'd be doing me a favour not looking at it! But if you feel the need to, it's on
my homenode.
Output from webalizer:
+------------------------------------------------------------------------------------+
| Daily Statistics for February 2002 |
|---+--------------------------------------------------------------------------------|
|Day| Hits | Files | Pages | Visits | Sites | KBytes |
|---+------+------+------+------+-----+------+-----+------+-----+------+------+------|
|24 | 1140| 0.28%| 1008| 0.29%| 314| 0.24%| 130| 0.53%| 114| 0.62%| 8211| 0.58%|
|---+------+------+------+------+-----+------+-----+------+-----+------+------+------|
|25 | 15395| 3.74%| 12621| 3.67%| 4355| 3.38%| 720| 2.94%| 691| 3.73%| 51919| 3.67%|
|---+------+------+------+------+-----+------+-----+------+-----+------+------+------|
|26 | 26219| 6.36%| 21651| 6.30%| 8311| 6.45%| 1116| 4.56%| 974| 5.26%| 83618| 5.91%|
|---+------+------+------+------+-----+------+-----+------+-----+------+------+------|
|27 | 95827|23.25%| 79030|23.00%|29181|22.65%| 4281|17.51%| 3710|20.03%|287036|20.29%|
|---+------+------+------+------+-----+------+-----+------+-----+------+------+------|
|28 |249326|59.62%|207291|59.67%|82035|62.31%|16978|65.46%|13345|68.13%|784752|54.64%|
+---+--------------------------------------------------------------------------------+
Friday 1 March, 2002 1255hrs GMT
Transfer Statistics: 1,833 hits per hour, approx 7MB/h
Percentage of usual transfer rate: 2,400%
Total hits today: 42,000 (est)
At some point the company hosting the images worked out what was going on and pulled the site. So my page has stopped working. Oh well. I've switched off the mirrors and put a low-fat page up explaining what happened, and I've already had people e-mailing me about hosting the page on their sites.
I just checked the logs from the mirror sites:
[jodrell@spod.uk.net]$ gunzip -c mirror1.log.gz mirror2.log.gz | wc --lines
170683
I think that they went up a bit too late really, but they still soaked up about a fifth of the total number of hits.
Anyway, I think it's subsiding now. Obviously people won't circulate a link that doesn't do anything. But it's a lesson learned - the web works opposite to the normal rule - the good stuff gets beaten down because of bandwidth costs, and the crap gets left hanging around.