Someone perpetrated what amounted to a denial of service attack against E2 in mid-January, which is why the site was so unresponsive for several days there. The perpetrator was a largely unrepentant noder of many years who, in violation of good manners, good conscience, and the law itself, was making a copy of all of the publicly available content of the site. If a piano falls on this gentleman, then cats jump out of that piano, then those cats mercilessly rend his still-twitching corpse, and then his mind is restored inside a computer, but he is only able to communicate by writing COBOL at 1 character per second for the remainder of eternity, I will have a renewed belief in karma. More likely, hopefully this experience makes him realize the damage that a small amount of coding knowledge can do, and perhaps he will be more communicative with those he might do harm to in the future.
But a single person spidering E2 should not be sufficient to cause a major negative effect on the site. The reasons it did have such an effect were manifold. For one, there were several superdocs where gods had played around with code in the past which were publicly viewable. These nodes were almost never accessed, but when they were, some of them threw crazy errors or made intense database queries which sometimes slowed down the whole site and, in some occasions, actually crashed Apache. I tidied up a few of these, but we coders should all vet the old superdocs in the coming weeks and delete or nerf them as necessary. For two, there aren't a great many safeguards to prevent bugs from killing the whole site. This is why all new code development should be done on the dev server. For three, the perpetrator repeatedly hit these problematic pages when they generated errors, compounding the problem.
There is good news to come of this, though. The problems which sprouted up prompted some positive code changes. Better logging was added to track down some problems, some failsafe code was added to prevent particularly broken code from seizing the database, and I got to test some logging and statistics software installed in the last few months. I also took a look into mod_security which has the capability to recognize some automated attacks of this sort and nip them in the bud. Given the rarity of this sort of problem and the resources required for such a solution, though, I laid off on that one. Still, forewarned is forearmed.
Bug fixes up the wazoo
So I know I'm sometimes short and impatient when dealing with bugs. That's pretty crummy from your perspective, because not only do you have to deal with something that's broken on the site, you then have to work your way through the accumulated grump of some coder dude to get it fixed. That's lame. I'm working on my attitude. Promise.
That said, bug reports and suggestions drive a huge amount of the improvements for E2, especially the ones I've put in this last month. I really appreciate everyone who takes the time report a bug, especially when there is sufficient detail that I can track it down and nail it without much effort. I appreciate that giving a good bug report is hard, especially if you're not a coder. A good bug report is essentially a story:
- Introduction: What you were doing
- Body: What you expected to happen
- Climax: What actually happened
- Conclusion: Why that's different than expected
Gap in the Logs
The last time I wrote up a root log, I think I stated something about high ideals in listing all of the patches for a month, regardless of authors, short commentaries, yadda, yadda. Hasn't happened yet. Sometimes big plans fail. Let's start small and get this month out while it's still fresh in my mind.
Enumerated List of All Patches By Me
- Everything2 Ajax
- Windows which were doing automatic updates were often getting their titles set to a blank string. A couple of people reported this one, but I fear I didn't keep track. Fixed. Thanks dudes!
- Javascript Compiler
- The left/right/center buttons in the WYSIWIG editor were non-functional. Had to fix this up. Again, thanks for the bug report forgotten noder!
- writeup maintenance create
This was a rare problem, but it got triggered by a new user and made them lose a fair amount of work. If you opened a window to a node, typed up a writeup, then opened another window to the same window, submitted that second writeup, then submitted the first, you'd annihilate the version of the writeup you submitted first.
This weird behavior was in there because writeups would get half-posted and then time out, or people would double click to post a writeup, and we just wanted to elegantly deal with it. But that was way back in the day. This also nipped raincomplex back in November when e was using an offsite-updater for a writeup. That should work now, even if op=new is accidentally sent with an update.
- Node backup
- Multiple patches. wertperch reported that he was unable to open the files created by Node Backup, and I noticed he had a writeup in ++ which was creating a file ++-(idea).html which I figured might be causing an error. Most non-alphanumeric characters are now replaced by dashes, so the writeup for ++ is titled -- (idea).html. Last I checked, I could open the resulting archive, but he still couldn't.
- chatterlight
Gorgonzola noted that he normally has the notifications nodelet turned off. As a result, whenever he went to chatterlight, it would pop up the settings widget, since he had no notifications turned on. I just made it so, if you have the notifications nodelet turned off normally, it doesn't show up in chatterlight.
Realistically, this might just be a stopgap. I think we should switch over to chatterlighter in general since it has shown to be superior to chatterlight in multiple ways, including the capability to customize nodelets more generally.
- Registry Information
- At the request of tentative, added a link to The Registries so it was easy to find where to enter more information after you've reviewed what you've entered.
- category display page
- displaywriteuptext
- category edit page
- Added the ability to enter a description for a category which will show up when you view the category.
- Various
- Did some security work to correct an issue reported by albinowax a while back. Thank you, albinowax!
- confirmop
- basesheet
- ajax update page
- zen user display page
- We have a rarely used feature to catch persistent problem users and mitigate the damage they can do. It's now easier for staff to identify these users and resolve the condition if there's a false positive.
- E2 Gift Shop
- Since Oolong made the Topic Archive, I added a display in the gift shop to report last topic setting for easy reference. This way you can copy/paste to reuse the old topic if you don't want to wipe out a recent topic change.
- Most Wanted
- mauler was doing some work with the Most Wanted nodelet. I helped out by adding some links. Feel free to enable it, now available in nodelet settings.
- Statistics
- mauler asked why statistics was disabled. I replied that some of the site-wide things it calculates are still super-expensive, and I hadn't devised a workaround. He asked why not just re-enable it with personal stats. Good question. Re-enabled.
- softlink
- avalyn pointed out that a lot of softlinks for nodes which were written as HTML entities, like ♥ looked ugly because they were escaped and looked like ♥. We had no reason to be doing this since we filter out dangerous titles in other places, so I removed the escaping. Prettier softlinks!
- softlink
- firmlinks
- linktrim
- Settings
- addfirmlink
- firmlink
- firmlink_note
-
SIGTITLE can now remove as well as add firmlinks. Added a security check. Added the ability to explicate firmlinks so they might look like:
See also: Black Swan for information about the animal and the 2010 movie
Sadly, no one has used this feature yet.
- Kernel Blue
- zenwriteups
- At the request of GhettoAardvark, all writeups which you can't vote on are styled similarly. Previously, just writeups you had voted on were styled differently. Now ones you wrote yourself are styled the same. For the zenmasters out there: I added the class mine to links to your own writeups in New Writeups, then styled that class the same way as hasvoted
- linktrim
- The list of favorite users in Settings now lets you remove users. Previously, this only worked for gods. Reported by GhettoAardvark. Thanks!
- epicenterZen
- Scratch Pads
- The "printable" link in Scratch Pads didn't work if you wanted to see a printable view of a writeup. Fixed. Also reported by GhettoAardvark. He's a bug smashing machine.
Now, ideally we'd just phase out the printable link entirely. The reason for this is that we serve a stylesheet which tells your browser how to style the site when printing. If you hit print preview, you should see the site rendered in its printable format. But for the time being, I opted to make the feature work rather than axe it.
- rtnsection_cwu
- In the Read This nodelet, only list each cooled writeup once, even if it has been C!ed multiple times.
ecore patches
- patchset c035e62ca6924d301bc1307abf4c597a80b6547b: Log HTTP parameters on errors. Prevent stupid SQL query from being run and log so we can track it down.
-
Log all CGI parameters when an error occurs so that we can easily relate the code to the inputs. (This was an annoying process of matching logs between machines before.)
Also, don't let a particularly stupid sort of query be run. This could only happen if a getNodeWhere() or similar function were called with broken parameters, but exactly that sort of broken thing was done in at least a few places.
- patchset 33708198307366110b39144548f0a49a57ef1adb: Blank HEADER_PARAMS earlier in the page processing so that opcodes can redirect.
-
If an opcode or maintenance node wanted to set header parameters to serve a cookie, make a redirect, or show a forbidden page, they were unable to do so because HEADER_PARAMS were blanked way too late in the page loading process. I moved this earlier. This was done in part for the writeup mainteance create patch noted earlier