root log: October 2010

It has been too long since my last root log. I am sorry. It has been a busy time.

You may perhaps have been wondering what we have been up to. Here are some things.

Nodeshells

I have recently made a page to list recently-created nodeshells, which is to say titles that people create which are waiting for someone to write something about them - nodes in the Everything2 net, that haven't netted any writeups yet.

The nodeshell has always been a controversial thing on Everything2, and one of the site's odder quirks. A nodeshell is an invitation to write, whose existence has formerly only been announced in the soft links that gather at the bottom of nodes, connecting one title to another. It sits there, existing only as a possibly-confusing set of links with a title, collecting new links as people traverse the site.

The new page lists nodeshells created within the last week, but at least half an hour old. The delay is to give people a chance to fill them in themselves before anyone else gets any ideas. Empty nodes which have been sitting there for a while are fair game, and a possible source of inspiration. If you are looking for inspiration, you might also like to look at Everything's Most Wanted, a list of things that people really want you to write about.

Softlinks

Softlinks are ordered by weight at the bottom of nodes, where the weight corresponds to how often users of the site have followed the link. The weight itself, however, has never been made visible to general users of the site. It is probably only of marginal interest to most people, so I didn't want it cluttering up the interface, but I figured it's interesting enough to get in there somewhere. If you are curious and technical enough, you can now find out the weight of any given softlink by finding out the class of its container - for instance class="sw12" means it has a weight of 12. I realise this is unwieldy, so let me know if you would be particularly interested in a more obvious way of finding it out.

The xml view of e2nodes (append ?displaytype=xmltrue to the url) now also includes softlink weight, as an attribute of the e2link tags. I have been having fun using Processing to produce visualisations of the softlink landscape, which I will be sharing with you once I feel like they're up to scratch.

There is much more that I think we could be doing with nodeshells, and softlinks. I may yet produce an RSS and/or Twitter feed of recent nodeshells, for example, and I think there could be some really interesting new ways of navigating the site.

Blocking of Users

In any group of people that is open to outsiders, people sometimes come along who just don't fit in. Even the most open of groups occasionally finds it needs to exclude people because their behaviour will otherwise lead others to exclude themselves.

Disappointingly I found myself spending a good deal of my coding time around about the middle of the year on measures to make it easier to exclude pathologically determined users. The nature of the internet is such that it is never possible to lock anybody out completely, and there are trade-offs to be made in attempting it, but it became clear that it would save us time and grief in the long term if we implemented some better mechanisms to make it awkward and time-consuming for people to keep getting in after they've managed to get themselves banned.

Audio

We have had an E2 Podcast produced infrequently for several years now. It's never seemed logical to me that there was no way of getting from writeups that people have recorded, to the recordings, so I have implemented a system for keeping links to mp3s on-site. Uploading to the E2 servers is still a little flaky, unfortunately, and only a few audio files are so far hosted on-site - still, you might like to check out the Recent Recordings node.

I've added mp3 links to writeups that have recording nodes in the database. These are just direct hyperlinks to the mp3 files, at the moment; it might be nice to put in a Flash player for them later.

Registries

Way back in the early days of Everything2, about a decade back, people pretty much posted any old thing about anything they wanted. As the site evolved the people who were in charge at the time figured it might be better if the site left out things that were of next to no interest to anyone who didn't know the people involved.

This decision made a huge difference to the evolution of the site, and the Everything2 you see before you would be very different if it had not focused on things that are potentially of interest to a large-ish audience. We probably would not have seen nearly so much really good writing otherwise, I think. On the other hand keeping out 'Getting To Know You' nodes did leave us with rather a limited number of ways of getting to know each other, which is perhaps odd on a site which has sustained itself largely on the back of housing an active and interesting community.

Community2 was an off-shoot of Everything2, created largely in response to this vacuum. With one thing and another Community2 didn't really work out, but being built on a modified version of the same code base, it has been possible to import some of its features with only minor modifications. Registries were one of its most successful features. Notably in10se's recent 'If I Were King of Everything2' includes several other changes which were implemented on C2 and worked quite well there: Voting and comments on pretty much everything, a 'journal' system pretty much like what he proposes to replace day logs as we know them. Many of his other suggestions are also things that we have been planning to do for some time. When we get some time.

I'm currently working on better integration and presentation of registries, so that we can replace the likes of the EMAR, the 'Mission Drive' and similar bits of home nodes, and so on.

The future

I've tended to avoid talking about big plans for the future, because we've seen a few sets of Big Plans For The Future come to nothing here, or possibly even a bit less than that. Still, maybe it's time I talked about it a bit more.

A major priority is switching from the mess of kludges that is Scratch Pads, Node Heaven etc. to a system of writeup drafts. We pretty much know what needs to be done here, it's just a question of finding the time to do it.

We all know that the messaging interface is broken; we have yet to implement the lessons of the last 10 years of interface design in that area. Replacing the Chatterbox as it is with something that doesn't lend itself to constant mistaken sharing of private messages is another high priority.

DonJaime's been working on switching all our JavaScript over to jQuery, which is really quite lovely, I find it much more pleasant to work with than the Prototype framework we've been using. Hopefully that'll make future JS coding here a bit simpler.

We're definitely revisiting the front page imminently, and call's mostly-wonderful mobile interface especially needs a good front page designing for it. Ruminations on what should and shouldn't be on the front page are currently being actively invited.

Raincomplex has done some sterling work on better integration of Categories, which is just waiting on a little bit of tidying up before we go live with it. The fact that both writeups and e2nodes can be added to categories complicates matters, especially in terms of presentation. Still, I'm quite excited about the prospect of adequate categorisation of the writing we have here; it's always been oddly absent from E2's interface. For reference, I'm picturing something functionally similar to Flickr's categorisations - with links to sets and usergroups a writeup is part of, and the next and previous entries within them. Perhaps also, as Flickr has, links to a user's next and previous writeups - an idea that's come up a couple of times and which I think would be useful enough to justify the clutter (one more line in a box containing all the different sorts of things a writeup belongs to). All this should be coming up very soon.

Of Libraries and $tuff

At some point during an online conversation, I had allowed myself to be volunteered to convert our JavaScript to use the jQuery library instead of the Protoculous library that most of it relied on. Both of them extend JavaScript functionality to make it easier to do things you often need to do on a web site. Protoculous does this in a way that is generally consistent with the conventions and mechanisms used by native JavaScript, making it appear that JavaScript is more powerful and versatile than usual. This means that unless you know exactly and exhaustively what functions and objects are supported by native JavaScript it is not always obvious that the code you are reading is reliant on an external library. It also means that it inherits some of the clumsiness built into JavaScript. jQuery is clearly visible in the code that uses it, easier to use, better documented, and better maintained. From my point of view its only drawback was that I knew next to nothing about it except that I liked the look of the documentation. One shared feature of both is the use of a function called $, whose name requires very little typing.

Towards jQuery and an e2 library

When an Everything2 page has been loaded from the server, it is but an inert pile of bytes. Letters on your screen, lifeless as the clay from which which we were formed. But then, by the power of JavaScript it comes alive: text boxes expand as you type into them! Widgets open and close at your behest! Your vote is packed into an empty rum bottle and cast into the heaving wild seas of the Internet! These functions had been added one by one, and the code that made them work and kept them working when the page was updated by AJAX was getting rather messy and strange, so I set up a generalised framework in default javascript to add JavaScript functions to an E2 page when it is loaded, and to any new page components added or updated later. This could be seen as a move towards a small JavaScript library for E2-specific functions.

Everything2 Ajax and Zen Nodelet Collapser needed updating to use the new mechanism. So since I was updating all three of our core JavaScript modules at once, I took the opportunity to start expanding my practical knowledge of jQuery by starting to replace Protoculous calls with jQuery, the initial target as far as that was concerned being replacement of the '$()' function provided by the one with the '$()' function provided by the other. Because there can be only one. Once that was done, a patch to Javascript Compiler reassigned '$' to jQuery from Protoculous for the three core scripts only, and things actually worked. So now we had both Protoculous and jQuery running in parallel.

Tweak and twiddle, seek out and destroy

A few things were broken as a result of the addition of jQuery, and to complete the move and drop Protoculous entirely it was necessary to find all the places it was being used. This was difficult for two reasons: firstly, as I mentioned, it doesn't advertise its presence, and secondly, it was present in a fair amount of obsolete and/or experimental code that wasn't actually in use. But the Revolution marched on inexorably, fixing:

• e2getFocus(): used to find where the focus was before replacing page elements and put it back inside them if it was in the replaced element.
• ColorClass.js, which is used in the E2 Color Toy, and which suffered from a strange incompatibility with jQuery.

and removing reliance on Protoculous from the core scripts and from:

• zensearchform: this is the search form at the top of the page. The full text search needs JavaScript. While I was at it, I fixed it so that it doesn't appear if JavaScript is switched off.
• The Notifications nodelet.
• Chatterbox. Also moved the in-line JS into a separate script.
• Chatterbox script: Moved the separate script into Everything2 Ajax and default javascript because it was getting loaded every time the chatterbox was refreshed, leading to large numbers of notifications in chatterlight that the chatterbox had stopped updating.
• script_tinyMCE, which sets up WYSIWYG text editing for the various text areas it's available for.
• classic user edit page: where you edit your profile. Or 'homenode', in our quaint jargon.
• Nodelet Section Collapser: a small javascript to make the non-collapsible nodelet sections (without [ - ] links) collapsible.

while also:

• Improving the consistency of the placing of widgets and making them respect the edges of the window.
• No longer setting the font size of expandable text input fields to 179px in Internet Explorer.
• Cleaning up and reorganising displaydebatecomment and showdebate for better-structured markup of usergroup debates, removing the tables used for formatting and using the show content htmlcode for consistency with markup elsewhere in the site. One day this code should be used for discussions of writeups.
• Adjusting pageheader to make it possible for pages to inject html into it. (It appears above the output of the page code, but is processed after it.)
• Putting the links from one settings page to the others in the page header.
• Making it possible for Guest User to try out different stylesheets using '?displaytype=choosetheme', so we can see what they would look like for Guest User.
• Adjusting registry display page to better control the length of registry entries using the general expandable text box code in default javascript.
• Making it possible to replace text in expandable text boxes that have reached their maximum permitted length by selecting existing text and typing over it.

... and writing replacements for a few functions present in Protoculous with no equivalent in jQuery:

• e2.periodical() to replace PeriodicUpdater()
• e2(<function>).defer() and e2(<function>).delay() to replace <function>.defer() and <function>.delay().

Guinea pigs required

I was fairly sure that no active code was left was left that needed Protoculous, and had already stopped serving it to Guest User, who gets a clearly-defined and relatively small amount of JavaScript, but to make sure, I stopped it being served to edev members and asked them to report any failure of function. That was on October 27th. Would the site collapse for the most tech-savvy portion of its users? Would DonJaime be accursèd among the noders for his unpardonable recklessness, or celebrated from the rooftops for his vision and for the swiftness of his pageloads? Find out in next month's even more exciting episode, coming soon to a root log near you!