Everything2 Hubs (thing)

Update - I wrote a caching e2 proxy. If you're in edev, you have the URL (check your inbox)... If you're not but you have a strong desire to see it in its horribly incomplete state, /msg s_alanet. And those of you viewing this page through the proxy... Salud! More information below under "The Implementation."

Introduction

E2 has some problems. We've got a huge database to store, and hefty bandwidth requirements. We don't have the cash to buy a shiny cluster to host the site, or a bigger, phatter pipe to feed it through. There isn't really cash to hire full time developers or editors, though we have a lot of really wonderful volunteers that should be paid.

I was thinking about the whole distributing e2 thing, and I realized that we need a stopgap. An interim measure. Maybe a long term measure (after all, making a fully distributed e2 system is a big task - it might not be done for years!).

I was also thinking about the difference between e2 and a blog. I like my friend's blogs because they're very personal. I look at whose I want. If they're fancy I can have little discussion in the comment boards with my friends. I post stuff, they post stuff back - it's a lot of fun. But not very interesting for other people. E2 is the opposite - it's all about high quality writing for everyone to enjoy. If I post a story about my dog, it had better be good, not just some drivel that only people who have seen my dog will appreciate. Factual information - which would be boring on a blog - has a strong presence here.

We also have a lot of neat tools - the e2 node tracker, the E2 Nodegel Visualizer, and a lot of really cool clients.

How - I thought to myself - could these wonderful things be combined? It seems like they're all connected... but how?

And then it all came together.

The Idea

An Everything2 Hub - or an E2 Blog, or whatever you want to call it.

What I have in mind is a website that you can drop into your webspace on your ISP or university. It's a PERL or PHP script that automagically does everything to set itself up - a seed, so to speak. It probably uses flat files for maximum compatibility.

• It provides a blog syched with your daylogs. You post as you see fit - it deals with making sure the right daylog writeup on E2 is appended to, edited, or created. Sort of a blogger for the E2 set. It also displays the blog entries (including converting hardlinks to href's).
• It shows statistics relevant to you - latest writeups, for instance. People who are interested in you can hit your e2 hub and see what you've written. Optionally, more statistics could be available - most popular writeups, "goodest" writeups, least voted on writeups, etc.
• It provides a cached view of E2. Click on a link in the blog and it'll take you to a cached view of the node. You only hit e2's server if you want to.
• It provides "user services" to the owner of the blog. An integrated node tracker would be a good start (and pretty much a necessary feature to do the statistics). A nodegel visualizer would also be useful - in fact, there could be a whole subsection devoted to managing one's homenode. In addition, a scratchpad would be good (saving E2 from rendering lots of pageviews of the scratchpad), and the ability to make rough drafts of writeups, then upload them when they're ready, would be even better.

Basically what I'm talking about is a web-based client integrated with a web-based user info tool. A hub for your everything experience.

The Implementation

In the process of writing this thing, I've found some lingering XML malformations in E2's code. I've submitted patches for these. If you're using the e2hub and you get a blank page, it's probably because you hit a page with an unescaped character, like a bare & or something.

I also went through and cleaned a lot of stuff up in the code... It caches user information, nodes, and displays them. I'm about ready to call it 1.0 - I have a few issues with Simpleton's wonderful E2Interface library, mostly relating to me using it in ways it was not designed to be used, and when I clear those up, I'll be posting the code to the hub somewhere.

Last updated 15:52 23/2

So I've actually started to code some of the stuff mentioned here. Specifically, what I have at this point is a web-based caching E2 proxy. In other words, it grabs data from the various XML sources in the site to generate an e2-look-alike. It's got a 100 megs of cache space, and it keeps data cached for one hour. This is all tweakable. Right now it supports viewing nodes and users. If you try to view something else it will tell you to go back to E2 and look at it. The whole thing is templated.

This is the key to the hub concept, in my mind. Without a cache - without an interface to E2 - then I'm just talking about fancy newsletters. Now that I have a pretty solid cache implementation, I can really go somewhere.

If you have any questions, feel free to contact me.

Extensions

It's also not hard to imagine such a tool being turned towards making E2 more approachable. How often have you heard, "It seems nice, but I just don't think I'd fit in..."? We are losing talented contributors every day because they don't know what to make of hot nude thespians. Sure, it's a part of E2's charm.... but we're reaching the end of where charm can take us. We need quality to reach the next stage of development.

How better to attract quality contributors for music writeups than a hub that focuses on, say, "Music Reviews"? Or a "Computer Science" hub that focuses on terminology and concepts? A "classical literature" hub? E2 is a big place. If we want to lure in experts and promote really quality writing, we're going to have to start filtering things to fit the audience. Metanodes are a good start, but they're not enough.

It should always be easy to start free-associating - clicking through the database, wherever the information flow takes you. But if we want to attract serious contributors, we need to be able to present them focused information.

Conclusion

There are a lot of other areas that this concept could be extended along - letting people check their messages from their hub, for instance, or making a hub "farm", sort of like Sourceforge does for software projects or LiveJournal does for blogs. Such a site could even charge people a small fee for advanced functionality, or faster service - thus helping E2 stay afloat.

Call for Comment

But of course, all this is just conjecture. I would like to make it, or something like it, a reality. E2 should live on! So please, if you have any comments, thoughts, or suggestions on this, drop s_alanet a message or e-mail me at <bgarney@purdue.edu>. I'll note them down here (as well as thanking you profusely for your willingness to tolerate my ramblings :). If you are struck by the urge to start coding this, talk to me, too... I'd like to help (I don't have time to write it on my own).