edev FAQ

Venerable members of this group:

jaybonci@, Oolong@+, danne, N-Wing, bol, Teiresias, edev Bot, Orange Julius, rdude, call, ascorbic@, Vice_hkpnx, in10se, elem_125, Devon, dafydd, opteek, Two Sheds, nosce, The Lush, timgoh0, craze, Redalien, maxClimb, randrews, DTal, DonJaime, Gryffon, themanwho, visudo, raincomplex, albinowax, No Springs, loughes, WaldemarExkul, prole, moosemanmoo, Rapscallion, DarkDigitalDream, jdporter, user-970414, rycerice, Scout, Aerobe, OldMiner, quintopia, mcd, E2D2, Serjeant's Muse, BaronWR, Old_New, Auspice@, Simone
This group of 53 members is led by jaybonci@

So Glowing Fish sez to me, he sez...

    So I am thinking of e2, and the e2izer. The biggest problem with e2 is
 the server bill, and it may happen that e2 eventually goes down if Hemos
 stops paying the bill.
    But there is really no reason why e2 has to all be on one server. e2 could 
be split up between hundreds or thousands of servers. The e2 program could turn 
into a protocol, where certain webpages, all on different servers, kept a 
header that kept track of reputation, C! and stuff...there perhaps could be one 
main page where the names of new nodes and users were kept, but it wouldn't be 
accessed all that much, and would take up a lot less bandwidth. I suppose 
authentication and quality control would be a lot harder, but it could probably 
still be done.

    What do you think? Is it possible?

So I sez to Glowing Fish, I sez...

Yes, I think it's pretty possible - from the perspective of nodes and users and such. It would fundamentally change the nature of e2, of course. :)

But it could be done using trust metrics and the like. The Proper Way to do it, in my opinion, would be to release an e2.net node package that would obey some certain XML-RPC based protocols. These would be put up by people with servers and bandwidth, and connect into the e2 network, acting like nodes in a gnutella network would. Catbox messages could be passed by similar means.

Advantages of the distributed model:

  • Nate et al would remain gods. Your high rankingness on an important server would remain useful!
  • Make yourself uber god on your own server. It doesn't matter. If your server is not respected then you get nothing.
  • Trusted content can be easily filtered from the untrusted and backed up and passed around.
  • Bandwidth et al becomes less of a problem, though updates and such could become Uglier. This might be alleviated by having some client side logic to make searches and stuff more intelligent. The trick is to make sure the distribution bandwidth used is less costly than the server bandwidth that would be used.
  • You could use whatever language you liked s'long as it obeyed the protocol. PHP, PERL, Brainfuck, C, J2EE, whatever.

Disadvantages of the distributed model:

  • More complex to the user. For consistency, things like chings! and votes would have to be managed per node and such. On different nodes you might have a different level and different attendant votes and chings. Messages wouldn't be global - they would pass between a few nodes, probably, or else there would inherently be more chatterboxes.
  • Fragmentation of E2. This is already the case, but this could exacerbate the problem. The possibility of not all nodes following the rules and such could also be a problem. This could be fixed by using the trust metric but might still be a problem.
  • More difficult to manage. Gods (unless they're granted permission on a per-site basis by the site hosts) can't just change stuff.

    So then E2 sez to me, they sez...

    I thought this might be interesting to the edev people, and perhaps others among us. Is it worthwhile to try to make E2 distributed? Is it feasible?

    I don't think it's possible or desirable at this stage to make ecore distributed. But could we make the application - Everything2 itself - live in more than one place at once?

    The benefits are tremendous, especially if we make it clever enough that people can extend it in different ways. Anyone could implement a method of having user data on their own node. Presumably the best method would win. Order could be enforced by the gods, of course... Fringe servers with little content could be used for weird dev stuff.

    The possibilities of this could be endless.

    What do you think?

    ponder says re On Making E2 Distributed: You ask "What do you think?" Where should people post replies? There is a danger of the node becoming a full blown GTKY discussion. BTW I want to reply.

    /msg s_alanet or e-mail me at <bgarney@purdue.edu>. My experience with edev nodes leads me to believe that the gods are not angered by what might be considered GTKY behaviour on them, but let's not tempt fate, k? Bad write ups, here as elsewhere, are sure to be blasted. E-mail/msg me first, then if it is warranted, add write up here.

    Thanks,

    s_alanet

  • After some review, I realized that I perceived this idea differently than it was actually written. It's still not the same as how I think it should work.

    Personal Opinion: This is too different from how usergroups work. Certain aspects, such as listing all things a user has listed in their homenode, would take a lot of work from a database standpoint, as I understand it, because it would require querying all known sets of data to find out if a user has listed anything. If the user wants people to know about these things from their homenode, they'll put them there -- that's how it used to be with the "foo everythingians" nodes.

    I'm so full of arrogant pride that I'm right (spawned by the fact I just woke up) that I'll even create a fake conversation to detail what I think is wrong with insanefuzzie's idea (which is a lot closer to what I want than one might think) and detail how I think things should be done.

    What was the purpose of the foo everythingians nodes, as sleeping wolf perceives it?
    To provide an index for users based on qualities they had, and to allow looking for commonalities.

    Okay, but insanefuzzie's idea does that.
    I don't think so. Let's take a theoretical example that never existed: Pagan Everythingians. As a node, it would be reasonably safe to say it might contain: (apologies to those whose names I use if I get anything wrong)

    Now, how would one go and use a search field to find Pagans in general if each of us popped exactly that in there?

    It's real simple. You tell users to semi-nodespace themselves, so that you would be Pagan: Animal Totemist.
    You really can't trust users to do such things properly. If you don't believe me, go to eBay and do a search for Plam. Look at all the Plam Pilots! Most users will fill out the field once and never check if they even spelled it right.

    Okay, then we restrict everything in advance.
    Not only would this require more code, but who comes up with the categories? One of the neatest things I've learned about Library Science is that it admits the futility of outright categorization. Above, sexuality is suggested as being restricted to 'straight' / 'gay' / 'lesbian' / 'bi' /'undecided'. Aside from the fact that this attempts to pigeonhole bisexuals (we're not all 50/50; there's your Kinsey 2s and 4s, your cyclics, those who have sapiosexuality), as well as the other groups (is being a 'vanilla' gay male the same as being a bear?), this also doesn't fulfill the "out" everythingians node — it tells us who is comfortable with the self-awareness they have.

    If you're so smart, how would you do it?
    As a basis:

    • take usergroups — a list of users with something in common.
    • Add a field, 255 characters of text for each user, linkable as per the homenode fields at the top of homenodes, to specify what about them makes them part of this group. This way in addition to listing characteristics it can list things like 'gay age' and such.
    • Allow users to add and remove themselves from the group via the group page, in addition to the group creator being able to add/remove/edit users.
    • Drop the /msg alias; if the group creator can add, it could turn into SPAM/flamewar/whatnot very, very quickly.
    • Retain the space at the top to put some text, but restrict it to links and such, since I think it currently supports superdoc functionality.
    • Creation of these 'user tag groups' would be a Level 5 power (because in mind it complements room creation rather well), allowing users to self-categorize.
    • Obviously, no experience would be granted for their creation.
    • Don't bother working at making it homenode-accessible; the whole idea, IMNSHO, is to go and provide indexes to users, not vice-versa.
    • Oh, and sort the list of users by username for fast lookup. No one wants to look at users sorted by date of addition to the list or order that they joined E2.

    Is my idea perfect? No. It doesn't work for birthdays (though it does work very well for addresses). It doesn't autoprovide links from homenodes, though I feel the structured part of homenodes is already overdone as-is -- imagine the homenode load time if it had to sift through all 'userlists' or 'usertags' (as I call them to myself) to find out what users had what to say about themselves.

    This is a proposal to add, essentially, a new dbtable (a database table belonging to an ecore, for those non-ecorey people) to the site: one containing user data, most of which would normally belong on homenodes, and make it universally searchable/viewable. This would end the need for, for instance, EMAR, The Everything People Registry, "out" everythingians, etc, and would also make it possible for users to edit their entries in these themselves, rather than taking up someone elses time, as well of course fixing the (IMO) main problem with moving this sort of stuff out of the nodegel - the loss of its easy viewing/searching.

    I've discussed this with several of you individually, as well as in #everything and the chatterbox, and the general consensus is that it's a good idea. I'd love to hear further comments: I'll add any blabbed /msgs I receive to the end of this node, and/or please discuss in edev or by adding a writeup. No-one I've talked to so far has actually expressed an opinion that this is a little GTKY-like, which it is, I suppose, but it will end up *removing* GTKY-type nodes from the nodegel, and will stop people complaining about the loss of a public "out" everythingians/other "blah" everythingians node.

    Essentially the dbtable would have three columns: 'user_id', which would be the node_id of the user, 'category', the name of a category, and 'data', the actual data for that category/user. For instance, 'category' might contain 'location' for a particular row, and for me the 'data' field would contain 'Southwick, Wiltshire, England'. Okay, this is a sorta yucky design, no unique column, kinda against the whole idea of a database, but it'd be silly to restrict us to arbitary columns like a normal table structure would have to do. An alternative might be to make category a mere numeric lookup from another table, this would be a little more complicated but probably better, especially if the level power I discuss below were to be implemented. N-Wing says vote and messageignore (db)tables don't have a primary key column.

    The category name and data fields would probably be restricted to the 'standard' 255 characters, although the prior should be perhaps be further (artificially) restricted to 50 or so. As CzarKhan says, "It is after-all for basic information and not your life story."

    You may wish to take my babbling about databases as just a way to show the design, rather than a way to actually implement the thing. I think we can actually just store this in a hash or something? I admit to having no idea about how that works (I'll work on that after I wake up), but just adding this to the 'user' dbtable would be nice, or even still having a separate table but something indexed by a unique user_id key, for (obvious) performance reasons. (This paragraph was added during period of extreme tiredness. It wouldn't work because searching/viewing in the way I describe wouldn't be (realistically) possible.)

    Searching/viewing and editing would be performed via two (or perhaps just one) superdocs. The editing superdoc would be similar to the way everything settings are entered, perhaps? Anyway, perhaps all predefined fields could already be on that page, ready to enter, complete with comboboxes for the ones with limited values, and while new values would be added/removes via a system similar to everything settings, and fields without any values in them at all would be simply ignored (a 'None' option could be added to combo boxes for this).

    The searching/viewing superdoc could allow people to view all entries of a certain category (sorted by data, or user name, or length of being a user, or whatever), and also to search for specific values inside that category (searching for 'England' in 'location', or 'bi' inside the 'sexuality' field, for instance). Categories should be selectable via a combobox.

    You would also modify the user display pages so that if a user had any categories attached to them then it would be displayed on their homenode; obviously this would be a user setting, which could also specify where the fields were to be displayed. I'd suggest being able to show them before the doctext, after the doctext and after the bookmarks, in some sort of table, and also inside the user details table in some sort of compressed form. There should probably also be a user option to specify whether their database fields should be displayed on their homenode at all, although I'm not sure if this is worth the effort or even sensible. Also, you could provide mini-links to search for other people with the same category existing, or with the same entry under that category, again being an optional user setting for these to appear.

    I'd suggest that there be a node under everything settings (only visible and editable by gods) which defines predefined fields and, in some cases, limits available values. brainwave suggests it would be a good idea for some sort of guidelines to be put in place for when this is to be changed (Will it be done after a certain number of requests are made?). Suggested fields, which obviously are just the opinion of me and a couple of other noders and should be defined by the admins:

    • 'location', freeform, but preferably in '[optional place ,] [state/county ,] country' form (perhaps the latter portion of this could be combobox based or something, maybe involving optional javascript, to try and keep it in the correct form), to replace The Everything People Registry. In fact we could also split it up into multiple edit boxes to keep it consistant.

    • 'address', freeform, to replace EMAR.

    • 'gender', restricted to 'male'/'female'/'other' or something similar. The consensus among the people I've talked to seems to be that allowing this field to be freeform wouldn't be a good idea, your opinion may differ. Maybe rename the title 'physical gender'/'sex' or something, although IMHO that's just silly, we all know what it means.

      Starke says Instead of having a bunch of angry transexuals and such, I suggest you have "I was born with a penis" or "I was born with a vagina"

      blondino says Gender: I consider myself male/female/neutral and I'd rather not say. (the "I'd rather not say" element shouldn't be an option, it should simply be defined by leaving the entry blank, surely? - fuzzie)

    • 'religion', freeform. Or alternatively we could make this a combined options/freeform selection (see first paragraph after this list) with the options being a large list of religion "categories".

    • 'sexuality', restricted to 'straight'/'gay'/'bi'/'lesbian'/'undecided'.

      brainwave says In the sexuality combo box I would like to see asexual as an option. You may think this is a frivilous request, but I think it is a helpful option for priests, nuns and others who are chaste for various reasons (although they may arguiably have another sexuality) as well as those who just plain prefer not to have sex.

      PLEASE READ THE PARAGRAPH AFTER THIS LIST BEFORE COMMENTING FURTHER ON THE RESTRICTED NUMBER OF CHOICES PROBLEM!

    • 'homepage', a URL, perhaps automatically placed into a hyperlink.

    • 'picture', a URL, also perhaps automatically placed into a hyperlink and/or img tag.

    • 'email', an e-mail address, again with the automatic hyperlink thing. (Thanks to The Librarian for suggesting this). We could automatically anti-spam this, perhaps randomly.

    • Perhaps a generic 'interests' field for a command-delimited set of interests. The same for 'hobbies', too, perhaps. I'm not sure about these two; some thinking needs to be done. mkb notes that this would work in a very similar way to LiveJournal.

    • 'aim', 'msn', 'icq', 'yahoo', 'jabber', for userids of people using the 'major' IM programs. We could check at least jabber, msn and icq for some sort of semi-validity, the others are pretty much freeform as far as I know. Also possibly 'irc' for their IRC nick. In most cases it's possible to have some kind of 'online' graphic or 'add me' link associated with these, perhaps we could have these where possible, obviously it would have to be a user pref whether to display them or not.

    • 'livejournal' for their LiveJournal id, which we could automatically link over there, seeming as though many people have moved their daylogging/daylogs over there.

    • 'birthday', which obviously has to be a date (we could validate this input to be in the correct form, a popup javascript calendar would be easy to code). This replaces who to send gifts to, and when.

    • wertperch says There's also MBTI personailty types, SparkMatch type, possibly others of that ilk.

      We could assimilate MBTI types of E2 members, for instance.

    • We could replace the E2 Family Tree with a superdoc constructed on the basis of an 'introduced by' category, perhaps.

    • CzarKhan says had you considered a freeform ethnicity portion?

    Specifics, such as details about sexuality, could be placed under 'sexuality details' or some other similar category name ... this should probably be bound into the edit page, so that a textbox for entering details is next to any combobox which might exist, to try and enforce naming conventions (note that you could fill in details while not choosing an option from the combobox). We could also bind the search page up like this (if the category is sexuality, you can search for a certain sexuality or 'sexuality details' keywords or both (which should be the default)). This would basically make the options just recommended options, which would be easier for most people, but would not make them mandatory, as they could fill in only the details if required, and most searches would still work.

    Or we could just scrap the whole combobox/choice system. sabby says Any box where you have to ponder wording, it's probably easiest to let be plaintext. If they don't fit into a generic "easy" choice, then they are going to want to explain. (Asexual preists, for instance, would most likely call themselves hetereosexual but not practicing, for instance. heh.)

    Things such as age, eye colour, height, etc, belong on homenodes; there would be nothing much useful gained by adding them to this list. However there is a suggestion by Oolong that perhaps anonymous statistics based on this might serve to help us work out such things as the average age of our userbase. So perhaps 'age' could be a category, with a checkbox making it anonymous if wished.

    Remember, people wouldn't have to fill in any of these fields at all, certainly not all of them, so if they didn't like the restricted options available in some cases, it's their choice whether to have that field at all (or indeed whether to have any field at all). One thing people have commented on the lack of is some sort of 'transsexual' option under 'gender': I personally don't think it belongs there, a separate field would be fine for that, and other people agree with that.

    Incidentally, note that we could easily change existing superdocs/create new superdocs to access this data in a nicer way, for instance a tree-like E2 Family Tree-like view for location, possibly entitled "The Everything People Registry" to maintain links which point there.


    I would be happy to finish the design of, and implement, this system, if that would be useful to the admins; I have been reviewing and discussing performance implications of this and the such. The other option would be to implement this off-site, which I don't think is an acceptable solution, and neither does anyone I've talked to about this, because it doesn't really help to resolve any of the issues involved, other than site load, again, IMHO.

    Another issue which I'd like to state my opinion on is the possibility of some of this being only available to level powers: The one case in which I think this might be a good idea is in the addition of *new* custom categories only being available to higher-level noders. This wouldn't restrict people too much - they could ask a sufficiently high-level noder to temporarily add a category onto themselves, thereby allowing someone to add the category themselves (the original high-level noder could then remove the category, and others could still add it due to it not being a new, unused category any more), and it would help to stop the addition of frivolous categories, which I feel would soon become confusing.

    Later note: Also, possibly the automatic linking of the 'homepage' and/or 'picture' categories from homepages could be restricted to higher-level noders (level 2 or above, say) or after verification from editors/much-higher-level noders or something. This would also cut down on abuse.


    (edev) sleeping wolf says That's one approach. The big problem is that people will want fields ad nauseam. Since, though, insanefuzzie has written up eir approach, now I a fire lit under me to write up mine. (I wanted to wait until I pseudo coded it)
    (edev) insanefuzzie says The ability to add whatever fields people want (preferably limited by if you can get a higher-level user to agree) is part of the whole point of my idea... something flexible which everyone can use for whatever they want.

    sleeping wolf feels some kind of nodetype would be best, and that everyone would want to look at things through the specific category, rather than viewing on homenodes. I personally feel that a database would be better, and that people would like to search both ways. Sie also seems to forget that, as the entries would be open to other users, spelling mistakes and the such would probably be noticed (and hence corrected, I'd hope) very quickly.

    More to come here as soon as I have managed to retrieve sleeping wolf's e-mail from my currently defunct machine.

    Mozilla has a nifty feature to add searching abilities to the sidebar/location bar. This will, at least in windows, but it "should" work in all OS's supported by Mozilla, give you an "E2" search. I don't know yet how to get it to keep the results in the sidebar, but such is life, i never use the sidebar. I would also like to patch Findings... to add in some easy start/stop tags.

    I need a 16x16 GIF for the search results etc that show up in the taskbar. Something just like the favicon.ico thing, only in gif format. It would also be nice to be able to implement the update features, although i'm not quite sure how they work, maybe a fullpage node or a regular file (gasp!) If we made it a fullpage superdoc (does such a beast exist?) it could be made modular to any ecore site. I'll talk to nate or jay about it...

    To install... copy the following src into a file called e2.src in your mozilla home directory, under Mozilla\searchplugins\
    On my machine this translates to c:\mozilla.org\Mozilla\searchplugins\e2.src


    #everything2.com search plugin by hawk@hawknetworks.com (Eraser_/E2)

    <search
    name="e2"
    description="Everything2.com Search"
    method="GET"
    action="http://www.everything2.com/index.pl"
    >

    <input name="node" user>
    <input name="searchy">
    <input name="type" value="oppressor_superdoc">
    <input name="type" value="superdocnolinks">
    <input name="type" value="superdoc">
    <input name="type" value="collaboration">
    <input name="type" value="collaboration ">
    <input name="type" value="restricted_superdoc">
    <input name="type" value="e2node">
    <input name="type" value="document">
    <input name="type" value="user">
    <input name="lastnode_id" value="124">

    <interpret
    browserResultType="result"
    resultListStart="the stuff we found"
    resultListEnd="<form"
    resultItemStart="<li>"
    resultItemEnd="</li>"
    >
    </search>
    <BROWSER
    update="http://www.everything2.com/e2.src"
    updateIcon="http://www.everything2.com/e2.gif"
    updateCheckDays="3"
    >
    So I recently whapped the edev list with a link to my recent XSL work, via a proxy widget at W3C that marries XSL transformations to XML files that don't specifically ask for them. My stylesheet, the output of which is available at <http://www.w3.org/2000/06/webdata/xslt? xslfile=http://www.gibberish.com/hacks/e2/catboxstyle.xsl&xmlfile=http://www.everything2.com/index.pl? node_id=1259846&transform=Submit>, takes the Universal Message XML Ticker and renders it into HTML on the client side. (Or, well, at the moment it renders it on the server side, but at least it's not our server. Whether anything but the W3C's widget can actually handle the advanced XPath fu in this stylesheet remains to be seen.)

    What's the point of that, you ask? Well, I'm no longer totally sure, but I can tell you what the point was when I started.

    I've heard much talk about the strain on the database and server when people sit on an expensive page and chat for extended periods. And really, when you consider that the user in question just wants to reload the chatterbox, all pages are expensive. There's no reason, technical or otherwise, to send the whole page over again just to refresh chat. The challenge, then, was to find a way to refresh one without the other.

    My idea was to use the existing chatterbox XML ticker to render the chatterbox on the client side, in the browsers that support client-side XSL. The resulting chatterbox simulacrum would then be targeted to an <iframe> tag, which sits in the browser like an image but can hold a separate HTML document like a frame. The chat form could then simply be targeted to a new ACTION attribute that only returned XML, and targeted the <iframe>. Users on non-XML-savvy browsers would simply get the old chatterbox. I even have some JavaScript that can dynamically resize an <iframe> so it scales to its content - no ugly scrollbars!

    But - and the worst thing is that it took me until I got the XSL working to realize this - XML is optional to this whole scenario. It wouldn't cost the server more to create a tiny static-HTML page of the current catbox than it does to create a similar XML page. Such an HTML page could then get stuck into an <iframe>, or even (God forbid) an <ilayer>, and get us a more stylish catbox even in browsers that don't deal with XML. If we're willing to put up with scrollbars, we could even lose the dependence on JavaScript.

    Here's the thing, though: is there a point? Would the elimination of the bandwidth inflation caused by chat reloading outweigh the fact that everyone who had the chatterbox turned on (and had a requisite browser) would be making TWO HTTP requests, rather than one, every time they loaded a page and weren't chatting? The savings to the database and computation time might tip the balance slightly in favor of an iframed chatterbox, but not by a whole lot... especially given all the people who just sit there looking at the catbox at any given time, and don't use it.

    So, in the end, I may have produced a very cool useless hack and nothing more. It'll be worth it, though, if I can open the possibility of in-browser, client-side rendering up for someone who's working on a problem that really does need it. I would certainly be willing to lend my expertise to any such project.