usenet

Given that the web would eventually replace usenet for most purposes, it is ironic that Tim Berners-Lee posted his so-called "world wide web" on a usenet newsgroup to see if anybody was interested. Does anybody know which newsgroup? Was he flamed for posting off-topic?

Update: WolfKeeper tells me that Berners-Lee first posted about the World Wide Web in alt.hypertext on the 6th August 1991, and further that it doesn't look like he was flamed, given that the post was on-topic for that group. The thread can be viewed at groups.google.com, which carries an extensive archive of usenet postings.

Articles or messages are posted to these newsgroups by people on computers with the appropriate software -- these articles are then broadcast to other interconnected computer systems via a wide variety of networks. The articles generally expires in a few days, but now there are a number of usenet archives that keep the messages for posterity.

Some newsgroups are moderated; in these newsgroups, the articles are first sent to a moderator for approval before appearing in the newsgroup.

Usenet is available on a wide variety of computer systems and networks, but the bulk of modern Usenet traffic is transported over either the Internet or UUCP.

after http://www.faqs.org/usenet/
http://www.faqs.org/faqs/usenet/what-is/part1/

Usenet /yoos'net/ or /yooz'net/ n.

[from `Users' Network'; the original spelling was USENET, but the mixed-case form is now widely preferred] A distributed bboard (bulletin board) system supported mainly by Unix machines. Originally implemented in 1979-1980 by Steve Bellovin, Jim Ellis, Tom Truscott, and Steve Daniel at Duke University, it has swiftly grown to become international in scope and is now probably the largest decentralized information utility in existence. As of early 1996, it hosts over 10,000 newsgroups and an average of over 500 megabytes (the equivalent of several thousand paper pages) of new technical articles, news, discussion, chatter, and flamage every day (and that leaves out the graphics...).

By the year the Internet hit the mainstream (1994) the original UUCP transport for Usenet was fading out of use (see UUCPNET) - almost all Usenet connections were over Internet links. A lot of newbies and journalists began to refer to "Internet newsgroups" as though Usenet was and always had been just another Internet service. This ignorance greatly annoys experienced Usenetters.

--The Jargon File version 4.3.1, ed. ESR, autonoded by rescdsk.

Usenet (short for "User's Network") is a worldwide, distributed messaging system, used largely for discussion and exchange of binary data. Users download and post messages to-and-from a server, which communicates with other servers to propagate those messages around the world.

Messages are organized, by subject, into 'newsgroups'. Each Usenet server administrator has complete control over that server: what groups it makes available to it's users, what groups it propagates, how long messages stay on the system before 'expiring', who can be a user etc. etc. The only clear-cut case of ownership or control of Usenet are those administrators; they can pretty much do as they please.

Usenet operates much like the Internet-at-large; there is no single governing body or owner with absolute control, but there are recognized systems and groups who keep the whole thing almost organized. Much of that organization is done via Usenet in the 'news' hierarchy; that is where lists of newsgroups are published, new groups created and announced and site administrators and other volunteers generally agree how things are going to run. Periodic postings to these groups are a large part of what keeps Usenet a cohesive system. Documents describing the systems for newsgroup creation, active groups, what is discussed in certain groups, who is responsible for certain actions and how to create groups are updated and posted on a regular basis. These documents form the 'official' view of Usenet only because they are widely propagated and followed.

A Brief History

In 1979, Jim Ellis and Tom Truscott, graduate students at Duke University, wanted to give universities the same kind of functionality that the ARPANET had with its mailing lists; they came up with a simple system to share files between UNIX systems using modem connections. Steve Bellovin, a University of North Carolina grad student, wrote the first version of the software as a few Bourne shell scripts, using the UUCP protocol.

The software periodically checked the timestamp of files in a specified directory, and sent files updated since the last check to the other system. The first two systems on Usenet were the UNC and Duke Computer Science departments. In 1980, the Duke Medical Center Department of Physiology was added.

Usenet was featured at the 1980 Usenix conference and the scripts were re-written as freely distributed C programs. Other Universities were invited to join the network, and as more nodes joined, the more useful and popular Usenet became.

Maintenance and development of the main software changed hands a few times and was constantly improved to keep up with increased demands. Since the whole system was open source and freely distributed, many other people began writing compatible server software and today server and reader software is available on many platforms including Linux and Windows.

News Propagation Algorithm

This process is described in RFC 850: Standard for Interchange of USENET Messages.

Usenet is generally described by a directed graph. Each node is a host and each link between hosts describes information flow to or from the designated host. Generally these links are bi-directional, but not necessarily so.

1. A user posts an article to a server on a list of newsgroups.
2. The server accepts it locally and then sends that article to it's neighbors. The neighboring servers will have notified the server as to which groups they are interested in receiving, and the server will honour those notifications.
3. Those neighbors will then receive the article locally and again send it out to their neighbors, and so on, resulting in the propagation of the message to the entire network.

Obviously it's important that there's a mechanism here to prevent loops, or else these articles would be propagated endlessly. Each host keeps track of what message it has seen according to a unique message ID. If an incoming message bears a previously seen message ID, the host will immediately drop that message.

RFC 850 also outlines a few additional optimizations (Section 5, page 17). Both optimizations involve checking the message header to see what hosts have left their mark in the 'path-line' or 'posting-version' (origination). If a neighbor is discovered to have already seen the message it will not be forwarded to it.

Usenet's Future

Usenet's discussion forum is a unique thing. It's still in wide use even though there are many distributed 'chat' and messaging systems that are very popular like IRC or ICQ. One possibility is that the Usenet discussion community is made up solely of old-timers and once they die off, so will Usenet. It's also possible that the Usenet forum provides something that other messaging systems don't. For instance, perceived weaknesses such as Usenet's lack of 'real-time' discussion add a different dimension to the conversation; since it's not real-time, statements are usually more thought out. The discussions are not constrained by time: it allows seamless communication between multiple people in multiple time zones in the same discussion. That's an application with longevity.

It's a system that has been evolving for 20 years, and has a flexibility limited only by the imagination of its users. Users have adapted the system for file sharing, and who says that file sharing is the last revolution to be introduced to the Usenet system?

Timeline

1979

North Carolina: Duke grad students Tom Truscott and Jim Ellis thought of an idea to hook computers together to share information with the Unix community.

Steve Bellovin, UNC grad student wrote the first version of Usenet. The shell scripts were installed on the first two sites: "unc" and "duke".

1980

Another machine at Duke, "phs" is added to the network.

Usenet was described at the January Usenix conference.

Steve Bellovin rewrote the scripts into C programs; Steve Daniel wrote another implementation in C for distribution.

Tom Truscott made further modifications, resulting in the "A" news release.

1981

U.C. Berkeley grad student Mark Horton and high school student Matt Glickman rewrote the software to add functionality and performance (the original version was only intended for a few articles per group per day) creating the "B" version.

1982

B News is publicly released. As the net grew the news software was expanded and modified.

RFC 822: Standard for the Format of ARPA Internet Text Messages, is published.

1983

RFC 850: Standard for Interchange of Usenet Messages is published

1984

Rick Adams from the Center for Seismic Studies took over development of the B News software.

Increasing volume of news was becoming a concern, and the mechanism for moderated groups was added to the software.

Brain Kantor (U.C. San Diego) and Phil Lapsley (Berkeley) develop the Network News Transfer Protocol (NNTP) between 1984 and 1985.

1986

A package was released implementing news transmission, posting, and reading using NNTP.

RFC 977: Network News Transfer Protocol, is published.

1987

C News is released by Geoff Collyer and Henry Spencer from the University of Toronto. This is a big-time rewrite improving many aspects such as speed and reliability.

RFC 1036: Standard for Interchange of Usenet Messages published.

1989

RFC 1123: Requirements for Internet Hosts -- Application and Support published.

1990

RFC 1153: Digest Message Format published.

1992

InterNetNews (INN), news software written for large UNIX hosts, is released by Rick Salz (UUNET).

Geoff Collyer (U of T) implemented News Overview (NOV)

Deja.com, started in 1995, was a free web based email and Usenet posting service and archive that also allowed users to read and post reviews of products and services in a proprietary, non-usenet database. recently, it was acquired by google, who still has the Usenet post database, but sold the proprietary product reviews portion. For several months, Google did not allow posting to Usenet; Newsgroups were Read only. Posting was restored on May 23, 2001. Usernames from the previous incarnation of Deja were not restored; Email users migrated to other free web based email sites. Minor impersonation of pre-google personalities ensued, but were quickly blown off.