A Modest Proposal: Gnutella and the Tragedy of the Commons


Let me state at the outset, that I was never a user of Napster, Gnutella or any other software that shares music or other media files. In fact, as far as I know, I have never played an MP3 compressed file. I'm interested in Gnutella and similar software for three reasons:

  1. Gnutella is an interesting distributed networking application.
  2. Watching the entertainment industry act in a stupid ways, as they have done by suppressing Napster, is entertaining.
  3. I hate censorship. I do not believe that the written word should ever be censored. Distributed file sharing systems provide a way to distribute controversal written material, in addition to music files.

I'm a software engineer so I am involved in the creation intellectual property. I strongly believe that software engineers, authors, musicians and other "content creators" should be rewarded for their work. Having worked in the computer industry, with its constant state of revolution, for twenty years I also know that we must change with the times. This is exactly what the entertainment industry in general refuses to do. The Djinn has been let out of the bottle. Nothing will put him back in. Only someone who is stupid or who has taken too many drugs would believe that suppressing Napster is a victory. They have simply assured that software like Gnutella will develop more rapidly. In fact, as I write this I have no doubt that this will be the beginning of a burst of Gnutella like software development.

A Modest Proposal

There are a number of problems with the kind of distributed sharing supported by software like Gnutella. Some of these problems are technical. Many people do not have high bandwidth connections to the Internet (I live in rural New Mexico, for example). If Gnutella users are limited to connections of, effectively, 40K bits/second, there is a limit to the amount of music that can be exchanged. High speed internet connections have not developed as fast as many hoped. Over time, however, what we currently regard as high speed connections to the Internet will become more common.

One of the biggest barriers to software like Gnutella being successful for distributing music is human nature. Specifically, the problem that is sometimes referred to as "the tragedy of the commons". There is an excellent article in Salon by Janelle Brown, titled The Gnutella paradox which discusses the Gnutella's problems as use grows. As noted above, some of these problems have technical solutions. But one of the biggest problems is that there are many users who down-load using Gnutella without hosting media files themselves. This places a large network load on the few sites that do share media. Ms. Brown cites an article by Eytan Adar and Bernardo A. Huberman at Xerox PARC, Free Riding on Gnutella. This article studies Gnutella users and shows that many people take advantage of the common resource without giving anything back. This seems to be a sad and unchangable feature of human nature.

One solution to this problem might be to re-engineer Gnutella (or similar software) so that supporting down-loads gives the provider credits which can be used to down-load music from other sites. Free riders would be greatly reduced. Many people might provide down-loads for the same currently popular music. However, in a distributed system like Gnutella, this simply makes the system more robust. Although people would still not be purchasing as much music as the entertainment industry would like, people would have to purchase music to support their local music base. The amount of "down-load" credit that a user had amassed would be kept secret, to avoid music industry software searchers that attempt to find and sue those with large amounts of down-load credit (and so lots of down-load support).

An Arms Race

Rather than realizing that they must find ways to profit in a changing technological environment, the entertainment industry may continue its course of attempting to suppress free distribution of copyrighted material. Taking this course to absurd lengths we would arrive at the Noir world of K.W. Jeter where people are executed for copyright infringement. Short of this, there are technological weapons that the entertainment industry could use.

Those who work on the open source software for distributed sharing could be sued. While they can argue in their defense that the software can be and is used for applications other than copyright infringement, the prospect of defending against such a well funded adversary would intimidate many people. However, there are groups like the Electronic Frontiers Foundation which might provide aid.

There are also technological weapons that could be used by "the industry". For example, network traffic could be monitored to find sites that supply lots of down-loads. Those sites would then have civil or criminal charges brought against them in a very public fashion in an attempt to intimidate other users. The effectiveness of this would be reduced if all Gnutella users also provided down-loads, but some sites would still be considerably more active than others. The possible backlash of a large industry attacking a bunch of teenage and twenty-something copyright infringers, who also happen to be their best customers, might be counter productive.

The entertainment industry will lose any technological war. They are not greatly liked by either their customers or those who create content (both groups feel exploited). There are a lot of people who are motivated to evolve the distributed sharing software base. The music industry does not have the money or the talent to stay ahead of an army of dedicated software hackers. In they end, they are doomed to lose any campaign of suppression.

Blindingly Fast Evolution

The Web has made communication orders of magnitude faster. Rather than requestion a paper (assuming one even knew about it), it can simply be looked up on the Web. This has increased software evolution to the point where it has become difficult or impossible to have even a surface familiarity with all of the open source software projects. One of these is freenet. Freenet is a second generation distributed file sharing system. As with Gnutella, the intent is to create a distributed system of anonymous data. According to the documentation, freenet has better performance and scalability than Gnutella. From what I have read freenet does not currently solve the free rider problem that also exists with Gnutella, although it does seem to distribute frequently requested data through the network.

Freedom and Suppression of Information

A friend of mine spent most of his life in the Ukraine when it was part of that great socialist paradise, the USSR (Soviet Union). My friend is also a computer scientist. His software group was part of a large factory that made computer equipment, based on DEC computers. The factory had local KGB (USSR secret police) people on site. He told me that on holidays the KGB people would take the platens out of the typewriters, making them unusable (for those of you who have never seen a typewriter, the platen is the roller in a typewriter that holds the paper and provides the backing against which the keys strike). This was done to stop people from using the typewriters in an unauthorized fashion - like copying manuscripts by banned writers.

The Soviet Union went to a great deal of trouble to control the flow of information. Although things had losened up some during my friend's time in the USSR, there was a time when people where shot or sent to the Gulag (prison camps) for possessing banned information. Even with these draconian measures, "subversive" information could not be entirely suppressed. For example, Mikhail Bulgakov wrote his impressionist satirical work The Master and Margarita during the rule of Stalin, when many intellectuals like Bulgakov where shot. Even the decimation of Soviet intellectuals and the threat of death could not entirely stop intellectual discourse.

Although the flow of information could not be stopped, even by draconian methods, the dire sanctions employed by the Soviet Union did have a significant effect. In theory a government could apply these techniques to suppress a system like freenet. As in the case of the Soviet Union, these techniques would not be entirely successful, but they would have an effect. Of course any nation that took such an approach would also suffer severe economic consequences, as the Soviet Union did.

Short of totalitarian suppression, distributed information systems like freenet pretty much guarantee that information will spread in a way that cannot be controlled by the government. The bizarre writings of L. Ron Hubbard and the "church" of Scientology will be distributed and there will be no one for the "church" to sue. A future version of the Pentagon Papers could be published anonymously and could never be suppressed. This also means that distasteful information, like child porn, bomb making instructions and guides to poisons will also be available. Nor will those who own intellectual property be able to control it once it reaches a network like freenet. This is the new world, like it or not.

Mojo Nation

After I wrote the first version of this web page I found out about the Mojo Nation file sharing system. Mojo Nation is an open source project. Mojo Nation attempts to address the issue of free riding by a majority of users by implementing a "karma points" system. Users that provide resources like bandwidth and disk storage to the system are awarded "Mojo" (karma points). The Mojo Nation protocol apparently encapsulates some information about a user's Mojo balance which can move users with high Mojo to the top of a request queue. Another interesting feature of Mojo nation is the use of a number of possibly low bandwith (e.g., dial-up) servers to provide a data feed. Mojo Nation calls this "swarm distribution":

Most peer-to-peer content delivery relies on a single peer sending a requested file upstream. If that peer is overloaded, the requestor is probably out of luck. Mojo Nation breaks each uploaded file into small pieces, then replicates each small piece in several places over the network. When a user requests a file, Mojo Nation contacts a swarm of peers -- rather than just one -- before reassembling the file for delivery. This collaborative delivery method even enables users with dialup connections to participate -- a dialup user cannot reasonably deliver a 4GB file to another peer, but can certainly contribute a 64K piece of it.

Section 1.2, What is "swarm distribution"?, Mojo Nation FAQ

This reduces the load on a single low bandwidth server, but it does not supply the data any faster. By distributing the load in this way, it makes users more willing to be servers, since the load on their system and internet connection should not be excessive.

Like many Internet company models, peer-to-peer file sharing networks don't provide much in the way of a profit model. This, coupled with the constant attacks of the RIAA have forced out many of the post-Napster companies, although the open source peer-to-peer community still remains strong. Mojo Nation seems to have gotten out of the explicit file sharing network business. They now offer a product called HiveCache, which uses "swarm" techniques for distributed file backup.

Anonymity and Markets

"We're told that by saying, 'You can't steal our product,' we're impeding technology," said Peter T. Paterno, a lawyer who represents Dr. Dre, Metallica and others. "As a country, we import everything, and one of the few things that we export -- entertainment -- Congress wants to give away to some technology company. I'm so fed up with that mentality. If I was running a record company, as opposed to the wimps that are running one, I'd say, "you know what, I have no interest in compromising, and I'm going to go sue little Johnny who's downloading this stuff.'"
Behind the Grammys, Revolt in the Industry, by Neil Strauss, The New York Times, Week in Review, Pg. 3, February 24, 2002

There are a number of driving forces behind the peer-to-peer networks, especially now that Napster has been forced off line by copyright litigation. The strongest of these forces is the desire of users of these networks to access music without paying for it. As the demise of Napster and the legal problems of mp3.com have shown, the recording industry has aggressively attacked music sharing (or pirating, depending on your view) networks. The only way to defeat the attacks of the recording industry is to provide anonnymity and a widely distributed system. When copyrighted works are widely distributed and the users and servers are difficult or impossible to track down, the recording industry will have no one to sue. Or perhaps as the quote above suggests, their only course will be to sue all of their customers.

After the desire to access music without fee, the second strongest force is information anarchy: the ability to place information on a distributed system in such a way that the servers and readers cannot be traced. And in such a way that as long as the information is of interest, there is no practical way to remove it. This is the ultimate expression of "information wants to be free". Once such a network reaches a size where it crosses national boundaries, there is no practical way for a goverment to suppress information placed on such a network. In such a system there would be no control over information, for good and for bad. Accounts of government corruption, recently issued novels, and classified documents and child pornography might all reside on such a network.

A network that supports anonymity makes support for a Mojo Nation style market difficult. An anonymous system means that it will be more difficult to avoid "free riders". Any "karma" accounting system is a potential security hole from the point of view of anonymity. On the other hand, the lack of anonymous support for sharing copyrighted works means that Mojo Nation, which discourages free riding, is unlikely to attract a large user base.

Attempting to Suppress Anarchy

No large publishing system supporting information anarchy has ever existed. Throughout history governments have had the ability to control and suppress information. In the case of information that most people regard as horrifying and offensive, like child pornography, there is a broad concensus that some kind of control is needed. By definition, control is entirely absent from a network that supports information anarchy. Governments will attempt to suppress such networks, since at least some of the information in them will be illegal. The presence of information like child pornography could be used by a government as justification for extreme measures to suppress the information anarchy network.

There are several ways a government could attack a network supporting information anarchy, including overt and covert methods. Most of the peer-to-peer information sharing systems are open source projects. A government could attack the source code base legally (e.g., the CVS repository). If the source base could be removed from the Internet, the network supporting information anarchy would die. In the United States this might be legally difficult, because the courts have not been willing to support the suppression of a reproduction medium that can be used for both legal and illegal purposes. In the United States a lawyer could argue that banning an information anarchy network makes as much sense as banning photography, which is also used for illegal purposes like producing child pornography. Even if the local legal system were willing to support an attack on the source base, it would be futile if the source base were mirrored in another country.

Covert method might include introducing servers into the system as trojan horses. Since the source base is available for download and is likely to be large, complex, and sparsely documented, a government agency could introduce tracking code into the servers. Even the authors of the software might not find such code if it were carefully crafted. Users who unknowingly downloaded these Trojan horse servers would introduce traking nodes into the network that might allow illegal information to be tracked to users.

Annotated References

Technical/Computer Science References

Both the Gnutella and the freenet sites include documentation. I've also started to gather a few random references here. This is by no means an attempt at a complete bibliography on open source peer-to-peer networking software and protocols.

On-line press references

Ian Kaplan
Created: February 2001
Revised: August 2004

back to Miscellaneous Technical Topics

back to home page