Blog | Commerce.net

P2P Traffic Measurement

After our decentralized filesharing post we discovered this item from ACM News Service, “Is P2P Dying or Just Hiding?”

High-order bits excerpted from the paper itself:

In our traces, P2P traffic volume has not dropped since 2003. Our datasets are inconsistent with claims of significant P2P traffic decline.
We present a methodology for identifying P2P traffic originating from several different P2P protocols. Our heuristics exploit common conventions of P2P protocols, such as the packet format.
We illustrate that over the last few years, P2P applications evolved to use arbitrary ports for communication.
We claim that accurate measurements are bound to remain difficult since P2P users promptly switch to new more sophisticated protocols, e.g., BitTorrent.

More bits:

CAIDA monitors capture 44 bytes 2 of each
packet (see section III), which leaves 4 bytes of TCP packets to
be examined (TCP headers are typically 40 bytes for packets that
have no options). While our payload heuristics would be capable
of effectively identifying all P2P packets if the whole payload
was available, this 4-byte payload restriction limits the number
of heuristics that can undoubtedly pinpoint P2P flows. For example,
BitTorrent string “GET /torrents/” requires 15 bytes of
payload for complete matching. Our 4-byte view of “GET ”
could potentially indicate a non-P2P web HTTP request.

The ACM News Service summary…

“Is P2P Dying or Just Hiding?”
CAIDA.org (10/04); Karagiannis, Thomas; Broido, Andre; Brownlee, Nevil

UC Riverside’s Thomas Karagiannis, the Cooperative Association for Internet Data Analysis’ (CAIDA) Andrew Broido, et al. dispute popular media reports that peer-to-peer (P2P) file-sharing has declined precipitously in the last year, and contend that the reverse is actually the case. The authors attempted to measure P2P traffic at the link level more accurately by gauging traffic of all known popular P2P protocols, reverse engineering the protocols, and labeling distinctive payload strings. The results support the conclusion that 2004 P2P traffic is at least comparable to 2003 levels, while rigid adherence to conventional P2P traffic measurement techniques leads to miscalculations. The percentage of P2P traffic was found to have increased by about 5 percent relative to traffic volume. Furthermore, comparisons between older and current P2P clients revealed that the use of arbitrary port numbers was elective in older clients, while current clients randomize the port number upon installment without the need for user action. Meanwhile, P2P population studies found that the ranks of IPs grew by about 60,000 in the last year, and the number of ASes participating in P2P flows expanded by roughly 70 percent. These findings outline several trends, including evolving tension between P2P users and the entertainment sector; increasing demand for home broadband links; plans to directly induce P2P applications into profitable traffic configurations; and a significant transformation in supply and demand in edge and access networks, provided that P2P traffic maintains its growth and legal entanglements are eliminated.

The full article (pdf) is contains many more bits for the interested reader…

October 31, 2004/by ams

Amazon Lets Customers Paste Photos

Commerce

CNET refers to an online trend — “the blurring of e-commerce and personal media such as Web logs and social networking sites”:

Amazon.com has quietly introduced a new feature on its Web store that lets customers post photos alongside product reviews–its latest effort to build a sense of community among customers.

The e-tailer introduced the feature, called Customer Images, last month for certain product categories including electronics, apparel, sporting goods and musical instruments. It added kitchen items, tools and hardware on Tuesday. The feature is in beta, meaning the company is still testing and tuning it.

“This feature allows customers to really showcase how they are using the product,” Amazon spokesman Craig Berman said. “It’s a great addition to our customer experience.”

The idea is to let customers highlight specific attributes of a product, such as size, and show the product in action, he said.

The Web and Web-based Services are finally discovering how to personalize their content to their customers, and it’s thrilling to watch this trend unfold.

October 30, 2004/by ams

Smart Health

Health Care

Today CommerceNet’s Smart Health Portal was launched during the Smart Health Summit. The initiative:

Smart Valley, a non-profit committed to turning Silicon Valley into the leading user of the technologies it invents, and CommerceNet will work with the Smart Health Forum, a community of healthcare provider, insurers, employers and foundations in Silicon Valley. The Forum, which is open to all interested parties, will collaborate on the implementation of network-based tools that will greatly facilitate the sharing of medical records without compromising patient privacy, data security or provider choice.

The network will allow patients and providers to ‘publish’ an entry in a data registry when a healthcare transaction occurs, whether it’s a visit to a provider, filling a prescription or getting a test result. All of the patient’s data remains in the source systems – only a pointer is recorded. The next time there is a visit, whether to a doctor’s office or an emergency room, authorized caregivers will be able to see the transactions and call up x-ray images, test results and other information needed to provide rapid diagnosis and treatment. Similarly, consumers will be able to see and update their personal information and that of their children. Patients and providers will set the policy that determines who can see what data.

These network tools will be piloted in several different environments. Projects being considered include a senior- and chronic care in the emergency room and a ‘Virtual Clinic’ concept that illustrates how physicians and community-based caregivers can share information.

The pilots will be funded through a collection of public and private funds. CommerceNet is applying for state and federal funds, some of which will require matching funds from local organizations. We will work with community-based foundations to help us raise funds.

When proven, Smart Health will be made available to the rest of the Bay Area, California, and the nation.

October 29, 2004/by ams

WikiHealth

Health Care

Brian Dear has an intriguing idea:

Imagine WikiHealth or WikiMed, an open, collaborative health database written by everyone in the world? If millions of people could contribute articles on health and well-being, diseases, treatments, symptoms, remedies, and personal experience with what worked and what didn’t with prescriptions, would the world be better off? Are you currently happy with the state of medical knowledge on the web? If you or someone you know is suffering from some condition, and you type the name of the condition into Google, are you satisfied you’re getting good results? In an age where, at least in the U.S., doctors are less and less likely to give you the time of day let alone spend time with you going into detail about everything there is to know about a condition, wouldn’t it be useful if there were an online resource with a strict NPOV (neutral point of view) containing in-depth encyclopedia information about health-related subjects?

WikiHealth. WikiMed. (Don’t bother, the domains are taken — maybe there’s hope!) But you get the idea: a worldwide open collaborative compendium of practical health and medical knowledge. Isn’t it time such a service existed?

Yes. Yes it is. For another example, there’s a related idea floating around CommerceNet’s healthcare team that it would be interesting to drop the NPOV and offer parametrized search — imagine if, for more controversial complementary-medicine theories, patients/advocates could just submit their anecdotal stories, but then you could ask for stories “similar” to your own to see what’s out there. In any case, WikiMed could be quite personalized to your medical “ideology” and idiopathy…

[Kind of like how the CreativeCommons search engine is only a slight, but suggestive, parametric constraint on ordinary Nutch searches]

October 28, 2004/by ams

Plott’s Cool Experiment

Decentralization

Science News: Best guess: economists explore betting markets as prediction tools

The research that led to future-predicting markets stems from the 1960s and 1970s, when Vernon Smith and Charles Plott, now of George Mason University and the California Institute of Technology in Pasadena, respectively, began using laboratory experiments to study different market designs. In the early 1980s, Plott and Shyam Sunder, now of Yale University, tested how well markets aggregate information by designing a set of virtual markets in which they carefully controlled what information each trader had.

In one experiment, Plott and Sunder permitted about a dozen study participants to trade a security, telling them only that it was worth one of three possible amounts–say, $1, $3, or $8–depending on which number was picked by chance. Plott and Sunder then gave two of the participants inside information by telling them which amount had been selected. Traders couldn’t communicate with each other; they could only buy and sell on the market.

“The question was, Would the market as a whole learn what the informed people knew?” Plott says. “It turned out that it would happen lightning fast and very accurately. Everyone would watch the movements of the market price, and within seconds, everyone was acting as if they were insiders.”

In another experiment, Plott and Sunder gave the inside traders less-complete information. For instance, if the outcome of the random pick were $3, they would tell some traders that it was not $1, and others that it was not $8. In these cases, the market sometimes failed to figure out the true value of the security.

However, if Plott and Sunder created separate securities for each of the three possible outcomes of the random pick instead of using one security worth three possible amounts, the market in which some traders had incomplete tips succeeded in aggregating the information.

The studies established that, at least in these simple cases, markets indeed can pull together strands of information and that different setups affect how well they do so.

This type of experiment gave researchers a “wind tunnel” in which to test different market designs, says John Ledyard, a Cal-tech economist who chairs the board of Net Exchange. “With experiments, we’re starting to zero in on what really works,” he says.

October 27, 2004/by ams