Back on March 10th, Bernardo Huberman’s group’s work at HP showed up in the Economist’s Technology Quarterly as an example of internal markets at work. But what commodity should one make a market in?
As the article excerpt points out, electricity is sold by the kilowatt-hour, but computing consists of (at least) three distinct qualities: storage, bandwidth, and processing. Further complicating matters is that privacy and security constraints affect how we allocate each type of computing resource differently. So I don’t know how successful “computons” might be, but we need some handle on that concept to successfully decentralized ‘grid’ computing…
Economist.com: Who wants to buy a computon?
Building just such a market mechanism is the nut that Bernardo Huberman, a researcher at Hewlett-Packard (HP), and his team have been trying to crack. The key, Dr Huberman realised, was to have a system that can allow users to assign different priorities to tasks, to reflect their importance. This rules out any system that would simply give each user a priority without differentiating that user’s many tasks. It also rules out a reservation-style system of the sort that airlines use, since a lot of processor cycles (like aeroplane seats) would end up unused, and the system would not be able to accommodate new tasks as they arose, even if they were extremely urgent. In a grid, it must be assumed, demand is changing constantly and unpredictably, and so is supply (since individual host computers on the grid come and go).
Mr Huberman’s answer is Tycoon, a piece of software for computing grids that turns them into a sort of %u201Cstockmarket or clearing house%u201D, he says. Users start by opening a bank account and getting credits. They then open a screen that shows all the available processors, their current workloads, and a price list. Users place bids for various processors, using a sliding price dial that looks like a volume control. Allocation is proportional, so that if one user bids $2 and the other $1, the first gets two-thirds of the resource and the second one-third. If the deadline of one task suddenly moves forward, the user can up his bid and immediately get more processor cycles for that task. As users consume cycles, the software deducts credits from their account.
The HP team has so far tried out Tycoon on a cluster of 22 Linux servers distributed between HP’s headquarters in Palo Alto, California, and its offices in Bristol, England. Tycoon did well in these tests, and several amusing animated films were rendered using its system. HP has now given Tycoon to CERN, the world’s largest particle physics laboratory and a hotbed of grid-computing research, for more testing.
This is only the beginning, of course. Mr Huberman reckons that Tycoon, in its current form, could run clusters of 500 host computers with perhaps 24 simultaneous users. But the ultimate vision of grid computing is for one gigantic network spanning the globe and accommodating unlimited numbers of users. So a lot still needs to happen.
For a start, the metaphor for computing grids as %u201Cutilities%u201D, similar to water or electricity supplies, is misleading, since there is no equivalent of litres or kilowatt-hours. Processor cycles are just one component of computing resources, alongside memory, disk storage and bandwidth. Mr Huberman would like to combine all of these factors into one handy unit, which he wants to call a %u201Ccomputon%u201D (a cross between %u201Ccomputation%u201D and %u201Cphoton%u201D, the name for a packet of electromagnetic energy). Tycoon’s descendants would then help to allocate computons across the grid’s global market. Of course, Mr Huberman adds, that will happen in a different decade.
Tycoon allocates computer resources in distributed
clusters like PlanetLab, the Grid, or a Utility Data Center
(UDC) using a market-based mechanism where user pay for their
usage using a currency. Tycoon allocates the cluster more efficiently
than time-sharing schemes, and allows users to change their
allocation in seconds.