Thebes

This is a repost of posting that dropped off of GridsWatch:

As promised, I'm going to beat on the scalability drum regarding grid computing.

The reason *the* grid is not happening is because current tools are too complex (and getting worse) and not scalable. Tacking extensions onto existing toolkits is only exacerbating the problem, potentially delaying a solution.

But wait, what is this thing called *the* grid? The difference between *a* grid and *the* grid is similar to the difference between an intranet and the Internet. *A* grid arbitrarily limits one or more critical items, generally due to scalability issues in the underlying software stack. Typically, job submitters are strictly limited, and addition of a job submitter is a complex event. Job submitters must be intimately aware of the grid environment, often having to hand select a resource to run on. The issue of crossing administrative domains is artificially resolved by creating virtual organizations that span administrative domains. In *the* grid, job submitters should be able to log in, select a grid-enabled application, make environment decisions, and launch it from the desktop, without concern over where it will run. In order to attract resources, Resource Owners (RO's) should be provided an easy to install, easy to use, easy to administer tools to publish their resource to *the* grid and determine with great confidence what individuals, groups, or groups of groups can run jobs on their hardware, and which applications will be allowed.

Notice that I'm not saying that all job submitters should be able to run on all machines. Just as some websites require credentials and block unauthorized persons, RO's on the grid must be able to pick and choose who runs what on their resources in a secure and scalable manner. RO's should be able to decide at install time who can run. This system should include grids built from existing tools, in fact an existing grid should be viewed as a resource, and current grid administrations should be viewed as RO's. Stringing together existing grids is an important part of the solution.

Despite an incredible amount of global funding that's been given to existing big science grid projects, the most successful computational grids on the planet still all follow the original seti@home model. Other examples include IBM's World Community Grid , United Devices' Grid.org , and Stanford's Folding@home . A central portal makes it easy for RO's to install these products on their machine, and RO's feel protected because the portal guarantees that only a select few researchers will run one of a group of well tested applications. The security of the 'grid' is based upon the security of the portal, RO's trust anyone the portal tells them to. In fact, the existance of an actual human researcher is less important to the RO then how good the Portal's story is. When this model works, it works well, but it does not meet the needs of *the* grid.

However, the big science grids play a critical role in the current HPC world. The money spent on these project is critical. I hope that future development will expose secure and scalable methods for connecting these projects together.

We have some radical ideas in this arena. More to follow...

Arnie Miles