[gridengine users] hedeby install howto?

Chris Dagdigian dag at sonsorol.org
Wed Jun 29 22:05:07 UTC 2011

My overall advice for people trying to run Grid Engine on the Amazon 
Cloud is this:

(1) If you just want to run Grid Engine in standalone mode on the Amazon 
Cloud then you should be using StarCluster 
http://web.mit.edu/stardev/cluster/ -- those folks made a fantastic and 
free system for elastic SGE clusters on the cloud with working shared 
filesystem, MPI configured etc. etc. It's freaking magical. They also 
track new AWS features closely and develop support for cool things that 
most people would not have time to implement themselves on a small 
project -- such as support for running under Spot Instances and within 
the odd networking sandbox of the new VPC environments. My company tends 
to be the one that runs into StarCluster limitations (such as 
inconsistent support for running inside a special VPC zone where a HTTP 
proxy was required for internet access) and the developers have been 
responsive, friendly and overall nice to interact with.

(2) If you *really* want to do the hybrid cloudbursting thing than I'd 
simply say go talk to the nice folks  at Univa.com - they already have 
far more up to date methods, software and (most importantly) happy 
customers using their cloud bursting stuff. This would be the modern and 
sustainable route.

(3) If you really don't want to use Univa and you want to bridge a local 
SGE cluster into the Amazon cloud than just skip the Hedeby 
overhead/complexity and just use Amazon VPC to link your local and 
remote subnets together. After that its just one big happy SGE cluster 
with a boatload of network/bandwidth limitations affecting some nodes 
more than others.

Hedeby was part of a massive project for "resource aware" stuff within 
Sun. SGE was just a tiny part. In my mind it has little use or utility 
in the context of open source software unless you are going all-in on 
all the other hedeby features. Using Hedeby today just to get SGE cloud 
adaptors is going to be a stressful exercise involving complex 
end-of-life software that is effectively a developmental dead end.

My $.02


Dave Love wrote:
> Allan Tran<tran.v.allan at gmail.com>  writes:
>> I'm trying to install hedeby to integrate sge62u5 and amazon cloud but can't
>> seems to find any good step by step article. All the links seem to be broken
>> since Oracle kills the open source GE.
> Maybe Chris D will chime in with useful guidance.  However, as well as
> doc at http://wikis.sun.com/display/gridengine62u5/Home (currently
> down), you should be able to get everything that was on sunsource via
> https://arc.liv.ac.uk/trac/SGE.  The source is available via darcs, hg,
> and git, or directly under http://arc.liv.ac.uk/repos/darcs/hedeby/&c
> (which should work to display the original web pages).  Unlike the
> actual gridengine source, it hasn't had any love since it was stashed.
>> I found this
>> http://wiki.gridengine.info/wiki/index.php/SGE-Hedeby-And-Amazon-EC2#Installing_the_Grid_Engine_Hedeby_Service_Adapter
>> but
>> stuck at step Setup SDM Master (sdmadm, etc, where is this sdmadm from? I
>> know I'm missing a lot of packages but is there a single place I can
>> download them all.
>> What I have so far now is a fresh working SGE 62u5 installed with JVM
>> enabled.
> Presumably you need the source from the hedeby, hedeby-ge-adapter, and
> hedeby-cloud-adapter repos as above.  I can't remember how difficult it
> is to build.  There are instructions somewhere in the www directory of
> the repo if I recall correctly.

More information about the users mailing list