[gridengine users] hedeby install howto?

William Bryce bbryce at univa.com
Wed Jun 29 23:08:59 UTC 2011

Hi All, 

Just to add....

If you really are not comfortable creating HPC clusters in the cloud (many of you are technical so this likely does not apply) and you want management, monitoring of nodes, flexibility, ability to add more functionality to your HPC cluster in the cloud please check out our Integration with Rightscale.  Univa and Rightscale have created a 'really simple' HPC cluster with Univa Grid Engine managed by Rightscale.  It does cost $$ so not for everyone.  Check out the following video it describes how easy it is to do:  http://www.univa.com/resources/video/hpc_cloud_video.php



On 2011-06-29, at 6:05 PM, Chris Dagdigian wrote:

> My overall advice for people trying to run Grid Engine on the Amazon Cloud is this:
> (1) If you just want to run Grid Engine in standalone mode on the Amazon Cloud then you should be using StarCluster http://web.mit.edu/stardev/cluster/ -- those folks made a fantastic and free system for elastic SGE clusters on the cloud with working shared filesystem, MPI configured etc. etc. It's freaking magical. They also track new AWS features closely and develop support for cool things that most people would not have time to implement themselves on a small project -- such as support for running under Spot Instances and within the odd networking sandbox of the new VPC environments. My company tends to be the one that runs into StarCluster limitations (such as inconsistent support for running inside a special VPC zone where a HTTP proxy was required for internet access) and the developers have been responsive, friendly and overall nice to interact with.
> (2) If you *really* want to do the hybrid cloudbursting thing than I'd simply say go talk to the nice folks  at Univa.com - they already have far more up to date methods, software and (most importantly) happy customers using their cloud bursting stuff. This would be the modern and sustainable route.
> (3) If you really don't want to use Univa and you want to bridge a local SGE cluster into the Amazon cloud than just skip the Hedeby overhead/complexity and just use Amazon VPC to link your local and remote subnets together. After that its just one big happy SGE cluster with a boatload of network/bandwidth limitations affecting some nodes more than others.
> Hedeby was part of a massive project for "resource aware" stuff within Sun. SGE was just a tiny part. In my mind it has little use or utility in the context of open source software unless you are going all-in on all the other hedeby features. Using Hedeby today just to get SGE cloud adaptors is going to be a stressful exercise involving complex end-of-life software that is effectively a developmental dead end.
> My $.02
> -Chris
> Dave Love wrote:
>> Allan Tran<tran.v.allan at gmail.com>  writes:
>>> I'm trying to install hedeby to integrate sge62u5 and amazon cloud but can't
>>> seems to find any good step by step article. All the links seem to be broken
>>> since Oracle kills the open source GE.
>> Maybe Chris D will chime in with useful guidance.  However, as well as
>> doc at http://wikis.sun.com/display/gridengine62u5/Home (currently
>> down), you should be able to get everything that was on sunsource via
>> https://arc.liv.ac.uk/trac/SGE.  The source is available via darcs, hg,
>> and git, or directly under http://arc.liv.ac.uk/repos/darcs/hedeby/&c
>> (which should work to display the original web pages).  Unlike the
>> actual gridengine source, it hasn't had any love since it was stashed.
>>> I found this
>>> http://wiki.gridengine.info/wiki/index.php/SGE-Hedeby-And-Amazon-EC2#Installing_the_Grid_Engine_Hedeby_Service_Adapter
>>> but
>>> stuck at step Setup SDM Master (sdmadm, etc, where is this sdmadm from? I
>>> know I'm missing a lot of packages but is there a single place I can
>>> download them all.
>>> What I have so far now is a fresh working SGE 62u5 installed with JVM
>>> enabled.
>> Presumably you need the source from the hedeby, hedeby-ge-adapter, and
>> hedeby-cloud-adapter repos as above.  I can't remember how difficult it
>> is to build.  There are instructions somewhere in the www directory of
>> the repo if I recall correctly.
> _______________________________________________
> users mailing list
> users at gridengine.org
> https://gridengine.org/mailman/listinfo/users

William Bryce | VP of Products 
Univa Corporation - 1001 Warrenville Road, Suite 100 Lisle, Il, 65032 USA 
Email bbryce at univa.com | Mobile: 512.751.8014 | Office: 905.237.4462 

More information about the users mailing list