[gridengine users] Re??? `cloud' nodes

Chi Chan chichan.gridengine at yahoo.com.tw
Thu Nov 3 05:12:11 UTC 2011

Hi SGE List,

Anyone has Opscode Chef recipes for SGE Grid Engine? I want to setup a simple test cluster and try IT automation and see if it is really useful.


----- 原始信件 ----
寄件者: Rayson Ho <rayrayson at gmail.com>
收件者: Jesse Becker <beckerjes at mail.nih.gov>
副本: Chi Chan <chichan.gridengine at yahoo.com.tw>; Kristen Eisenberg <kristen.eisenberg at yahoo.com>; "users at gridengine.org" <users at gridengine.org>
寄件日期: 2011/10/18 (二) 5:08 PM
主旨: Re: [gridengine users] Re??? `cloud' nodes

On Tue, Oct 11, 2011 at 4:39 PM, Rayson Ho <rayrayson at gmail.com> wrote:
> On Tue, Oct 11, 2011 at 3:58 PM, Jesse Becker <beckerjes at mail.nih.gov> wrote:
>> We're getting a bit off topic here, but CFEngine fits the one of your
>> requirements (but probably not the other). 狢t is written in C, is quite
>> fast, and has a much lower resource footprint than anything based on
>> Ruby.
> Like many other things, some people who are against configuration
> management and some don't...

Re-replying to Jesse's message...

I attended an IBM training on BladeCenter recently, and talked to an
IBMer during lunchtime.

He mentioned U of Toronto's Scinet, which is a TOP500 supercomputer
(highest ranking: #16 in Jun 09). Scinet uses IBM's xCAT, which is
developed for HPC clusters, for provisioning.

In fact some other people use xCAT as a replacement for Platform's
Scali Cluster Manager:


xCAT is written in Perl, so it also is another package to install & maintain.


> It all comes down to, is it cheaper to manually manage a cluster by
> hand, or should we use tools like Chef, Puppet, or Tivoli and hire 1
> less person.
> (But for some HPC sites, running tools in the background is not
> possible or acceptable. For example, the original Catamount OS in
> DoE's Red Store could only run 1 single-threaded process at a time on
> the compute PEs.)
> I brought up the StarCluster Chef integration because doing things by
> hand is not possible in the EC2 scale - in the pre-cloud days, you can
> setup machines 1 at a time, but when you can launch hundreds of
> machines in minutes, doing things by hand is way too slow & expensive.
> BTW, some tutorial videos by Justin & Chris:
> - StarCluster 0.91 Demo: http://www.youtube.com/watch?v=vC3lJcPq1FY
> - Launching a Cluster on Amazon EC2 Spot Instances Using StarCluster:
> http://www.youtube.com/watch?v=2Ym7epCYnSk
> Rayson
>> 贌owever, it is probably not "simpler to learn" by a long shot.
>> Configuration management is deceptively complex once you get beyond the
>> "golden master" view of the world.
>>> On Tue, Oct 11, 2011 at 12:45 PM, Rayson Ho <rayrayson at gmail.com> wrote:
>>> 2) And the BioTeam integrates StarCluster with Opscode Chef, so you
>>> can automate many of the administrative tasks (create users, package
>>> management, service setup, etc) of EC2 SGE clusters:
>>> http://bioteam.net/2011/03/dude-you-got-some-chef-in-my-starcluster/
>>> While I have more experience with IBM Tivoli & Puppet, I am really
>>> impressed with the Chef EC2 module. And Chef is gaining quite a lot of
>>> momentum lately. E.g. Dell recently open sourced Crowbar, which is an
>>> OpenStack installer based on Chef.
>>> I will wait for Puppet Enterprise 2.0, which is supposed to have new
>>> EC2 & VMware provisioning & orchestration capabilities, and see how
>>> Puppet compares with Chef before I decide if I am switching to Chef.
>>> But configuration management is real and it can cut down a lot of IT
>>> infrastructure maintenance.
>>> Rayson
>>> On Mon, Oct 10, 2011 at 7:01 PM, Kristen Eisenberg
>>> <kristen.eisenberg at yahoo.com> wrote:
>>>> Chris Dagdigian <dag at sonsorol.org> writes:
>>>>> By FAR the best way to run standalone Grid Engine clusters on the Amazon
>>>>> Cloud today is to simply use MIT Starcluster :
>>>>> http://web.mit.edu/stardev/cluster/index.html
>>>> I didn't mention it as I got the impression that that wasn't the OP's
>>>> case, but probably it should be mentioned in the same place on
>>>> gridengine.info, assuming that's still the best place for such things.
>>>> Kristen Eisenberg
>>>> Billige Fl???e
>>>> Marketing GmbH
>>>> Emanuelstr. 3,
>>>> 10317 Berlin
>>>> Deutschland
>>>> Telefon: +49 (33)
>>>> 5310967
>>>> Email:
>>>> utebachmeier at
>>>> gmail.com
>>>> Site:
>>>> http://flug.airego.de - Billige Fl???e vergleichen
>>>> _______________________________________________
>>>> users mailing list
>>>> users at gridengine.org
>>>> https://gridengine.org/mailman/listinfo/users
>>> _______________________________________________
>>> users mailing list
>>> users at gridengine.org
>>> https://gridengine.org/mailman/listinfo/users
>>> _______________________________________________
>>> users mailing list
>>> users at gridengine.org
>>> https://gridengine.org/mailman/listinfo/users
>> --
>> Jesse Becker
>> NHGRI Linux support (Digicon Contractor)

More information about the users mailing list