[gridengine users] Linking two SoGE Clusters?

Reuti reuti at staff.uni-marburg.de
Fri Mar 31 15:39:30 UTC 2017


Hi,

> Am 31.03.2017 um 16:37 schrieb Florian Zillner <f.zillner at karso.de>:
> 
> Hi Guys,
> 
> I have a question regarding connecting two existing SoGE (8.1.8)
> clusters. They can reach each other, but besides that, they're
> independent, at least SoGE-wise.
> 
> The idea is to be able to submit jobs to the remote (remote = the other
> cluster) location, when the local cluster is fully utilized. I've read
> about the transfer queues at
> http://gridscheduler.sourceforge.net/howto/TransferQueues/transferqueues.html
> but I'm confused.
> 
> Is there an updated / extended description available somewhere? Is there
> another way of linking the clusters?

This procedure was there to outline the necessary setup to get Transfer Queues back, which were implemented in Codine 5.2 before it became GridEngine 5.3 without this feature.

But IIRC the Howto did not work directly with SGE 6.0, and has several constraints which were not feasible in our environment, especially:

"both clusters share a common "namespace" or "administrative domain". By this we mean: common usernames/UIDs/GIDs, common filesystem(s), mutually accessible hosts and hostnames, behind the same firewalls, etc."

===

As the UID/GID and file systems were different in our setup, I implemented it in a different way:

- It was limited to certain applications, which are already installed on the remote cluster.
- The necessary input and output files (resp. directories) were named in the job context, to allow an automatic file staging in both directions.
- Passphraseless SSH was necessary for the file staging (or hostbased), i.e. between the head nodes of the two clusters.

I never assembled a Howto for it, as I realized that it was a highly specialized solution. An installation of the plain files starter.sh/terminate.sh won't give a working solution, but I can send them to you in case you are interested in PM.

-- Reuti
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 203 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://gridengine.org/pipermail/users/attachments/20170331/547a4be8/attachment.sig>


More information about the users mailing list