[gridengine users] maintaining spooldb/sge_job

Dave Love d.love at liverpool.ac.uk
Tue Jun 12 13:43:42 UTC 2012

Ron Chen <ron_chen_123 at yahoo.com> writes:

> We even spent time and read the BerkeleyDB source code before we added
> that feature in GE 2011.11, so just pointing to the man page and say
> that something is wrong may be too naive.

Or not.  People interested should research the issue and do the
experiments, like some of us did.

Bill Bryce presumably has good enough advice from people who
wrote/maintained the DB code and probably had easy access to the DB

> As mentioned by Rayson, we
> only enabled the DB_PRIVATE flag when there is no need for explict run
> db_checkpoint or db_archive process operating on the BDB, and that is
> only when the cluster is not using a BerkeleyDB RPC server for
> spooling.

Contrast <http://gridengine.org/pipermail/users/2012-May/003584.html>.
Users have no indication they shouldn't be doing things like db_dump.

> Other processes do not read or write the Berkeley DB
> directly, and qmaster is the only one performing I/O against the
> DB. Long story short, just read the email from Rayson that explains
> the details at:
> http://gridengine.org/pipermail/users/2012-May/003683.html

The thread has the bad advice to run external commands under cron on
what we must assume is a privately-opened DB spool:
<http://gridengine.org/pipermail/users/2012-May/003562.html>.  If BDB
doc is wrong about DB_PRIVATE as suggested, why would it matter anyhow?

> This was what's happened: we helped Dave

Come off it.

> who claimed that he wanted to
> collaborate, and then later on he stabbed our back by saying that the
> work we shared with him is "definitely not safe":

It should be clear from the context that it's the operation of external
commands which is not safe with a private database.  That's why making
the private spool optional may be reasonable, given sufficient health
warning.  (The "code" is the two tokens "| DB_PRIVATE" which come from
the DB documentation.)

The stupid conspiracy theories and -- at best -- misrepresentation of
others are unbecoming.  Not everyone is malicious and many are just
helpful, without commercial interests to push.  It's unfortunate if some
people's infallibility is more important than others' data integrity,
notwithstanding rants about "professionalism".

Community Grid Engine:  http://arc.liv.ac.uk/SGE/

More information about the users mailing list