[gridengine users] Epilog to print out usage summary?

David Chin david.chin at drexel.edu
Fri Jan 23 16:00:03 UTC 2015


I use Univa Grid Engine 8.2.0, and they do have a database to store
accounting info, in addition to the usual accounting flat file. Haven't
played with it much, I'm afraid: the db (PostgreSQL 9.2) runs on the GE
server, but the analysis tool is ARCo and runs off a pre-built VM.

--Dave

On Fri, Jan 23, 2015 at 5:41 AM, Jesse Becker <beckerje at mail.nih.gov> wrote:

> Perhaps not a new thought, but a simple DBM (or NoSQL I suppose) database
> to store job accounting information could work really well here.  I'm not
> suggesting that SGE do that natively (although that'd be cool too),
> but a simple script to post-process the logs as they come in could work
> really well.
>
> On Fri, Jan 23, 2015 at 08:23:46AM +0000, James Abbott wrote:
>
>> Do you rotate your accounting file at all? qacct performance suffers
>> badly when the accounting file gets too big. I  drop a file containing the
>> following into /etc/logrotate.d on the qmaster (replacing the path to the
>> accounting file to it's location on the system...) :
>>
>> /opt/sge/current/bss/common/accounting {
>>    compress
>>    nocreate
>>    dateext
>>    delaycompress
>>    ifempty
>>    # keep logs essentually "forever"
>>    rotate 5000
>>    monthly
>> }
>>
>> which gives me a date-stamped, gzipped monthly accounting file, so qacct
>> continues to perform in a sensible way.
>>
>> James
>>
>> On 23/01/15 01:55, David Chin wrote:
>>
>>> Dear GE users:
>>>
>>> Is it possible to have an epilog script which prints a summary of
>>> resource usage (wallclock time, mem, cpu-seconds)?
>>>
>>> I have tried an epilog bash script which calls qacct, but it took a long
>>> time to generate that output.
>>>
>>> Thanks,
>>>    Dave Chin
>>>
>>> --
>>> David Chin, Ph.D.
>>> david.chin at drexel.edu <mailto:david.chin at drexel.edu>    Sr. Systems
>>> Administrator, URCF, Drexel U.
>>> http://www.drexel.edu/research/urcf/ https://linuxfollies.blogspot.com/
>>> 215.221.4747 (mobile) https://github.com/prehensilecode
>>>
>>>
>>>
>>> _______________________________________________
>>> users mailing list
>>> users at gridengine.org
>>> https://gridengine.org/mailman/listinfo/users
>>>
>>
>>
>> --
>> Dr. James Abbott
>> Lead Bioinformatician
>> Bioinformatics Support Service
>> Imperial College, London
>>
>>
>  _______________________________________________
>> users mailing list
>> users at gridengine.org
>> https://gridengine.org/mailman/listinfo/users
>>
>
>
> --
> Jesse Becker (Contractor)
>
> _______________________________________________
> users mailing list
> users at gridengine.org
> https://gridengine.org/mailman/listinfo/users
>



-- 
David Chin, Ph.D.
david.chin at drexel.edu    Sr. Systems Administrator, URCF, Drexel U.
http://www.drexel.edu/research/urcf/
https://linuxfollies.blogspot.com/
215.221.4747 (mobile)
https://github.com/prehensilecode
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gridengine.org/pipermail/users/attachments/20150123/a594d520/attachment.html>


More information about the users mailing list