[gridengine users] Writing Monthly CPU Usage Reports

Gowtham g at mtu.edu
Thu Jun 14 10:28:28 UTC 2018


Good morning, Doug.

I'd start with Fred's recommendation/approach. It's similar to an approach
I developed here at Michigan Tech but I use MySQL database. Having had the
some experience with it before, I started created user accounts (e.g.,
john, jane, jill, james, etc.) with their own research group named after
the PI as the primary UNIX group (e.g., amy-users). One of the tables in
MySQL database keeps information about their full names, department and
college affiliations, date of account creation, etc. If your user accounts
aren't created this way, you could probably do a key-value match type
process to affiliate a given user with a her/his research group.

SGE accounting log has, I believe, 46 fields for each entry (i.e.,
completed job or simulation) separated by colon (:). The description for
the each field is available in the man page or
http://gridscheduler.sourceforge.net/htmlman/htmlman5/accounting.html. One
of this fields is the primary unix group. Once the data is in the MySQL
table (a cron job could update it once a day or so), you could extract and
prepare reports as you feel necessary. My workflow uses LaTeX to generate
PDFs  and a cron job to mail out the report to PIs every Monday morning.
University administrators get it once a quarter.

Best regards,
Gowtham

--
Gowtham, PhD
Director of Research Computing, IT
Research Associate Professor, ECE
Michigan Technological University

P: (906) 487-4096
F: (906) 487-2787
https://it.mtu.edu
https://hpc.mtu.edu


On Thu, Jun 14, 2018 at 5:12 AM Tina Friedrich <tina.friedrich at it.ox.ac.uk>
wrote:

> On Tuesday, 12 June 2018 21:55:08 BST Douglas Duckworth wrote:
> > Hi Everyone
> >
> >
> > We're looking into writing a monthly report which lists CPU usage by
> group,
> > within share tree, along with a list of users therein as well as their
> > usage.
> >
> >
> > Does anyone have any advice on writing such a report or perhaps know of
> any
> > sample reports which they could share?
>
> Hi Doug,
>
> I did something like this using R & markup in my old job (as in, there was
> a
> markup report with embedded R code that would generate the stats). I
> believe I
> generated both html and pdf reports from it. If memory serves right, I had
> it
> as a postrotate on my monthly logrotate for the accounting file... Worked
> pretty well, actually.
>
> I'm pretty sure I wrote some functions to read accounting files into R
> etc.
> Would you like me to dig out the code?
>
> Tina
>
> > --
> > Thanks,
> >
> > Douglas Duckworth, MSc, LFCS
> > HPC System Administrator
> > Scientific Computing Unit
> > Weill Cornell Medicine
> > E: doug at med.cornell.edu
> > O: 212-746-6305
> > F: 212-746-8690
>
>
> --
> Tina Friedrich, Snr HPC Systems Administrator, Advanced Research Computing
> Research Computing and Support Services, Academic IT
> IT Services, University of Oxford
> http://www.arc.ox.ac.uk
> _______________________________________________
> users mailing list
> users at gridengine.org
> https://gridengine.org/mailman/listinfo/users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gridengine.org/pipermail/users/attachments/20180614/11b7fccf/attachment.html>


More information about the users mailing list