Ensembl/ensembl-hive

View on GitHub
t/04.meadow/fake_bin/bacct

Summary

Maintainability
Test Coverage
#!/bin/bash

read -r -d '' bacct << "EOF"
Accounting information about jobs that are: 
  - submitted by all users.
  - accounted on all projects.
  - completed normally or exited
  - executed on all hosts.
  - submitted to all queues.
  - accounted on all service classes.
------------------------------------------------------------------------------

Job <2581807[1]>, Job Name <protein_83_dumps-Hive-1Gb_job-240[1]>, User <cc21>,
                     Project <ensembl-compara>, Status <DONE>, Queue <normal>, 
                     Command </nfs/users/nfs_c/cc21/src/ensembl-hive/scripts/ru
                     nWorker.pl -url mysql://ensadmin:${EHIVE_PASS}@compara3:33
                     06/cc21_protein_83_dumps -rc_name 1Gb_job --reg_conf /nfs/
                     users/nfs_c/cc21/src/ensembl-compara/scripts/pipeline/prod
                     uction_reg_conf.pl>, Share group charged </WTSI/Computatio
                     nalGenomics/Annotation/ensembl-grp/ensembl-compara/cc21>
Thu Nov 26 14:22:42: Submitted from host <farm3-head2>, CWD <$HOME/src>, Output
                      File </dev/null>;
Thu Nov 26 14:25:09: [1] dispatched to <bc-25-1-10>, Effective RES_REQ <select[
                     (mem>1000) && (type == any )] order[r15s:pg] rusage[mem=10
                     00.00] >;
Thu Nov 26 14:25:12: Completed <done>.

 EXCEPTION STATUS:  underrun 

Accounting information about this job:
     Share group charged </WTSI/ComputationalGenomics/Annotation/ensembl-grp/ensembl-compara/cc21>
     CPU_T     WAIT     TURNAROUND   STATUS     HOG_FACTOR    MEM    SWAP
      2.74      147            150     done         0.0182    28M    144M
------------------------------------------------------------------------------

Job <2581801[48]>, Job Name <protein_83_dumps-Hive-1Gb_job-239[48]>, User <cc21
                     >, Project <ensembl-compara>, Status <DONE>, Queue <normal
                     >, Command </nfs/users/nfs_c/cc21/src/ensembl-hive/scripts
                     /runWorker.pl -url mysql://ensadmin:${EHIVE_PASS}@compara3
                     :3306/cc21_protein_83_dumps -rc_name 1Gb_job --reg_conf /n
                     fs/users/nfs_c/cc21/src/ensembl-compara/scripts/pipeline/p
                     roduction_reg_conf.pl>, Share group charged </WTSI/Computa
                     tionalGenomics/Annotation/ensembl-grp/ensembl-compara/cc21
                     >
Thu Nov 26 14:21:41: Submitted from host <farm3-head2>, CWD <$HOME/src>, Output
                      File </dev/null>;
Thu Nov 26 14:24:57: [48] dispatched to <bc-27-2-07>, Effective RES_REQ <select
                     [(mem>1000) && (type == any )] order[r15s:pg] rusage[mem=1
                     000.00] >;
Thu Nov 26 14:25:16: Completed <done>.

 EXCEPTION STATUS:  underrun 

Accounting information about this job:
     Share group charged </WTSI/ComputationalGenomics/Annotation/ensembl-grp/ensembl-compara/cc21>
     CPU_T     WAIT     TURNAROUND   STATUS     HOG_FACTOR    MEM    SWAP
      2.61      196            215     done         0.0121    50M    269M
------------------------------------------------------------------------------

Job <3194397[75]>, Job Name <orth_qm_mammals-Hive-default-430[75]>, User <cc21>
                     , Project <ensembl-compara>, Status <EXIT>, Queue <normal>
                     , Command </nfs/users/nfs_c/cc21/src/ensembl-hive/scripts/
                     runWorker.pl -url mysql://ensadmin:${EHIVE_PASS}@compara3:
                     3306/cc21_orth_qm_mammals -rc_name default>, Share group c
                     harged </WTSI/ComputationalGenomics/Annotation/ensembl-grp
                     /ensembl-compara/cc21>
Wed Dec  2 13:52:32: Submitted from host <farm3-head2>, CWD <$HOME/OrthologQM>,
                      Output File </dev/null>;
Wed Dec  2 13:52:34: [75] dispatched to <bc-31-2-11>, Effective RES_REQ <select
                     [(mem>100) && (type == any )] order[r15s:pg] rusage[mem=10
                     0.00] >;
Wed Dec  2 13:53:29: Completed <exit>; TERM_MEMLIMIT: job killed after reaching
                      LSF memory usage limit.

 EXCEPTION STATUS:  underrun

Accounting information about this job:
     Share group charged </WTSI/ComputationalGenomics/Annotation/ensembl-grp/ensembl-compara/cc21>
     CPU_T     WAIT     TURNAROUND   STATUS     HOG_FACTOR    MEM SWAP
      6.97        2             57     exit         0.1223   102M 218M
------------------------------------------------------------------------------

SUMMARY:      ( time unit: second ) 
 Total number of done jobs:       2      Total number of exited jobs:     0
 Total CPU time consumed:       5.3      Average CPU time consumed:     2.7
 Maximum CPU time of a job:     2.7      Minimum CPU time of a job:     2.6
 Total wait time in queues:   343.0
 Average wait time in queue:  171.5
 Maximum wait time in queue:  196.0      Minimum wait time in queue:  147.0
 Average turnaround time:       182 (seconds/job)
 Maximum turnaround time:       215      Minimum turnaround time:       150
 Average hog factor of a job:  0.02 ( cpu time / turnaround time )
 Maximum hog factor of a job:  0.02      Minimum hog factor of a job:  0.01
 Total throughput:          1800.00 (jobs/hour)  during    0.00 hours
 Beginning time:       Nov 26 14:25      Ending time:          Nov 26 14:25
EOF


received="$*"
expected="$EHIVE_EXPECTED_BACCT"

if [ "$received" = "$expected" ];
then
  echo "$bacct"
else
  echo "diff: received $received"
  echo "diff: was expecting $expected"
fi