CDF DB Applications

Calibration DB

According to Petar, this is the least known application, mostly in terms of operational procedures.

1. How to generate and add calibration tables to the database

   CDF Note 5641

   see also additional notes from internal cdf email

2. How to make FroNTier aware of new calibration table

   Dennis's cookbook approach to adding FroNTier software

   See also Frontier related software in CDF wiki entry

3. Useful contacts:
   • Petar Maksimov was the CDF contact person for this process, but he is rarely at FNAL now and is hoping to pass the baton.
   • Matt Martin is another CDF expert on this process.
   • css-dsg@fnal.gov (Nelly, Anil, Diana, Julie) are DBAs who are invaluable in helping with database passwords, explaining unexpected database behavior, and the like.
   • Randy Herber (herber@fnal.gov) knows quite a bit about the TomCat servers, jdbc connections, and various CDF DB schema, all of which come in to play with FroNTier. He can often explain unexpected behavior in startling detail.
   • Dennis Box (dbox@fnal.gov) wrote a lot of the original Codegen system and helped productize the Frontier back end

4. Holes in this list:
   • OK I have created a table, tested the access (read and write) software in development, integration and production databases. Who do I inform so that
     • calibrations get written to new tables
     • calibrations make it into used sets
     • calibrations make it into passes?
5. Useful links
   • CDF Fast Navigator
   • Most Recently Added Calibration DB Tables
   • CDF DB GroupWiki

Contacts: Rick Snider, Bo Jayatilaka, Matt Martin (?)

Luminosity DB

Pre-scales application/recalculation effort is ongoing. January data has been updated. The hope is to eliminate data backlog by/during summer shutdown. Concern is that development database is being used for production purposes, and that development server does not have sufficient power for this task.

Contact: Jonathan Louis, Petar.

SAM

Offline production control system. Supported by REX. Replaced file catalog database.

Contact: Adam Lyon

Frontier

Distributed data caching system. Based on HTTP caching software. Cache is one-key-to-blob kind. Widely used in production.

In most use cases (when caching is beneficial) helps significantly reduce load on Oracle. The product is in support/maintenance mode.

Some work may be required to write additional back-ends as new pieces of data added to databases, or users try to use new tables.

Contact: Randolph Herber, Petar Maksimovic

Monte-Carlo Trigger Simulation

Some simulation jobs contact database. Some of them use badly designed DB access module, which uses DB resources in highly inefficient manner.

Dennis is helping debug/optimize the code. Also, another approach is to add Frontier backend for this piece of data.

Contacts: Petar Maksimovic, Barry Blumenfeld, Simona Rolli

DBSMON

Database performance/access online monitoring tool, based on MySQL database. Currently supported by Randy Herber. Originally developed/maintained by Eric Wicklund, Yuyi Guo, Margherita Vittone.

Claimed to be used in CDF Control Room procedures.

Online Database

Little is known about. Dennis wrote and knows some of the software involved. Operational issues/procedures are not known.

Contact: Bill Badgett

Other Databases

Trigger, Hardware, Runs

Other Application Notes

Here is a 'capture' of the most database intensive applications on 6/12/2007. I (dbox) have knowlege of ProductionExe and TRGSim++ but not the others. Our mission, should we decide to accept it, is to:

1. Understand what they are doing to which databases and
2. Frontierize the applications where it makes sense. ProductionExe is frontierized, TRGSim++ is not (yet).

----- Top Apps regardless of many same-name apps ------


 - Query used is : 
select application, sum(duration) as etime /nfrom Job j, APIkeyed a where ( j.startTime > (1181710800- 24*3600) /nand j.startTime <= 1181710800 ) and j.jobId=a.jobId /ngroup by application order by etime desc limit 20
 Apps 		  Sum(duration) 
cdfSim		1155076.2191509
TRGSim++		1106372.530475
ProductionExe		420973.741204999
stnmaker_prod.exe		21694.8568989999
BitsRunSummaryExe		16867.5311780001
TopFind		16026.477388
AC++Dump		7531.87963200004
BFracSkim		3628.76670300001
CalibExe		2069.188569
BcFindExe		1474.479438
OCUevNExe		1387.04492
leptons.exe		837.590638999998
DFinderExample.exe		829.311808000001
MyAnaExe		775.039395000001
ucntuple.exe		632.023324
cdfSim_6.1.4mc_HW_LH_LNF_BCgen_final		521.509832000001
strip		396.439775
CandsExe		191.491642
TrigMon		100.712406
siHitAnalyzer		93.997157

Contacts

Petar Maksimovic - CDF, Jon Hopkins University
Randy Herber - CD
Rick Snider - CD
Dennis Box - CD