Production System Design

The following diagrams and text represent some thoughts on a production system for CMS MC processing. The proposed system would have three main components:

1. Scheduler/Policy Controller Central tool that keeps track of processing requests and priorities and assigns work to Production Agents (ProdAgents)
2. Production Manager (ProdMgr) Semi centralised DB based service for maintaining details of processing requests and tracking progress of those requests.
3. ProductionAgent (ProdAgent) Distributed Queuing and management system to create, submit and track MC jobs in some execution environment (eg a grid system or a local farm etc). The ProdAgent is assigned work by the Scheduler, which tells it where the ProdMgr is that contains its work. The ProdAgent then requests work on its own terms from the ProdMgr. The ProdMgr manages task allocations to the ProdAgent and keeps a running count. The ProdAgent reports back its progress for each allocation to the ProdMgr.

Details showing the ProdAgent interactions with other components:

Wiki Page

A Wiki containing design notes for each component of the Production system is available:

• Wiki Pages

Further details

• Notes on the Architecture (text)
• Notes on the Interfaces (text)

Diagrams (all PNG format)

• ProdMgrEntityRelation
• ExampleWorkflowModelling.png
• ProdAgentNetwork.png