DAGMan for orchestrating complex workflows on HTC resources (High Throughput Computing)

Approved: Yes
Title: DAGMan for orchestrating complex workflows on HTC resources (High Throughput Computing)
Category: Tool
Skill Level:
Intermediate (305), Advanced (306)

Description:
DAGMan (Directed Acyclic Graph Manager) is a meta-scheduler for HTCondor. It
manages dependencies between jobs at a higher level than the HTCondor
Scheduler.

It is a workflow management system developed by the High-Throughput Computing
(HTC) community, specifically for managing large-scale scientific
computations and data analysis tasks. It enables users to define complex
workflows as directed acyclic graphs (DAGs). In a DAG, nodes represent
individual computational tasks, and the directed edges represent dependencies
between the tasks. DAGMan manages the execution of these tasks and ensures
that they are executed in the correct order based on their dependencies.

The primary purpose of DAGMan is to simplify the management of large-scale
computations that consist of numerous interdependent tasks. By defining the
dependencies between tasks in a DAG, users can easily express the order of
execution and allow DAGMan to handle the scheduling and coordination of the
tasks. This simplifies the development and execution of complex scientific
workflows, making it easier to manage and track the progress of computations.


Link to Resource:
- DAGMan (https://htcondor.org/dagman/dagman.html)
- DAGMan Workflows (https://htcondor.readthedocs.io/en/latest/users-manual/dagman-workflows.html)

Tags:
open-science-grid (718)

Domain:
ACCESS CSSN (780), Campus Champions (572), CAREERS (323), CCMNet (835), Great Plains (311), Kentucky (322), Northeast (308)

Would you like to associate this resource with an Affinity Group?: {Empty}