Google Cloud Composer (GCC)

Workflow

Google Cloud Composer (GCC) is a managed workflow automation service built on Apache Airflow, used to develop, schedule and monitor batch-oriented processing. Automating execution from within GCC creates an additional island of automation for IT Operations to manage. This allows you to easily integrate directed acyclic graphs (Airflow DAGs) with your existing enterprise workload automation solution. You are instantly inheriting the advanced capabilities of your enterprise solution to deliver your digital transformation quicker and more successfully. By extending your existing enterprise automation to GCC you maintain end-to-end visibility, regain centralized command and control, alerting, SLA management, reporting and auditing.

Products

Automic
AutoSys
dSeries
ESP
CA 7

Benefits

Centralize Airflow pipeline management within a single pane of glass to eliminate manual intervention.
Schedule DAGs and configure cross-DAG dependencies to ensure proper run order.
Track runs automatically to completion and trigger instant notifications on failures.

Use case

Multi-Source Data Pipeline Orchestration

Execute scheduled or on-demand Airflow DAGs to orchestrate ETL workflows that extract, transform, and load data across systems. Monitor the DAG status and progress of individual tasks in real time and automatically recover from failures by triggering retries or fallback DAGs based on scheduler logic.

Features

Schedule execution of Apache Airflow DAGs
Monitor to completion Airflow DAGs
Retrieve DAG logs for problem diagnosis
Restart DAG processing
Alert on failure of DAG processing
Create dependencies between DAG executions
Incorporate DAG processing in existing enterprise automation workflows including dependency enforcement

Automic

Resource Docs

AutoSys

Video Resource Docs