Airflow Xcom Exclusive [work] May 2026

In Apache Airflow, XCom (short for "cross-communication") is the primary mechanism for tasks to share small pieces of data within a DAG run. Unlike global Variables, which are designed for static configuration, XComs are tied to specific task instances and the lifecycle of a single execution. Core Functionality: Push & Pull

• Data Size Limit: By default, Airflow stores XCom values in the metadata database (often PostgreSQL or MySQL). The maximum size is technically determined by the database column type (e.g., BYTEA or BLOB), but practically, you should limit XComs to roughly 48KB to 1MB. Storing large dataframes or files in XCom can crash your scheduler or significantly slow down your database.
• Serializable Data: Data pushed to XCom must be serializable (picklable) if using the default backend. Common formats include JSON strings, integers, or strings.
• Task Context: XComs are tied to a specific dag_id, task_id, and execution_date. They are immutable once written for that specific context.

• Different task_id => different XCom rows even if key matches; no overwrite.

In a multi-tenant environment, you might want to ensure that Task B can pull data from Task A, but Task C (perhaps a notification task) cannot. While Airflow doesn't have native "per-key" permissions, developers implement exclusivity through: airflow xcom exclusive