Apache Airflow

What is Airflow?

Apache Airflow is an open-source workflow management platform…

Back to Table of Contents

What issues does Airflow resolve?

Crons are an old technique of task scheduling…

Back to Table of Contents

Explain how workflow is designed in Airflow?

A directed acyclic graph (DAG) is used to design an Airflow workflow…

Back to Table of Contents

Explain Airflow Architecture and its components?

There are four major components to airflow. + Webserver + This is the Airflow UI built on the Flask…

Back to Table of Contents

What are the types of Executors in Airflow?

The executors are the components that actually execute the tasks…

Back to Table of Contents

What are the pros and cons of SequentialExecutor?

Pros: + It’s simple and straightforward to set up. + It’s a good way to test DAGs while they’re being developed…

Back to Table of Contents

What are the pros and cons of LocalExecutor?

Pros: + Able to perform multiple tasks. + Can be used to run DAGs during development…

Back to Table of Contents

What are the pros and cons of CeleryExecutor?

Pros: + It allows for scalability. + Celery is responsible for managing the workers…

Back to Table of Contents

What are the pros and cons of KubernetesExecutor?

Pros: It combines the benefits of CeleryExecutor and LocalExecutor in terms of scalability and simplicity…

Back to Table of Contents

How to define a workflow in Airflow?

Python files are used to define workflows. DAG (Directed Acyclic Graph)…

Back to Table of Contents

How do you make the module available to airflow if you’re using Docker Compose?

If we are using Docker Compose, then we will need to use a custom image with our own additional dependencies…

Back to Table of Contents

How to schedule DAG in Airflow?

DAGs could be scheduled by passing a timedelta or a cron expression (or one of the @ presets)…

Back to Table of Contents

What is XComs In Airflow?

XCom (short for cross-communication) are messages that allow data to be sent between tasks…

Back to Table of Contents

What is xcom_pull in XCom Airflow?

The xcom push and xcom pull methods on Task Instances are used to explicitly “push” and “pull” XComs…

Back to Table of Contents

What is Jinja templates?

Jinja is a templating engine that is quick, expressive, and extendable…

Back to Table of Contents

How to use Airflow XComs in Jinja templates?

We can use XComs in Jinja templates as given below: + SELECT * FROM {{ task_instance.xcom_pull(task_ids=’foo’, key=’table_name’) }}

Back to Table of Contents