October 20, 11:59 PM ISTOct 20, 11:59 PM IST
September 26, 9:55 AM ISTSep 26, 9:55 AM IST
2 left (3 attempts every 24 hours)
30 minutes per attempt30 min per attempt
1 left (1 total within the time limit)
To pass you need at least 80%. We keep your highest score.
Graded Assignment • 30 min
Some of the orchestration engines require that you define your data pipeline as a Directed Acyclic Graph (DAG). What does that mean?
A DAG is a visual representation of a data pipeline, depicting tasks as nodes and their dependencies as edges, ensuring a directed flow without any cycles.
How is orchestration different from simple Cron scheduling?
Unlike simple Cron scheduling, orchestration enables you to set up task dependencies so certain tasks don't start until others have successfully completed, monitor tasks, receive alerts on task failures and retry tasks.
One of the Airflow’s core components is the scheduler. What is the role of the scheduler?
The scheduler is a critical component of Airflow. It constantly checks for tasks that are ready to be executed based on their schedule or the completion of their dependencies. The scheduler pushes ready tasks to a queue and uses an executor to manage their execution.
Which of the following tasks can you do in the Airflow UI? Select all that apply.
This is one of the UI features that allows you to test your DAG.
This is one of the UI features that helps you understand the specific reasons behind task failures, by offering detailed error messages that guide you in troubleshooting and resolving issues with your DAGs.
You can manually create global variables in the UI and use them in your code using the method `Variable.get()`.
When you instantiate a DAG in Airflow, which parameter do you need to specify if you don’t want the scheduler to start the DAG runs that have not been run since the last data interval?
The scheduler, by default, will start a DAG Run for any data interval that has not been run since the last data interval. If you don’t want your DAG to handle its catchup, then you need to set catchup to False.
XComs are used to share data between tasks in Airflow. For which of the following use cases can XCom be used? Select all that apply.
XComs are designed to pass small amounts of data between tasks; a single value metric is an example of such data.
XComs are designed to pass small amounts of data between tasks; single value query results is an example of such data.
XComs are designed to pass small amounts of data between tasks; task metadata is an example of such data.
Consider the following DAG.
Which of the following statements correctly represents the DAG structure?
Nice job!
Which of the following statements is true about TaskFlow API?
Nice job! TaskFlow API offers a more friendly coding experience through the use of decorators, that you can use instead of explicitly using Airflow operators.
Which of the following code pieces follows the best practices in writing DAGs or their constituent tasks in Airflow?
It’s a good practice to use templated fields as they help keep your DAGs idempotent. For more information about Airflow best practices, check out this reading item.
Consider the following code:
Which of the following statements correctly define the dependencies?
Nice job! In this example, you used the variables that represent the PythonOperator instances.
