You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+2-3
Original file line number
Diff line number
Diff line change
@@ -8,9 +8,8 @@ Project Contents
8
8
9
9
Your Astro project contains the following files and folders:
10
10
11
-
- dags: This folder contains the Python files for your Airflow DAGs. By default, this directory includes two example DAGs:
12
-
-`example_dag_basic`: This DAG shows a simple ETL data pipeline example with three TaskFlow API tasks that run daily.
13
-
-`example_dag_advanced`: This advanced DAG showcases a variety of Airflow features like branching, Jinja templates, task groups and several Airflow operators.
11
+
- dags: This folder contains the Python files for your Airflow DAGs. By default, this directory includes one example DAG:
12
+
-`example_astronauts`: This DAG shows a simple ETL pipeline example that queries the list of astronauts currently in space from the Open Notify API and prints a statement for each astronaut. The DAG uses the TaskFlow API to define tasks in Python, and dynamic task mapping to dynamically print a statement for each astronaut. For more on how this DAG works, see our [Getting started tutorial](https://docs.astronomer.io/learn/get-started-with-airflow).
14
13
- Dockerfile: This file contains a versioned Astro Runtime Docker image that provides a differentiated Airflow experience. If you want to execute other commands or overrides at runtime, specify them here.
15
14
- include: This folder contains any additional files that you want to include as part of your project. It is empty by default.
16
15
- packages.txt: Install OS-level packages needed for your project by adding them to this file. It is empty by default.
This DAG queries the list of astronauts currently in space from the
5
+
Open Notify API and prints each astronaut's name and flying craft.
6
+
7
+
There are two tasks, one to get the data from the API and save the results,
8
+
and another to print the results. Both tasks are written in Python using
9
+
Airflow's TaskFlow API, which allows you to easily turn Python functions into
10
+
Airflow tasks, and automatically infer dependencies and pass data.
11
+
12
+
The second task uses dynamic task mapping to create a copy of the task for
13
+
each Astronaut in the list retrieved from the API. This list will change
14
+
depending on how many Astronauts are in space, and the DAG will adjust
15
+
accordingly each time it runs.
16
+
17
+
For more explanation and getting started instructions, see our Write your
18
+
first DAG tutorial: https://docs.astronomer.io/learn/get-started-with-airflow
19
+
20
+

21
+
"""
22
+
23
+
fromairflowimportDataset
24
+
fromairflow.decoratorsimportdag, task
25
+
frompendulumimportdatetime
26
+
importrequests
27
+
28
+
#Define the basic parameters of the DAG, like schedule and start_date
29
+
@dag(
30
+
start_date=datetime(2024, 1, 1),
31
+
schedule="@daily",
32
+
catchup=False,
33
+
doc_md=__doc__,
34
+
default_args={"owner": "Astro", "retries": 3},
35
+
tags=["example"],
36
+
)
37
+
defexample_astronauts():
38
+
#Define tasks
39
+
@task(
40
+
#Define a dataset outlet for the task. This can be used to schedule downstream DAGs when this task has run.
41
+
outlets=[Dataset("current_astronauts")]
42
+
) # Define that this task updates the `current_astronauts` Dataset
43
+
defget_astronauts(**context) ->list[dict]:
44
+
"""
45
+
This task uses the requests library to retrieve a list of Astronauts
46
+
currently in space. The results are pushed to XCom with a specific key
47
+
so they can be used in a downstream pipeline. The task returns a list
0 commit comments