poc-airflow-dbt

This repository contains the documentation and code related to the Radiant POC.

Directory structure

• docs/experiments: Experiment journals and resources (image files).
• radiant: Various code files
  • radiant/experiments: Contains all the experiments' code and configuration.
  • radiant/iceberg: docker-compose.yml specification & is used as the docker compose workspace.

Note:

Experiments exp001, exp002, exp004 all use the same Stack, specified by the radiant/iceberg/docker-compose.yml file.

However, exp004 which adds the airflow stack, has its own specific docker-compose.yml. See Experiment #4: Airflow + Cosmos for more details.

Experiments

• Experiment #1: External sources
• Experiment #2: External Materialization
• Experiment #3: Incremental Materialization
• Experiment #4: Airflow + Cosmos