Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Separated code blocks and added linking for testing directory #15636

Merged
merged 2 commits into from
Aug 14, 2022
Merged

Separated code blocks and added linking for testing directory #15636

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
682e045
Separated code blocks and added linking for testing directory
ryankfu Aug 14, 2022
239b3ba
Updated links to relative links and added clearer wording for
ryankfu Aug 14, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 37 additions & 12 deletions docs/contributing-to-airbyte/gradle-cheatsheet.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,10 @@ We have 3 ways of slicing our builds:

In our CI we run **Build Platform** and **Build Connectors Base**. Then separately, on a regular cadence, we build each connector and run its integration tests.

We split Build Platform and Build Connectors Base from each other for a few reasons: 1. The tech stacks are very different. The Platform is almost entirely Java. Because of differing needs around separating environments, the Platform build can be optimized separately from the Connectors one. 2. We want to the iteration cycles of people working on connectors or the platform faster _and_ independent. e.g. Before this change someone working on a Platform feature needs to run formatting on the entire codebase \(including connectors\). This led to a lot of cosmetic build failures that obfuscated actually problems. Ideally a failure on the connectors side should not block progress on the platform side. 3. The lifecycles are different. One can safely release the Platform even if parts of Connectors Base is failing \(and vice versa\).
We split Build Platform and Build Connectors Base from each other for a few reasons:
1. The tech stacks are very different. The Platform is almost entirely Java. Because of differing needs around separating environments, the Platform build can be optimized separately from the Connectors one.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

technically in markdown there needs to be an empty line between the bullet list and the paragraph before it I think?

Suggested change
1. The tech stacks are very different. The Platform is almost entirely Java. Because of differing needs around separating environments, the Platform build can be optimized separately from the Connectors one.
1. The tech stacks are very different. The Platform is almost entirely Java. Because of differing needs around separating environments, the Platform build can be optimized separately from the Connectors one.

Copy link
Contributor Author

@ryankfu ryankfu Aug 14, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless somehow the markdown viewer on IntelliJ is incorrect I don't think there needs to be an empty line between bulleted list and paragraph

Screen Shot 2022-08-13 at 22 39 57

Based on this bulleted list tutorial it doesn't seem like a empty line is neccesary

2. We want to the iteration cycles of people working on connectors or the platform faster _and_ independent. e.g. Before this change someone working on a Platform feature needs to run formatting on the entire codebase \(including connectors\). This led to a lot of cosmetic build failures that obfuscated actually problems. Ideally a failure on the connectors side should not block progress on the platform side.
3. The lifecycles are different. One can safely release the Platform even if parts of Connectors Base is failing \(and vice versa\).

Future Work: The next step here is to figure out how to more formally split connectors and platform. Right now we exploit behavior in `settings.gradle` to separate them. This is not a best practice. Ultimately, we want these two builds to be totally separate. We do not know what that will look like yet.

Expand Down Expand Up @@ -48,10 +51,10 @@ In order to "build" the project. This task includes producing all artifacts and

For example all the following are valid.

```text
./gradlew build
SUB_BUILD=PLATFORM ./gradlew build
SUB_BUILD=CONNECTORS_BASE ./gradlew build
```shell
./gradlew build # builds the entire Airbyte project including every single connector supported
SUB_BUILD=PLATFORM ./gradlew build -x test # builds Airbyte Platform without running tests
SUB_BUILD=CONNECTORS_BASE ./gradlew build # builds all Airbyte connectors and runs unit tests
```

### Formatting
Expand All @@ -60,7 +63,7 @@ The build system has a custom task called `format`. It is not called as part of

For example all the following are valid.

```text
```shell
./gradlew format
SUB_BUILD=PLATFORM ./gradlew format
SUB_BUILD=CONNECTORS_BASE ./gradlew format
Expand All @@ -72,14 +75,21 @@ SUB_BUILD=CONNECTORS_BASE ./gradlew format

This command just builds the docker images that are used as artifacts in the platform. It bypasses running tests.

```text
```shell
SUB_BUILD=PLATFORM ./gradlew build
```

#### Running Tests

The Platform has 3 different levels of tests: Unit Tests, Acceptance Tests, Frontend Acceptance Tests.

| Test | Used | Description |
|:------------|:----:|:----------------------------------------------------------------------------------------------|
| Unit | X | Aims to test each component (e.g. a method function) |
| Integration | | Checks the data flow from one module to other modules |
| System | | Tests overall interaction of components, includes load, performance, reliability and security |
| Acceptance | X | Assess whether the Product is working for the user's viewpoint |

**Unit Tests**

Unit Tests can be run using the `:test` task on any submodule. These test class-level behavior. They should avoid using external resources \(e.g. calling staging services or pulling resources from the internet\). We do allow these tests to spin up local resources \(usually in docker containers\). For example, we use test containers frequently to spin up test postgres databases.
Expand All @@ -91,13 +101,18 @@ We split Acceptance Tests into 2 different test suites:
* Platform Acceptance Tests: These tests are a coarse test to sanity check that each major feature in the platform. They are run with the following command: `SUB_BUILD=PLATFORM ./gradlew :airbyte-tests:acceptanceTests`. These tests expect to find a local version of Airbyte running. For testing the docker version start Airbyte locally. For an example, see the [script](https://github.com/airbytehq/airbyte/blob/master/tools/bin/acceptance_test.sh) that is used by the CI. For Kubernetes, see the [script](https://github.com/airbytehq/airbyte/blob/master/tools/bin/acceptance_test_kube.sh) that is used by the CI.
* Migration Acceptance Tests: These tests make sure the end-to-end process of migrating from one version of Airbyte to the next works. These tests are run with the following command: `SUB_BUILD=PLATFORM ./gradlew :airbyte-tests:automaticMigrationAcceptanceTest --scan`. These tests do not expect there to be a separate deployment of Airbyte running.

These tests currently all live in `airbyte-tests`
These tests currently all live in [airbyte-tests](../.././airbyte-tests)

**Frontend Acceptance Tests**

These are acceptance tests for the frontend. They are run with `SUB_BUILD=PLATFORM ./gradlew --no-daemon :airbyte-webapp-e2e-tests:e2etest`. Like the Platform Acceptance Tests, they expect Airbyte to be running locally. See the [script](https://github.com/airbytehq/airbyte/blob/master/tools/bin/e2e_test.sh) that is used by the CI.
These are acceptance tests for the frontend. They are run with
```shell
SUB_BUILD=PLATFORM ./gradlew --no-daemon :airbyte-webapp-e2e-tests:e2etest
```
Like the Platform Acceptance Tests, they expect Airbyte to be running locally. See the [script](https://github.com/airbytehq/airbyte/blob/master/tools/bin/e2e_test.sh) that is used by the CI.


These tests currently all live in `airbyte-webapp-e2e-tests`.
These tests currently all live in [airbyte-webapp-e2e-tests](../.././airbyte-webapp-e2e-tests)

**Future Work**

Expand All @@ -109,13 +124,23 @@ Our story around "integration testing" or "E2E testing" is a little ambiguous. O

All connectors, regardless of implementation language, implement the following interface to allow uniformity in the build system when run from CI:

**Build connector, run unit tests, and build Docker image**: `./gradlew :airbyte-integrations:connectors:<name>:build` **Run integration tests**: `./gradlew :airbyte-integrations:connectors:<name>:integrationTest`
**Build connector, run unit tests, and build Docker image**:
```shell
./gradlew :airbyte-integrations:connectors:<connector_name>:build
```
**Run integration tests**:
```shell
./gradlew :airbyte-integrations:connectors:<connector_name>:integrationTest
```

#### Python

The ideal end state for a Python connector developer is that they shouldn't have to know Gradle exists.

We're almost there, but today there is only one Gradle command that's needed when developing in Python, used for formatting code.

**Formatting python module**: `./gradlew :airbyte-integrations:connectors:<name>:airbytePythonFormat`
**Formatting python module**:
```shell
./gradlew :airbyte-integrations:connectors:<name>:airbytePythonFormat
```