From 4190dbcba243bfda6903b40e94f8cf046b46b0f7 Mon Sep 17 00:00:00 2001 From: LiRen Tu Date: Thu, 6 Jan 2022 15:10:51 -0800 Subject: [PATCH] =?UTF-8?q?=F0=9F=93=96=20Add=20service=20account=20doc=20?= =?UTF-8?q?to=20gcs=20connector=20readme=20(#9347)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Update gcs readme about service accounts * Update doc --- .../connectors/destination-gcs/README.md | 29 +++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/airbyte-integrations/connectors/destination-gcs/README.md b/airbyte-integrations/connectors/destination-gcs/README.md index 6ad38446997d7..fbd7eaec1ccd4 100644 --- a/airbyte-integrations/connectors/destination-gcs/README.md +++ b/airbyte-integrations/connectors/destination-gcs/README.md @@ -17,6 +17,35 @@ As a community contributor, you can follow these steps to run integration tests. - Access the `destination gcs creds` secrets on Last Pass, and put it in `sample_secrets/config.json`. - Rename the directory from `sample_secrets` to `secrets`. +### GCP Service Account for Testing +Two service accounts have been created in our GCP for testing this destination. Both of them have access to Cloud Storage through HMAC keys. The keys are persisted together with the connector integration test credentials in LastPass. + +- Account: `gcs-destination-connector-test@dataline-integration-testing.iam.gserviceaccount.com` + - This account has the required permission to pass the integration test. Note that the uploader needs `storage.multipartUploads` permissions, which may not be intuitive. + - Role: `GCS Destination User` + - Permissions: + ``` + storage.multipartUploads.abort + storage.multipartUploads.create + storage.objects.create + storage.objects.delete + storage.objects.get + storage.objects.list + ``` + - LastPass entry: `destination gcs creds` + +- Account: `gcs-destination-failure-test@dataline-integration-testing.iam.gserviceaccount.com` + - This account does not have the `storage.multipartUploads` permissions, and will fail the integration test. The purpose of this account is to test that the `check` command can correctly detect the lack of these permissions and return an error message. + - Role: `GCS Destination User Without Multipart Permission` + - Permissions: + ``` + storage.objects.create + storage.objects.delete + storage.objects.get + storage.objects.list + ``` + - LastPass entry: `destination gcs creds (no multipart permission)` + ## Add New Output Format - Add a new enum in `S3Format`. - Modify `spec.json` to specify the configuration of this new format.