Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/requester pays #668

Merged
merged 27 commits into from
Jul 31, 2019
Merged

Feat/requester pays #668

merged 27 commits into from
Jul 31, 2019

Conversation

Avantol13
Copy link
Contributor

@Avantol13 Avantol13 commented Jul 29, 2019

New Features

  • new optional/configurable support for accessing Google buckets w/ requester pays enabled
  • allow configuration of default billing projects for accessing Google buckets w/ requester pays enabled
  • allow configuration for automatic billing permission assignment for Google service accounts to ease data access for Google buckets w/ requester pays enabled
  • add userProject field to Google signed urls if a billing project is provided (by user or by default configuration)
  • give Google service accounts billing permission to default billing project for Data Access Method: Temporary Service Account Credentials (if configured to do so)

Breaking Changes

Bug Fixes

Improvements

Dependency updates

  • cirrus to 1.1.0

Deployment changes

  • configuration added for requester pays Google Buckets. if this is needed, you'll need to update configuration. default config does not support requester pays bucket access

@github-actions
Copy link

The style in this PR agrees with black. ✔️

This formatting comment was generated automatically by a script in uc-cdis/wool.

@coveralls
Copy link

coveralls commented Jul 29, 2019

Pull Request Test Coverage Report for Build 7399

  • 21 of 50 (42.0%) changed or added relevant lines in 6 files are covered.
  • 1 unchanged line in 1 file lost coverage.
  • Overall coverage remained the same at ?%

Changes Missing Coverage Covered Lines Changed/Added Lines %
fence/blueprints/storage_creds/google.py 3 4 75.0%
fence/blueprints/data/indexd.py 7 9 77.78%
fence/blueprints/google.py 4 6 66.67%
fence/config.py 3 9 33.33%
fence/resources/google/utils.py 2 20 10.0%
Files with Coverage Reduction New Missed Lines %
fence/blueprints/google.py 1 68.01%
Totals Coverage Status
Change from base Build 7388: 0.0%
Covered Lines:
Relevant Lines: 0

💛 - Coveralls

Copy link
Contributor

@paulineribeyre paulineribeyre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't forget to update the PR description

Avantol13 and others added 8 commits July 30, 2019 15:24
Co-Authored-By: Pauline Ribeyre <ribeyre@uchicago.edu>
Co-Authored-By: Pauline Ribeyre <ribeyre@uchicago.edu>
Co-Authored-By: Pauline Ribeyre <ribeyre@uchicago.edu>
Co-Authored-By: Pauline Ribeyre <ribeyre@uchicago.edu>
Co-Authored-By: Pauline Ribeyre <ribeyre@uchicago.edu>
@@ -233,7 +206,7 @@ def give_service_account_billing_access_if_necessary(
is_default_billing = True

if r_pays_project:
sa_account_id = get_sa_email_from_private_key(sa_private_key)
sa_account_id = sa_private_key.get("client_email")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you could pull this out of the if instead of doing it once in each if

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants