Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add helpful debug statements to Google Cloud Dataproc hook #41789

Closed
1 of 2 tasks
flacode opened this issue Aug 27, 2024 · 3 comments
Closed
1 of 2 tasks

Add helpful debug statements to Google Cloud Dataproc hook #41789

flacode opened this issue Aug 27, 2024 · 3 comments
Labels
area:providers good first issue kind:bug This is a clearly a bug provider:google Google (including GCP) related issues

Comments

@flacode
Copy link
Contributor

flacode commented Aug 27, 2024

Apache Airflow Provider(s)

google

Versions of Apache Airflow Providers

No response

Apache Airflow version

airflow-2.9.1

Operating System

NA

Deployment

Google Cloud Composer

Deployment details

No response

What happened

The various hooks in the Google Cloud Dataproc hook do not have debug statements that would make it easier for developers to identify and resolve issues related to passing incorrect field values to the APIs.

One specific example is the create_batch, method. By including a debug statement that prints the batch object, developers can quickly verify the object's contents and identify any discrepancies between the fields and the Dataproc API.

What you think should happen instead

No response

How to reproduce

Submit any DAG using Google Cloud Batch Operator

Anything else

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@flacode flacode added area:providers kind:bug This is a clearly a bug needs-triage label for new issues that we didn't triage yet labels Aug 27, 2024
@dosubot dosubot bot added the provider:google Google (including GCP) related issues label Aug 27, 2024
@potiuk potiuk added good first issue and removed needs-triage label for new issues that we didn't triage yet labels Aug 27, 2024
@potiuk
Copy link
Member

potiuk commented Aug 27, 2024

If you want - feel free to contribute - otherwise it will have to wait for someone to pick it up- marked as good first issue.

@gaurav-bakale
Copy link

If you want - feel free to contribute - otherwise it will have to wait for someone to pick it up- marked as good first issue.

I would like to pick it up. Can you please tell me whom to contact if I need some help while adding statements.

@eladkal
Copy link
Contributor

eladkal commented Oct 28, 2024

fixed in #43265

@eladkal eladkal closed this as completed Oct 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:providers good first issue kind:bug This is a clearly a bug provider:google Google (including GCP) related issues
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants