Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improving validation of task retries to handle None values #42532

Merged
merged 5 commits into from
Oct 9, 2024

Conversation

sonu4578
Copy link
Contributor

@sonu4578 sonu4578 commented Sep 27, 2024


This PR fixes issue #42273 which causes an error when retries is set to None at task level

The issue has been fixed by improving the validation of task retries to handle None values.

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

Copy link

boring-cyborg bot commented Sep 27, 2024

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our pre-commits will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

sonu4578 and others added 2 commits September 28, 2024 10:37
Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com>
@uranusjr
Copy link
Member

Would it be better if task.retries always returns an int instead? We can fix this in BaseOperator.

@sonu4578
Copy link
Contributor Author

sonu4578 commented Oct 5, 2024

@uranusjr Thank you for the suggestion. I have added a change in BaseOperator for handling the None value to return a zero

@jscheffl jscheffl marked this pull request as ready for review October 9, 2024 18:45
@jscheffl jscheffl requested a review from uranusjr as a code owner October 9, 2024 18:45
@jscheffl jscheffl merged commit 1114ab2 into apache:main Oct 9, 2024
52 checks passed
Copy link

boring-cyborg bot commented Oct 9, 2024

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

@jscheffl jscheffl added this to the Airflow 2.10.3 milestone Oct 10, 2024
@jscheffl
Copy link
Contributor

Let's back-port this to 2.10!

closes: #42273

jscheffl pushed a commit that referenced this pull request Oct 10, 2024
* Improving validation of task retries to handle None values

* Updated the validation check to use the "or" logical operator

Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com>

* Added a comment for the new validation check

* Update baseoperator.py

* Update taskinstance.py

---------

Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com>
(cherry picked from commit 1114ab2)
@jscheffl jscheffl added type:bug-fix Changelog: Bug Fixes area:core labels Oct 10, 2024
jscheffl added a commit that referenced this pull request Oct 10, 2024
…42915)

* Improving validation of task retries to handle None values

* Updated the validation check to use the "or" logical operator

Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com>

* Added a comment for the new validation check

* Update baseoperator.py

* Update taskinstance.py

---------

Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com>
(cherry picked from commit 1114ab2)

Co-authored-by: sonu4578 <veetech4455@gmail.com>
kunaljubce pushed a commit to kunaljubce/airflow that referenced this pull request Oct 13, 2024
)

* Improving validation of task retries to handle None values

* Updated the validation check to use the "or" logical operator

Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com>

* Added a comment for the new validation check

* Update baseoperator.py

* Update taskinstance.py

---------

Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com>
pavansharma36 pushed a commit to pavansharma36/airflow that referenced this pull request Oct 14, 2024
)

* Improving validation of task retries to handle None values

* Updated the validation check to use the "or" logical operator

Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com>

* Added a comment for the new validation check

* Update baseoperator.py

* Update taskinstance.py

---------

Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com>
R7L208 pushed a commit to R7L208/airflow that referenced this pull request Oct 17, 2024
)

* Improving validation of task retries to handle None values

* Updated the validation check to use the "or" logical operator

Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com>

* Added a comment for the new validation check

* Update baseoperator.py

* Update taskinstance.py

---------

Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com>
utkarsharma2 pushed a commit that referenced this pull request Oct 23, 2024
…42915)

* Improving validation of task retries to handle None values

* Updated the validation check to use the "or" logical operator

Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com>

* Added a comment for the new validation check

* Update baseoperator.py

* Update taskinstance.py

---------

Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com>
(cherry picked from commit 1114ab2)

Co-authored-by: sonu4578 <veetech4455@gmail.com>
harjeevanmaan pushed a commit to harjeevanmaan/airflow that referenced this pull request Oct 23, 2024
)

* Improving validation of task retries to handle None values

* Updated the validation check to use the "or" logical operator

Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com>

* Added a comment for the new validation check

* Update baseoperator.py

* Update taskinstance.py

---------

Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com>
PaulKobow7536 pushed a commit to PaulKobow7536/airflow that referenced this pull request Oct 24, 2024
)

* Improving validation of task retries to handle None values

* Updated the validation check to use the "or" logical operator

Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com>

* Added a comment for the new validation check

* Update baseoperator.py

* Update taskinstance.py

---------

Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com>
utkarsharma2 pushed a commit that referenced this pull request Oct 24, 2024
…42915)

* Improving validation of task retries to handle None values

* Updated the validation check to use the "or" logical operator

Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com>

* Added a comment for the new validation check

* Update baseoperator.py

* Update taskinstance.py

---------

Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com>
(cherry picked from commit 1114ab2)

Co-authored-by: sonu4578 <veetech4455@gmail.com>
ellisms pushed a commit to ellisms/airflow that referenced this pull request Nov 13, 2024
)

* Improving validation of task retries to handle None values

* Updated the validation check to use the "or" logical operator

Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com>

* Added a comment for the new validation check

* Update baseoperator.py

* Update taskinstance.py

---------

Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:core type:bug-fix Changelog: Bug Fixes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants