Fix revoke Dag stale permission on airflow < 2.10 #42844
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Tested with airflow 2.9.3:
access_control={
"Viewer": {'can_read', 'can_edit', 'can_delete'}
}
access_control={
"Viewer": {'can_read', 'can_edit'}
}
|
2 tasks
joaopamaral
changed the title
vincbeck
approved these changes
Oct 10, 2024
|
Yeah. And it needs to be rebased after providers have been moved to top level. |
|
Would also be nice if there is a unit test for it. |
Co-authored-by: Ulada Zakharava <vlada_zakharava@epam.com>
- Label deprecated items with deprecated decorator. - Refactor vertex_ai AutoMLHook.create_auto_ml_text_training_job, to avoid calling get_auto_ml_text_training_job, to not trigger mypy errors after applying @deprecate on hook method. - Update docs. - Delete irrelevant system tests. Co-authored-by: Oleg Kachur <kachur@google.com>
Ensure that the EMR cluster stays running for long enough for all the tasks in the DAG to complete their operations against the cluster.
Co-authored-by: Ephraim Anierobi <splendidzigy24@gmail.com>
* Switch Ui codegen to fastAPI spec * Rebase * read access control from config --------- Co-authored-by: pierrejeambrun <pierrejbrun@gmail.com>
* Add documentation for FAB DB commands We recently separated FAB migration from Airflow's migration and added some commands for handling DB upgrade and downgrade in FAB. This PR adds user-facing documentation on how to upgrade and downgrade FAB and information about the different commands. I also added documentation on how contributors can hook their application's migration into Airflow's migration. The update_migration_references pre-commit was also updated to include FAB migrations. * fixup! Add documentation for FAB DB commands * fixup! fixup! Add documentation for FAB DB commands
This makes it so that we can alembic commands from the root repo dir. E.g. ``` alembic -c airflow/alembic.ini revision --autogenerate -m "Do something" ``` Sometimes when running from airflow subfolder we get errors.
This index is not needed because there's another index on the table that leads with dag_id.
This is just to ensure that we get each hook on a single line when running in split pane on macbook pro 14".
* Fix pre-commit for auto update of fab migration versions This fixes the pre-commit for updating the version in FAB migration and also, updated the actual version as 1.4 since that's going to be the next version At the same time, added Daniel's fix for airflow alembic.ini file * fixup! Fix pre-commit for auto update of fab migration versions * fixup! fixup! Fix pre-commit for auto update of fab migration versions
* Prepare docs for Sep 1st wave of providers * fix spelling * update fab
* Drop Python 3.8 support in core Add newsfragment * Drop Python 3.8 support in provider packages
…pache#42711) * Add ability to switch between table and card views for listing items * add test for DagCard and loading states * Add tests for loading states * PR feedback * use semantic tokens for reused colors * Remove eslint-ignore lost in rebase
…27.2 and pre-commit-hooks to v5.0.0 (apache#42809)
Previously we were getting 'PytestDeprecationWarning: The configuration option "asyncio_default_fixture_loop_scope" is unset.' This sets it to the new default, and makes the warning go away.
* feat: allow customizing podManagementPolicy * fix: adjust schema and identation * ci: adjust lint --------- Co-authored-by: Arakaki, Everton <everton.arakaki@rabobank.nl>
Previously it worked in general, but did not in certain test scenarios where we did not initialize the database. This is one solution.
related to apache#41816 Adds a warning log to indicate failure if length of task_key>100.
This is a no-op change right now, but as part of the provider re-org in apache#42505 this sets us up to be able to load the providers code in the tests The reason this change is done separately is that changes to breeze code form forks doesn't take effect, and this small change makes it easier to land on main without having to re-create that large PR.
This PR moves Scarf entry in Release notes at top of 2.10.0 entries and also adds URL/IP-address info that is covered by [Scarf's privacy policy](https://about.scarf.sh/privacy-policy).
* Add ability to provide proxy for dbt Cloud connection * Running pre-commit checks * Update current tests and add new test with proxy
I haven't been that active coding or reviewing. Although I am now returning back to focus on Task SDK (AIP-72) so narrowing down my entries in CODEOWNERS.
…42832) The [existing documentation](https://airflow.apache.org/docs/apache-airflow-providers-amazon/stable/secrets-backends/aws-secrets-manager.html#example-of-storing-google-secrets-in-aws-secrets-manager) on how to set up Google secrets in AWS Secrets Manager is out of date. It led me on a merry chase for hours. I hereby submit this PR to update the doc. I solemnly swear that the content has been verified using DAG code similar to this ```python gsheet = GSheetsHook(gcp_conn_id=gcp_conn_id) values = gsheet.get_values( spreadsheet_id=spreadsheet_id, range_=f"{sheet_name}!B1:B2", ) `
Just simplified redundant code from apache#42737
…ale-permission-on-version-2.9.x' into fix/fab-access-control-revoke-stale-permission-on-version-2.9.x
…on-on-version-2.9.x
…on-on-version-2.9.x
potiuk
approved these changes
Oct 25, 2024
This was referenced Oct 27, 2024
ellisms
pushed a commit
to ellisms/airflow
that referenced
this pull request
Nov 13, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
closes: #42743
This reported issue was happening on airflow < 2.10 using the old access_control format
dict[str, set]and it was captured by the testtests/providers/fab/auth_manager/test_security.pywhen run the airflow 2.9.3 with fab 1.4.0:After this fix the same test pass:
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.