Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci(datasets): Accelerate CI using uv #569

Merged
merged 13 commits into from
Mar 1, 2024
Merged

Conversation

astrojuanlu
Copy link
Member

Description

Development notes

  • Rename ill-named CI check for docs
  • Use uv to accelerate dependency installation on CI

Checklist

  • Opened this PR as a 'Draft Pull Request' if it is work-in-progress
  • Updated the documentation to reflect the code changes
  • Added a description of this change in the relevant RELEASE.md file
  • Added tests to cover my changes

@astrojuanlu astrojuanlu changed the title dev(datasets): Small CI improvements ci(datasets): Small CI improvements Feb 25, 2024
Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
@astrojuanlu astrojuanlu force-pushed the dev/small-ci-improvements branch 3 times, most recently from 478641b to d24f428 Compare February 26, 2024 00:04
@astrojuanlu
Copy link
Member Author

(uv internal error) when running pytest tests on Windows... 🤷🏽‍♂️

@astrojuanlu astrojuanlu changed the title ci(datasets): Small CI improvements ci(datasets): Accelerate CI using uv Feb 26, 2024
Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
See astral-sh/uv#1988

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
@astrojuanlu astrojuanlu force-pushed the dev/small-ci-improvements branch from 13a22d4 to 7789dc3 Compare February 26, 2024 18:17
Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
@astrojuanlu astrojuanlu marked this pull request as ready for review February 26, 2024 18:40
@astrojuanlu astrojuanlu reopened this Feb 26, 2024
Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
@astrojuanlu astrojuanlu force-pushed the dev/small-ci-improvements branch from 645f3cd to 0f280ec Compare February 26, 2024 19:03
@astrojuanlu
Copy link
Member Author

I could work around some problems of uv on Windows or I can just wait a few days for the team to address them 😬

@astrojuanlu astrojuanlu marked this pull request as draft February 26, 2024 19:17
@datajoely
Copy link
Contributor

Out of interest- what's the rough difference in time?

@astrojuanlu
Copy link
Member Author

In the worst case, from minutes to seconds

@astrojuanlu astrojuanlu marked this pull request as ready for review February 29, 2024 21:29
@astrojuanlu
Copy link
Member Author

This is now ready!

Copy link
Member

@merelcht merelcht left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very excited for this to go in 🚀 kedro next? 😄

Copy link
Contributor

@ankatiyar ankatiyar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀 🚀 🚀

@astrojuanlu
Copy link
Member Author

CI won't pass because of the ongoing kedro-docker breakage. So I'll wait for the rest of the checks to go green and then I'll merge.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
@astrojuanlu astrojuanlu force-pushed the dev/small-ci-improvements branch from d8b3842 to e8dc93e Compare March 1, 2024 15:41
@astrojuanlu astrojuanlu merged commit e2201c4 into main Mar 1, 2024
42 of 47 checks passed
@astrojuanlu astrojuanlu deleted the dev/small-ci-improvements branch March 1, 2024 19:44
tgoelles pushed a commit to tgoelles/kedro-plugins that referenced this pull request Jun 6, 2024
* Rename ill-named CI check for docs

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Use uv to accelerate dependency installation on CI

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Use smaller dataset for Hugging Face doctest

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Fix pytest invokation on Windows

See astral-sh/uv#1988

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Definitive fix to Hugging Face doctest

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Trigger CI

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Debug

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Bump to next uv version

* Restore pytest invokation

* Upgrade to latest version of uv

* Fix pyproject.toml

* Make docs check plugin-specific

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

---------

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Signed-off-by: tgoelles <thomas.goelles@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants