Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stop raising RuntimeError when setting download=True (#8637) #8638

Closed
wants to merge 1 commit into from
Closed

Stop raising RuntimeError when setting download=True (#8637) #8638

wants to merge 1 commit into from

Conversation

imkasen
Copy link

@imkasen imkasen commented Sep 6, 2024

I fixed the problem (#8637) after reading the source code of torchvision.datasets.CIFAR10(). RuntimeError is no longer thrown when downloading, and the downloaded compressed package will be verified.

Copy link

pytorch-bot bot commented Sep 6, 2024

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/vision/8638

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot
Copy link

Hi @imkasen!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks!


def _download(self):
if self._check_exists():
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR @imkasen .

@pmeier do you remember why we're erroring instead of just doing an early return like in other datasets e.g. clevr ?

def _download(self) -> None:
if self._check_exists():
return

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The general idea was to raise on download=True if we can't properly verify the integrity. For example, after a folder is extracted, there is no way for us to know if the images inside this folder have not been tempered with.

Contrast this with datasets like MNIST or CIFAR, where we have checksums for each file on record. Here we can properly check the integrity and thus there is no need to raise on download=True.

Now, both CLEVRClassification and Imagenette fall into the first category. I don't know why we didn't follow our own "rule" for the former, but it is likely too late to change it now. However, I would treat "relaxing" the rule for the latter with caution. I don't think anything has changed that warrants walking back on our previous decision. However, if you do, you probably want to do it for all datasets to keep things consistent.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants