fix Swin Transformer inplace mutation

[ain-soph]

Fix an issue that in previous codes, shift_size is modified in-place, which might modifies the attribute value self.shift_size as well.

cc @YosuaMichael

[ain-soph]

@YosuaMichael Do you think this fix requires a model retraining for the current released pretrained weights for Swin Transformer?

[YosuaMichael]

Thanks @ain-soph , the PR looks good!

I don't think we need to do any retraining since the model parameters layout is not changed.

[NicolasHug]

Thanks for the fix @ain-soph !

The layout isn't changed, but if self.shift_size was set to 0 in some cases as in the current main branch, it's still possible that this affects the training procedure (and thus the accuracy) eventually, in particular when

# If window size is larger than feature size

I don't really know enough to know whether that case gets hit during our trainings though.

[ain-soph]

@NicolasHug It should be okay. For each attn layer, its input size is actually fixed during training for ImageNet dataset because we resize the input images. If the shift_size is set to 0 for 1 image, it means shift_size should be set to 0 for all images.

And different blocks have individual shift_size attribute (while share the same window_size though), so I think it doesn't influence current training pipeline.

[YosuaMichael]

Just to give more context for @NicolasHug, so previously when we train the models the shift_size actually an int, therefore it doesnt have any in-place mutation danger. And actually I am the one introducing the bug in this PR: https://github.com/pytorch/vision/pull/6088/files#diff-58328880bdc0a38a5a29afcfedd7bca85a4b72631a5d0416d312f6505331deefL64
when I need the shift_size to be a list.

So this PR should fix the problem on that and the training behaviour should be more similar with the one when we train the weight.

[github-actions]

Hey @YosuaMichael!

You merged this PR, but no labels were added. The list of valid labels is available at https://github.com/pytorch/vision/blob/main/.github/process_commit.py