[Cherrypick] Refactor swin transfomer so its component can be reused on the 3d version #6100
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…on (pytorch#6088) * Use List[int] instead of int for window_size and shift_size * Make PatchMerging and SwinTransformerBlock able to handle 2d and 3d cases * Separate patch embedding from SwinTransformer and enable to get model without head by specifying num_heads=None * Dont use if before padding so it is fx friendly * Put the handling on window_size edge cases on separate function and wrap with torch.fx.wrap so it is excluded from tracing * Update the weight url to the converted weight with new structure * Update the accuracy of swin_transformer * Change assert to Exception and nit * Make num_classes optional * Add typing output for _fix_window_and_shift_size function * init head to None to make it jit scriptable * Revert the change to make num_classes optional * Revert unneccesarry changes that might be risky * Remove self.head declaration
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Cherry pick #6088
This is useful so our current swin transformer 2d can be more compatible with the 3d version (developed in facebookresearch/multimodal#43)