-
Notifications
You must be signed in to change notification settings - Fork 186
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve performance of the push operation #88
Merged
laurilehmijoki
merged 29 commits into
laurilehmijoki:master
from
concord-consortium:fast-diff
May 6, 2014
Merged
Improve performance of the push operation #88
laurilehmijoki
merged 29 commits into
laurilehmijoki:master
from
concord-consortium:fast-diff
May 6, 2014
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
It will be necessary to calculate MD5 of gzipped files during diff calculation.
Now it's handled by gzip helper.
.gitkeep is excluded from upload to make cassette smaller / simpler.
Note that by default gzip includes timestamp, so it's not obvious that two gizpped files will have the same content.
Excellent work, thank you! I've released this pull request in the version 1.7.5. All s3_website users will benefit from this significant performance improvement. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
These changes improve performance of push operation (in particular the diff calculation). See #44.
Main changes:
ignore_on_server
to optimise diff.gzip
config option and push, all the files affected by this setting will be updated. We can consider it as a breaking change or kind of bug fix.3 is a separate pull request to filey-diff.
I also recorded almost all VCR tests again and did minor changes in their configuration. New tests are also added:
gzip
config change (what's related to 4.)I didn't update gem version, as I'm 100% not sure how. These changes are mostly about performance, but it also depends on how we treat change in behaviour described in 4.
We are already testing this branch to deploy a site that consists of about 15,000 files and it seems to work fine - previously the diff operation couldn't be completed due to performance issues.