Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cache: improve diff ref release logic. #2563

Merged
merged 1 commit into from
Jan 19, 2022
Merged

Conversation

sipsma
Copy link
Collaborator

@sipsma sipsma commented Jan 17, 2022

Before this change, the lower and upper parents provided to the cache
manager Diff method were not cloned, which resulted in some code paths
incorrectly providing them directly as the parents of the returned ref.
This meant that if they were released after the call to Diff, the diff
ref could become incorrectly invalidated.

Now, the lower and upper are cloned and unit test coverage has been
added to test that ref release is handled correctly.

Signed-off-by: Erik Sipsma erik@sipsma.dev

@aaronlehmann
Copy link
Collaborator

I'm curious about the impacts of the bug this is fixing. How would it manifest to a user, and what's the severity?

@sipsma
Copy link
Collaborator Author

sipsma commented Jan 18, 2022

I'm curious about the impacts of the bug this is fixing. How would it manifest to a user, and what's the severity?

The bug could result in errors when trying to use diff refs, in which case the only sure-fire remediation would be to prune the cache of the diff ref, so I'd classify it as an important fix for any users of DiffOp. It could occur after a prune is run (manual or automatic), in which case the prune may release the cache entries for the diff ref's lower and upper input even if the diff ref is still lazy, making the diff ref unmountable. Whether that is a common case or not depends heavily on the use case, but I would say it would probably be bound to get hit by most users eventually w/out this fix.

Side note, this is different than the problem described in #2557. I found this bug while fixing that one, but wanted to get this out first as it's more severe and not a pre-existing issue.

@aaronlehmann
Copy link
Collaborator

aaronlehmann commented Jan 18, 2022 via email

Copy link
Member

@tonistiigi tonistiigi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add comments so that it is clear where the releases happens.

Eg.

  • on additional clones: "released in ...."
  • on createDiffRef call "createDiffRef takes ownership of the parents refs". (iiuc). Note that on public API these functions should not take ownership but clone internally so that tracking is easier. In private functions like these it can be more optimized with passing ownership.

@sipsma sipsma force-pushed the diffref-fix branch 2 times, most recently from 393e180 to de4ea73 Compare January 19, 2022 21:52
@sipsma
Copy link
Collaborator Author

sipsma commented Jan 19, 2022

Can you add comments so that it is clear where the releases happens.

Done

Copy link
Member

@tonistiigi tonistiigi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for nitpicking here. I think the failure cases (defer) are ok without special comment (unless specifically tricky case). Reader can usually understand the cases that are contained in the same function. It gets confusing when ownership is passed around to another function or retained in a struct and these can benefit from the extra info.

Before this change, the lower and upper parents provided to the cache
manager Diff method were not cloned, which resulted in some code paths
incorrectly providing them directly as the parents of the returned ref.
This meant that if they were released after the call to Diff, the diff
ref could become incorrectly invalidated.

Now, the lower and upper are cloned and unit test coverage has been
added to test that ref release is handled correctly.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
@sipsma sipsma merged commit eb473d0 into moby:master Jan 19, 2022
@crazy-max crazy-max added this to the v0.10.0 milestone Feb 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants