May I ask someone for a review? Maybe @TomAugspurger @martindurant since you reviewed the last PRs? Thank you in advance.

Is it possible to test this change, something that would fail without it?

Hi @martindurant , thank you for the reply. In the original issue #462, we discovered that

There seems to be an issue when 2 instances of this file system write to the same blob storage from 2 different processes in parallel, where one of the uploads fails

I think it is not easy to write a pytest to show this issue explicitly.

I'm afraid I don't know anything about the function of these IDs... The problem was with assigning a constant ID to multiple uploads? I'm not sure I understand "who wins" in the case that two processes are writing at the same time.

Please add

from __future__ import annotations

Someone should also update the CI to a newer version of python, I would say at least 3.10.

Hi @martindurant ,

I'm afraid I don't know anything about the function of these IDs... The problem was with assigning a constant ID to multiple uploads? I'm not sure I understand "who wins" in the case that two processes are writing at the same time.

The problem happens even when two different blocks are being written to the blob storage at the same time, because the original implementation only take into account the number of blocks. This is the case we want to solve here.

Please add

from __future__ import annotations

Someone should also update the CI to a newer version of python, I would say at least 3.10.

d83487a should have fixed the pipeline for py38. Please run the pipeline again, thanks.

Rather than hashing the data (which might be expensive), would any random value do?