19 RC2 - Space usage is shown incorrectly

[SimplyCorbett]

Setup:
SSD for installation, CephFS for the data folder.
Nginx + HAProxy
7.4 PHP-FPM
Postgresql DB
19 RC2 Nextcloud - fresh install
Linux

There are no errors in the nextcloud.log and I'm not sure what other information is useful here.

---

Hi there!

Seems to be a typo here:

The correct wording should be 1.4TB of 8TB used. The percentage calculated appears to be correct.

[Nils160988]

cc @GretaD ?

[GretaD]

this is an interesting bug, because normally you should not have an hover tooltip, also the total space is in bracket. And here it shown in % where it should not. I have to check it out, because on 18th and lasted dev master version its shown correctly. thank you @dragon988 for the report

[GretaD]

this is an interesting bug, because normally you should not have an hover tooltip, also the total space is in bracket. And here it shown in % where it should not. I have to check it out, because on 18th and lasted dev master version its shown correctly. thank you @dragon988 for the report

i wrongly assumed you were talking about the user quota. So @dragon988 i think your problem is related to this one: #16943, can you please double check if this is your case by trying to reproduce #16943?

[SimplyCorbett]

i wrongly assumed you were talking about the user quota.

@GretaD

Yes, this is what I'm referring to. External storage is disabled.

Attached images.

[SimplyCorbett]

Also, running files:scan does not fix quota. It does update the space used though.

Edit: Not to get confused I manually moved the files outside of nextcloud for this test, it deletes and adds files correctly on it's own.

See attached.

[SimplyCorbett]

The admin area also always shows the correct quota settings. It's the user end that has issues.

[ggeorgg]

Same issue on my side, I think:
Personal settings:

I assigned 500GB of user space to my own admin account. Before that, I had 400GB but I ran out of space so I increased the value. Apparently this value didn't really get updated... Also I don't understand why it is 376GB. I never had a value like this.

Admin user configuration:

Files app:

I am on 19.0.1 now. I think I didn't see this problem on 18.x.x.

Also every time I switch to the files app I get a quota warning which I didn't get before (on 18.xx), at least I think so... The annoying this is, the quota warning gets triggered twice, but I guess this is off-topic here...

[ggeorgg]

Related/Same issue: #10693

In my case it has nothing to do with an external storage but I think I need to mention, that I am using btrfs as filesystem!
I guess this is not really tested and supported in nextcloud.

I had a look at the file where the quota calculation comes from.

server/lib/private/legacy/OC_Helper.php

Lines 521 to 558 in 2bc4094

	// TODO: need a better way to get total space from storage
	if ($sourceStorage->instanceOfStorage('\OC\Files\Storage\Wrapper\Quota')) {
	/** @var \OC\Files\Storage\Wrapper\Quota $storage */
	$quota = $sourceStorage->getQuota();
	}
	$free = $sourceStorage->free_space($rootInfo->getInternalPath());
	if ($free >= 0) {
	$total = $free + $used;
	} else {
	$total = $free; //either unknown or unlimited
	}
	if ($total > 0) {
	if ($quota > 0 && $total > $quota) {
	$total = $quota;
	}
	// prevent division by zero or error codes (negative values)
	$relative = round(($used / $total) * 10000) / 100;
	} else {
	$relative = 0;
	}
	$ownerId = $storage->getOwner($path);
	$ownerDisplayName = '';
	$owner = \OC::$server->getUserManager()->get($ownerId);
	if ($owner) {
	$ownerDisplayName = $owner->getDisplayName();
	}
	return [
	'free' => $free,
	'used' => $used,
	'quota' => $quota,
	'total' => $total,
	'relative' => $relative,
	'owner' => $ownerId,
	'ownerDisplayName' => $ownerDisplayName,
	'mountType' => $mount->getMountType()
	];

Correct command to get the estimated free filesystem size when using btrfs:

df -h does not give the correct free filesystem size when using btrfs:

I replaced

server/lib/private/legacy/OC_Helper.php

Line 553 in 2bc4094

'total' => $total,

with 'total' => $free,
and had a look in the files app in firefox to see whats happening. It gives me back a new "total" of 21.6GB

This value is far too small and wrong for the $free variable! It should be 727.20GB...

Because of this small value $total is smaller than $quota so that

server/lib/private/legacy/OC_Helper.php

Line 533 in 2bc4094

if ($quota > 0 && $total > $quota) {

is never true and the $total variable does not get the value of $quota in the following line.

Conclusion:

server/lib/private/legacy/OC_Helper.php

Line 526 in 2bc4094

$free = $sourceStorage->free_space($rootInfo->getInternalPath());

does not give back the correct value for $free.

Unfortunately I don't have the skills do correct this calculation. Can someone have a look please?

Thanks!

[ggeorgg]

@dragon988 Is it possible, that your CephFS is acting similar to my BTRFS and therefore we have the same problem? Can you check the $free variable like I did and compare it to the actual free space on your disk?

[SimplyCorbett]

@dragon988 Is it possible, that your CephFS is acting similar to my BTRFS and therefore we have the same problem? Can you check the $free variable like I did and compare it to the actual free space on your disk?

My problem has been resolved for some time and I'm currently using 19.0.2. I've kept this ticket open for you.

df -h does give the correct output for me.

[SimplyCorbett]

@ggeorgg I'm curious on what version of btrfs you are using along with distro. Have you filed a bug report upstream?

[ggeorgg]

I am currently using Open Suse Leap 15.2 with kernel version 5.3.18.
No, I did not file a separate bug report because this one did sound so similar to my problem.

By the way, I found an interesting behaviour: When I use the web UI of nextcloud and upload a small text file by drag and drop, the total space in the left bottom gets updated to the correct value. When I reload the page, it reverts back to the wrong value...

Does the function "free_space(...)" really use "df -h" to get the free space of the filesystem? Would it be possible to include a switch to use "btrfs filesystem usage /" when using btrfs as filesystem? How does this work?

[ggeorgg]

Ok. I guess we rely on the php function disk_free_space in here

server/lib/private/Files/Storage/Local.php

Line 372 in 41d95d9

$space = @disk_free_space($sourcePath);

In another github conversation (osclass/Osclass#533 (comment)) I found "we should not rely on this function on Linux BTRFS filesystems".
I think the market share of BTRFS still increases. For example Fedora will now have BTRFS as default filesystem. (https://itsfoss.com/btrfs-default-fedora/).
As BTRFS is a very nice filesystem it would be good to have a solution for this problem. Otherwise I guess I will have to use a more common setup...

I am not even sure if we really need the $free filesystem space to calculate the $total variable. In my opinion it is the task of the administrator to make sure the assigned quota values do not exceed the $free space...

[SimplyCorbett]

cc @icewind1991

[ggeorgg]

I got in contact with the php bug mailing list but didn't get an answer until now...
https://news-web.php.net/php.bugs/228877
Just referencing here in case someone else stumbles over it.

[ggeorgg]

Looks like php needs to use the function static int btrfs_statfs(struct dentry *dentry, struct kstatfs *buf) defined in https://elixir.bootlin.com/linux/v5.2.21/source/fs/btrfs/super.c#L2032 when the filesystem is btrfs.
I just don't know how to put it into the php code and I don't know if it is easily possible. I guess the modification needs to be done in this function in php: https://github.com/php/php-src/blob/master/ext/standard/filestat.c#L201

[zedocrob]

I also have the same problem when updating NC 18 to NC 19 on Odroid HC2 with BTRFS (on NextcloudPi)
tried to reformat the drive, same problem.

Normal size on NC18:

Wrong size on NC19:

So for now i'm stuck on NC 18

[zedocrob]

Looks like the problem is gone for me with latest NextcloudPi image and Nextcloud 20.0.4 update.

[szaimen]

Based on zedocrob's comment I am closing this. Feel free to open a new issue if you should still experience this on NC21.0.2 or above. Thanks!