-
-
Notifications
You must be signed in to change notification settings - Fork 4.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Very slow External Storage performance (via WebDAV or Local+dav2fs mount) #40338
Comments
The following additional logging is from Profiler: [ |
Any reason you're using it as a Local rather than WebDAV mount in External Storage? I only ask because (a) I'm curious how it compares; (b): in theory NC would then be aware the underlying filesystem doesn't really have Local filesystem characteristics. |
Mmm. I thought I went this route because a local mount would allow me to create separate External Storage mounts with different group permissions for different subdirectories.. And the speed issue is resolved I think! Thanks so much Josh. |
@joshtrichards - any thoughts? It seems like performance gets exponentially worse for larger directories - and perhaps after attribution / metadata is added to the underlying filesystem read? |
Unfortunately without an Alfresco server test bed it's challenging to do more than make scattered suggestions to try to isolate why this is occurring in your use case. Here are some ideas that come to mind: Rule out some file intensive apps (which also have background jobs...):Disable:
See if the access times changes with different data volumes
Eliminate debugging impacting on performanceDisable:
Set Different mount optionsOn the External Storage WebDAV mount under the three dots menu to the far right try some different combinations of:
Alfresco WebDAV endpointsIt looks like Alfresco has two different WebDAV endpoints to try: https://docs.alfresco.com/content-services/latest/admin/troubleshoot/#troubleshoot-webdav |
Thanks @joshtrichards I'll try those. It does seem strange considering the underlying performance in a shell and the initial read performance I experienced. There isn't anything in the app design resulting in iterative or recursive reads through the directory or tree is there? |
New experience. Accessing the external storage via the android app as opposed to Chrome or Edge browser (on separate windows laptop or on same phone as Android app). yields near file system access performance. Are you aware of a reason why accessing a large webdav mount would take long via browser vs the mobile app? Regards Marc |
Maybe #40473. Might be able to simulate a performance comparison (for the browser side) without having to update by adding |
Have set 'enable_file_metadata' => false Not sure there is much improvement. Bash Davfs mount. Very large directory with subdirectories. real 0m1.982s For equivalent External Files webdav mount: Browser returns variable performance. Now some strange performance on mobile app. Seems to cache previous results which it shows as you go into a directory. And then updates these. Subdirectories in the results may disappear if they have lots of nested content. If you click on them before the "cached" result is shown, then it gives an error that the content is no longer present. It seems that nested content below a webdav directory affects the speed at which that directory shows in the external file results. So even a high level result with 4 entries in the directory, subdirectory content affects speed. |
Is this something still affecting you? I'm using nextcloud on my Pi 4B (25.0.2) and the client app on Linux uploads files fine (60MiB in about 15 seconds) but the WebDAV integration with the KDE file manager uploads files at 300... bits per second. I'm really perplexed. I've opened a discussion on the KDE forums but considering the open tickets in the past and present here, it might be Nextcloud 🤔 |
Observation with Gnome/GVFS (or however the mounts done with GOA are called): |
Bug description
Browsing an External Storage mount (local - underlying dav2fs mount) of a directory containing 436 items takes 98 seconds to respond via the web interface. The same mount lists within a shell in 4,3s and via the same Nextcloud PHP executed via CURL in the shell in 6,2s.
The entire subdirectory contains 36GB of content when listed recursively (ie the full content of the External Storage mount).
This performance is on Nextcloud AIO Dockerised with almost all other apps disabled.
External Storage:
The underlying webdav is from an Alfresco instance.
(Profiler and debug logging enabled to diagnose this issue)
Events:
time ls -l /mnt/alfresco/Infobase/documentLibrary
real 0m4.277s
user 0m0.014s
sys 0m0.062s
time curl -u user:pass https://server/remote.php/dav/files/admin/Infobase/documentLibrary
real 0m6.207s
user 0m0.112s
sys 0m0.069s
time curl -X PROPFIND -H "Depth: 0" -u user:pass https://server/remote.php/dav/files/admin/Infobase/documentLibrary
real 0m5.420s
user 0m0.080s
sys 0m0.091s
I have read extensively and deleted authtoken entries in the database based on findings such as this:
#33453
I have also disabled almost all other apps with no impact. I have disabled LDAP user auth with no impact.
Nextcloud AIO is running in Docker on a dedicated server on a dedicated Xen VM.
The server has 256GB of memory, 4x Intel Xeon E5-4650 @ 3.5GHz and 3TB of SSD running as RAID5.
I am desperate to resolve this - accessing Alfresco via External storage offers a path to migration or co-existence.
The Alfresco server is a dedicated server running a dedicated Xen VM and Dockerised instance.
Both the Alfresco and Nextcloud servers are behind a Nginx reverse proxy (a further server) all on the same LAN and subnet.
Alfresco access on the Nextcloud server and from within the Docker container seems quick as evidenced by the shell access times above.
Steps to reproduce
2.Expose mount to Nextcloud container via Docker compose configuration.
3.Create External Storage local mount within Nextcloud.
Expected behavior
I would expect External Storage browsing response times in the Nextcloud Files app to be similar to those of the same underlying PHP script executed via CURL from the shell.
Installation method
Community Docker image
Nextcloud Server version
27
Operating system
Debian/Ubuntu
PHP engine version
PHP 8.1
Web server
Apache (supported)
Database engine version
PostgreSQL
Is this bug present after an update or on a fresh install?
Fresh Nextcloud Server install
Are you using the Nextcloud Server Encryption module?
Encryption is Disabled
What user-backends are you using?
Configuration report
List of activated Apps
Nextcloud Signing status
Nextcloud Logs
Additional info
No response
The text was updated successfully, but these errors were encountered: