Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When there are multiple engines on the classpath, DJL might randomly select a non-optimal one #3296

Closed
syskin345 opened this issue Jul 3, 2024 · 0 comments · Fixed by #3300
Labels
bug Something isn't working

Comments

@syskin345
Copy link

Description

See Platform::detectPlatform() method.

If you have multiple engines on your classpath, DJL will evaluate them in order given by classloader's listing of "native/lib/" + engine + ".properties" resources. The first acceptable engine is selected. The ClassLoader's resource listing is documented as "the ordering is not specified and may be very unpredictable"

This would be OK-ish, except the evaluation accepts a CPU engine even if the current system is CUDA capable. So if you have both, and the system is CUDA capable, you might or might not get CUDA from run to run.

Expected Behavior

If current platform is CUDA, find CUDA engine. Only if it's missing, use CPU engine.
In general, perhaps forcing the order to anything consistent might be a good idea.

I realise most systems will not have a random combination of engines, but in our case we distribute with CPU engine and drop CUDA engine as a plugin, and that's what we end up with.

@syskin345 syskin345 added the bug Something isn't working label Jul 3, 2024
@syskin345 syskin345 changed the title When there are multiple engines on the classpath, DJL will randomly select the wrong one When there are multiple engines on the classpath, DJL might randomly select a non-optimal one Jul 3, 2024
frankfliu added a commit to frankfliu/djl that referenced this issue Jul 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant