Failed assertion in ggml-metal.m

[bjnortier]

I have been testing the SwiftUI example app and I'm getting the following error in ggml-metal.m when running on iPhone and Mac:

-[MTLDebugComputeCommandEncoder setThreadgroupMemoryLength:atIndex:]:817: failed assertion `length(4) must be a multiple of 16 bytes.'

at this line:

whisper.cpp/ggml-metal.m

Line 1033 in 0463028

[encoder setThreadgroupMemoryLength:nth/32*sizeof(float) atIndex:0];

since nth is 32 and 32/32*4=4 is not a multiple of 16.

If I remove the /32 it works but I'm not sure what the correct value should be.

Also, running the app without the debugger attached doesn't seem to generate a problem (the assertion doesn't fail).

[iandundas]

Thanks, I was just about to post this too!

[ggerganov]

There is a fix in llama.cpp : ggml-org/llama.cpp#3938

Will sync it here tomorrow

[bjnortier]

Just FYI to anyone else reading this I see the reason why it doesn't fail on Simulator is because use_gpu is false.

#if targetEnvironment(simulator)
        params.use_gpu = false
        print("Running on the simulator, using CPU")
#endif