Add SSE3 and fp16 conversion lookup table

[abitofevrything]

Adds SSE3 support for SIMD and support for using Imath for fp16-fp32 conversions. Imath can be faster on systems where whisper.cpp doesn't already have a native method for doing the conversion as it uses a lookup table, leading to an ~3.5x speed increase on my system.

[abitofevrything]

Drafting as I am unsure what value to put for GGML_F32_STEP and GGML_F16_STEP - guidance on this would be appreciated.

[abitofevrything]

A quick test seems to show that 32 leads to better performance than 16 or 64

[ggerganov]

A quick test seems to show that 32 leads to better performance than 16 or 64

Yes, that's what I do - trial and error to find the best value :)

This is a great contribution.
Before merging, I would like to avoid the Imath dependency.
We can simply generate a lookup table in ggml.c and use it instead of relying on Imath.
Take a look at the existing lookup tables for gelu and exp:

whisper.cpp/ggml.c

Lines 246 to 250 in a0d4f8e

	// precomputed gelu table for f16 (128 KB)
	static ggml_fp16_t table_gelu_f16[1 << 16];
	// precomputed exp table for f16 (128 KB)
	static ggml_fp16_t table_exp_f16[1 << 16];

I'm very curious to see if this F16 LUT will speed-up the WASM examples, because WASM does not have an intrinsic for FP16 <-> FP32 conversion so it fallbacks to the naive conversion method.

[abitofevrything]

Leaving as a draft for now as I want to see if I can get rid of some of the memcpy calls in the ggml_lookup_fp16_to_fo32 function.

A review would be appreciated as I am almost done with this though.

[abitofevrything]

Turns out the memcpy calls are optimised out by the compiler anyways :) Marking this as ready.

[ggerganov]

@abitofevrything
Good news! As expected, the lookup table improves the WASM performance.
On MacBook M1 Pro, I observe 25% faster using Firefox and 35% faster using Chrome