Getting convert-pt-to-ggml.py to work being an absolute ignorant

[github-bryld]

Humbly asking for some guidance here...

I have tried to get some Icelandic audio transcribed using the compiled multilingual ggml's but the results have been a bit disappointing.

I would very much like to try using an Icelandic-trained model instead and found this promising:
https://huggingface.co/Valdimarb13/whisper-small-icelandic

However, the provided pytorch_model.bin should be converted into ggml, and my attempts to do so using a portable hit&run solution with https://winpython.github.io/ have failed, as I have next to no idea of how exactly to do this (if I knew my way around python, I'd probably not be trying the cpp-way in the first place).

I struggle to see, where the paths should go from within my winpython-folder,
I'm unaquainted with the ~/. prefix, and where to obtain the medium.pt file in the .cace-path.
Also exactly where to put ~/path/to/repo/whisper/, and if the repo is both "whisper-main" or just the "whisper"-folder within it.
Also, what is meant with "You need to have the original models in ~/.cache/whisper/"? Does that really mean each and everyone of those huge files - even if I only want to convert the small version?

Sorry to ask so stupidly, but the instructions seem to presuppose a level of python-understanding, I simply do not possess.

Would anyone kindly try to lead me through these hoops, please?

[misutoneko]

Well, I don't know about winpython (I'm on Linux myself), but I can explain some things.
First of all, for Huggingface models you'll have to use the h5 variant of the script: convert-h5-to-ggml.py.
There are some instructions in the beginning of that script, well I guess you knew that but just in case...

About ~/.cache/whisper, I don't think it's needed for Huggingface models at all?
The tilde character ~ in the path is just a shortcut, it means user's home directory ($HOME env variable).
The dot in front of directory name means that it's a hidden directory.
But since you're on Windows I guess you can just forget about those and just use the full path names.

As for path locations - I think you can pretty much decide them yourself.
You just need to tell the script where they reside.
Here's the invocation I used on Linux:

python3 /mygitpath/whisper.cpp/models/convert-h5-to-ggml.py /mygitpath/whisper-small-icelandic/ /mygitpath/whisper_openai .

It's a slightly simplified version but hopefully you get the idea. So, just clone the repositories somewhere and give them as parameters for the script. As you can see, I use "whisper_openai" and not "whisper_openai/whisper" here. The script will complain about not finding assets, if it's not right.

One more thing, it seems the script is using a python module called transformers. So you'll need to install that somehow, if you haven't already.

Good luck!

[github-bryld]

I eventually gave up on winpython and instead installed a virtual Ubuntu on a Mac.

I then did the following:

git clone https://github.com/openai/whisper
git clone https://github.com/ggerganov/whisper.cpp
git clone https://huggingface.co/Valdimarb13/whisper-small-icelandic

python3 ./whisper.cpp/models/convert-h5-to-ggml.py ./whisper-small-icelandic/ ./whisper .

then it failed to import torch

Did I miss anything?

[misutoneko]

Yes you need to install some python modules. There are various ways to do it (I used pip3 myself), actually I'm a bit hesitant to give directions about python as I don't know all that much about it myself.
There's a bunch of imports that the script needs (grep for "import") but I guess most of them are installed when the python version of whisper is installed (yes torch is one them). Since I already had played with various pythonesque whisper versions I had those already, the only one that I didn't have was that transformers module.
Btw, some of these dependencies are huge. I guess that's the price to pay if you want to be a part of AI revolution with python :D

[github-bryld]

Well, I'm super-slowly advancing here, but things seem to push back at me.

Don't really see, why the system does not acknowledge that I have installed lfs now...

parallels@ubuntu-linux-22-04-desktop:~$ python3 ./whisper.cpp/models/convert-h5-to-ggml.py ./whisper-small-icelandic/ ./whisper .
Traceback (most recent call last):
  File "/home/parallels/.local/lib/python3.10/site-packages/transformers/modeling_utils.py", line 415, in load_state_dict
    return torch.load(checkpoint_file, map_location="cpu")
  File "/home/parallels/.local/lib/python3.10/site-packages/torch/serialization.py", line 815, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
  File "/home/parallels/.local/lib/python3.10/site-packages/torch/serialization.py", line 1033, in _legacy_load
    magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, 'v'.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/parallels/./whisper.cpp/models/convert-h5-to-ggml.py", line 89, in <module>
    model = WhisperForConditionalGeneration.from_pretrained(dir_model)
  File "/home/parallels/.local/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2429, in from_pretrained
    state_dict = load_state_dict(resolved_archive_file)
  File "/home/parallels/.local/lib/python3.10/site-packages/transformers/modeling_utils.py", line 420, in load_state_dict
    raise OSError(
OSError: You seem to have cloned a repository without having git-lfs installed. Please install git-lfs and run `git lfs install` followed by `git lfs pull` in the folder you cloned.
parallels@ubuntu-linux-22-04-desktop:~$ pip install git-lfs
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: git-lfs in ./.local/lib/python3.10/site-packages (1.6)
parallels@ubuntu-linux-22-04-desktop:~$ git lfs --help
git: 'lfs' is not a git command. See 'git --help'.

[misutoneko]

I suppose you should use your system's package manager to install git-lfs, rather than pip.
But git-lfs isn't a hard requirement. You can always download the model with a web browser (as there's only one file to download, so it's not overly inconvenient).

[github-bryld]

I got things working now.
Thanks for all your help and encouragement.

My ggml-converted icelandic-modified model is only half the size of the original, but that seems to be a ggml-feature. It's comparable with the other conversions, as far as I can see.

Unfortunately, this modified version seems just as bad in transcribing my Icelandic audio as the multiligual model. Maybe my audio is too complicated. It is an Icelandic comedy featuring a young girl speaking quite fast with a lot of swear words mixed in. It might be beyond, what the model was trained with (Icelandic parliamentary procedure as I recall).