How to obtain the lip landmark

[wanyu42]

It seems that in the datasets/datatset.py, AudioVisualDataset expects also to see "landmarks" of the video, which I guess should refer to the lip landmark. However, I did not see any description on how to obtain the CREMA-D video landmark. Could you please illustrate further about how to obtain the audio encoding, how to organize the dataset folder structure, and how to include the landmark for training process?

[Yihuan-qaq]

似乎在 datasets/datatset.py 中，AudioVisualDataset 还希望看到视频的“地标”，我猜应该是指唇部地标。但是，我没有看到有关如何获取 CREMA-D 视频地标的任何描述。您能否进一步说明如何获取音频编码、如何组织数据集文件夹结构以及如何将地标包含在训练过程中？

I also encountered this problem, have you solved it?