Qwen2VL, 报错 TypeError: sequence item 0: expected str instance, NoneType found

[dong-168]

Can you help me?

I've completed the previous operations, such as: 1 2 3
if [ "$choice" = "1" ]; then
python3 convert_checkpoint.py
--model_dir=${MODEL_PATH}
--output_dir=tmp/trt_models/${MODEL_NAME}/fp16/1-gpu
--dtype float16

elif [ "$choice" = "2" ]; then
trtllm-build --checkpoint_dir tmp/trt_models/${MODEL_NAME}/fp16/1-gpu
--output_dir tmp/trt_engines/${MODEL_NAME}/fp16/1-gpu/llm
--gemm_plugin=float16
--gpt_attention_plugin=float16
--max_batch_size=1
--max_input_len=2048
--max_seq_len=3072
--max_multimodal_len=1296
--max_num_tokens=$((3072 * 1))

elif [ "$choice" = "3" ]; then
python build_visual_engine.py
--model_type qwen2_vl --model_path ${MODEL_PATH}
--output_dir tmp/trt_engines/${MODEL_NAME}/fp16/1-gpu/vision

elif [ "$choice" = "4" ]; then
python3 run.py
--hf_model_dir ${MODEL_PATH}
--engine_dir tmp/trt_engines/${MODEL_NAME}/fp16/1-gpu/
--image_path /home/w/fsdownload/image/03568.jpg

Now, when I execute python3 run.py, it reports an error when reaching the following code in multimodal_model_runner.py:
output_beams_list = [
self.tokenizer.batch_decode(
output_ids[batch_idx, :, input_lengths[batch_idx]:],
skip_special_tokens=True
)
for batch_idx in range(self.args.batch_size)
]
Since my GPU only has 8GB of memory, I changed the type to float16 when running build_visual_engine.py before. Could this be related to the previous construction of the LLM or visual engine?

Error：
Traceback (most recent call last):
File "/home/w/文档/2.Qwen/TensorRT-main/Qwen2-VL-tensorRT/multimodal/run.py", line 125, in
input_text, output_text = model.run(args.input_text,
File "/home/w/anaconda3/envs/TRT/lib/python3.10/site-packages/tensorrt_llm/runtime/multimodal_model_runner.py", line 2016, in run
output_text = self.generate(pre_prompt,
File "/home/dong/anaconda3/envs/TRT/lib/python3.10/site-packages/tensorrt_llm/runtime/multimodal_model_runner.py", line 1108, in generate
output_beams_list = [
File "/home/w/anaconda3/envs/TRT/lib/python3.10/site-packages/tensorrt_llm/runtime/multimodal_model_runner.py", line 1109, in
self.tokenizer.batch_decode(
File "/home/w/anaconda3/envs/TRT/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 3907, in batch_decode
return [
File "/home/w/anaconda3/envs/TRT/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 3908, in
self.decode(
File "/home/w/anaconda3/envs/TRT/lib/python3.10/site-packages/transformers/models/qwen2/tokenization_qwen2.py", line 299, in decode
return super().decode(
File "/home/w/anaconda3/envs/TRT/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 3947, in decode
return self._decode(
File "/home/w/anaconda3/envs/TRT/lib/python3.10/site-packages/transformers/tokenization_utils.py", line 1104, in _decode
sub_texts.append(self.convert_tokens_to_string(current_sub_text))
File "/home/w/anaconda3/envs/TRT/lib/python3.10/site-packages/transformers/models/qwen2/tokenization_qwen2.py", line 285, in convert_tokens_to_string
text = "".join(tokens)
TypeError: sequence item 0: expected str instance, NoneType found
[TensorRT-LLM][INFO] Refreshed the MPI local session