Inference after converting from PyTorch to Onnx to TensorRT

[kartik1395]

I've converted the PyTorch TSN model to Onnx and I'm planning to convert it to TensorRT. I wanted to know how can I make a prediction for a video input using the TensorRT model. Do I need to make any changes in the Prediction.py file ?

[innerlee]

Sorry which Prediction.py?

[kartik1395]

Sorry which Prediction.py?

tools/prediction.py

[kartik1395]

Sorry which Prediction.py?

Sorry I used https://github.com/open-mmlab/mmaction2/blob/master/docs/getting_started.md#high-level-apis-for-testing-a-video-and-rawframes to make my own prediction.py.

Nonetheless, could you guide on how to make predictions using TensorRT using similar Hight Level API ?

[innerlee]

Reference code: https://github.com/open-mmlab/mmaction2/blob/master/tools/pytorch2onnx.py#L77-L96

[kartik1395]

I see this is for a single image input if I'm not mistaken. How can it be replicated for a video file ? Sorry I'm a bit confused.

[innerlee]

Ignore the variable name one_img, it was copied from elsewhere. Input shape is determined by input_shape.

[kartik1395]

I understand. I've been using the high level API for inference that passes video file name as a parameter for making a prediction that prepares data using inference.py file https://github.com/open-mmlab/mmaction2/blob/master/mmaction/apis/inference.py.

Want to know how I can similarly pass the whole video through the TensorRT model.

[innerlee]

@dreamerlin maybe we can add support for this in inference.py? Like, adding a inference_recognizer_trt() function?

edit: trt

[kartik1395]

@dreamerlin maybe we can add support for this in inference.py? Like, adding a inference_recognizer_trt() function?

edit: trt

That would be really helpful, thank you!

[irvingzhang0512]

I plan to implement this in two months along with inference_recoginizer_onnx. May also implement a tool to test inference speed.

[dreamerlin]

Ref: open-mmlab/mmpretrain#153