|
| 1 | +Using torchvision models in C++ |
| 2 | +=============================== |
| 3 | + |
| 4 | +This is a minimal example of getting TorchVision models to work in C++ with |
| 5 | +Torchscript. The model is first scripted in Python and exported to a file, and |
| 6 | +then loaded in C++. For a similar tutorial, see [this |
| 7 | +tutorial](https://pytorch.org/tutorials/advanced/cpp_export.html). |
| 8 | + |
| 9 | +In order to successfully compile this example, make sure you have ``LibTorch`` |
| 10 | +installed. You can either: |
| 11 | + |
| 12 | +- Install PyTorch normally |
| 13 | +- Or download the LibTorch C++ distribution. |
| 14 | + |
| 15 | +In both cases refer [here](https://pytorch.org/get-started/locally/) the |
| 16 | +corresponding install or download instructions. |
| 17 | + |
| 18 | +Some torchvision models only depend on PyTorch operators, and can be used in C++ |
| 19 | +without depending on the torchvision lib. Other models rely on torchvision's C++ |
| 20 | +operators like NMS, RoiAlign (typically the detection models) and those need to |
| 21 | +be linked against the torchvision lib. |
| 22 | + |
| 23 | +We'll first see the simpler case of running a model without the torchvision lib |
| 24 | +dependency. |
| 25 | + |
| 26 | +Running a model that doesn't need torchvision lib |
| 27 | +------------------------------------------------- |
| 28 | + |
| 29 | +Create a ``build`` directory inside the current one. |
| 30 | + |
| 31 | +```bash |
| 32 | +mkdir build |
| 33 | +cd build |
| 34 | +``` |
| 35 | + |
| 36 | +Then run `python ../trace_model.py` which should create a `resnet18.pt` file in |
| 37 | +the build directory. This is the scripted model that will be used in the C++ |
| 38 | +code. |
| 39 | + |
| 40 | +We can now start building with CMake. We have to tell CMake where it can find |
| 41 | +the necessary PyTorch resources. If you installed PyTorch normally, you can do: |
| 42 | + |
| 43 | +```bash |
| 44 | +TORCH_PATH=$(python -c "import pathlib, torch; print(pathlib.Path(torch.__path__[0]))") |
| 45 | +Torch_DIR="${TORCH_PATH}/share/cmake/Torch" # there should be .cmake files in there |
| 46 | +
|
| 47 | +cmake .. -DTorch_DIR=$Torch_DIR |
| 48 | +``` |
| 49 | + |
| 50 | +If instead you downloaded the LibTorch somewhere, you can do: |
| 51 | + |
| 52 | +```bash |
| 53 | +cmake .. -DCMAKE_PREFIX_PATH=/path/to/libtorch |
| 54 | +``` |
| 55 | + |
| 56 | +Then `cmake --build .` and you should now be able to run |
| 57 | + |
| 58 | +```bash |
| 59 | +./run_model resnet18.pt |
| 60 | +``` |
| 61 | + |
| 62 | +If you try to run the model with a model that depends on the torchvision lib, like |
| 63 | +`./run_model fasterrcnn_resnet50_fpn.pt`, you should get a runtime error. This is |
| 64 | +because the executable wasn't linked against the torchvision lib. |
| 65 | + |
| 66 | + |
| 67 | +Running a model that needs torchvision lib |
| 68 | +------------------------------------------ |
| 69 | + |
| 70 | +First, we need to build the torchvision lib. To build the torchvision lib go to |
| 71 | +the root of the torchvision project and run: |
| 72 | + |
| 73 | +```bash |
| 74 | +mkdir build |
| 75 | +cd build |
| 76 | +cmake .. -DCMAKE_PREFIX_PATH=/path/to/libtorch # or -DTorch_DIR= if you installed PyTorch normally, see above |
| 77 | +cmake --build . |
| 78 | +cmake --install . |
| 79 | +``` |
| 80 | + |
| 81 | +You may want to pass `-DCMAKE_INSTALL_PREFIX=/path/to/libtorchvision` for |
| 82 | +cmake to copy/install the files to a specific location (e.g. `$CONDA_PREFIX`). |
| 83 | + |
| 84 | +**DISCLAIMER**: the `libtorchvision` library includes the torchvision |
| 85 | +custom ops as well as most of the C++ torchvision APIs. Those APIs do not come |
| 86 | +with any backward-compatibility guarantees and may change from one version to |
| 87 | +the next. Only the Python APIs are stable and with backward-compatibility |
| 88 | +guarantees. So, if you need stability within a C++ environment, your best bet is |
| 89 | +to export the Python APIs via torchscript. |
| 90 | + |
| 91 | +Now that libtorchvision is built and installed we can tell our project to use |
| 92 | +and link to it via the `-DUSE_TORCHVISION` flag. We also need to tell CMake |
| 93 | +where to find it, just like we did with LibTorch, e.g.: |
| 94 | + |
| 95 | +```bash |
| 96 | +cmake .. -DTorch_DIR=$Torch_DIR -DTorchVision_DIR=path/to/libtorchvision -DUSE_TORCHVISION=ON |
| 97 | +cmake --build . |
| 98 | +``` |
| 99 | + |
| 100 | +Now the `run_model` executable should be able to run the |
| 101 | +`fasterrcnn_resnet50_fpn.pt` file. |
0 commit comments