Skip to content

Commit

Permalink
Respond to review comments
Browse files Browse the repository at this point in the history
- Rename torch_gpu_suffix to torch_cuda_suffix
- Update documentation to be more specific about RedisAI and ML
  library dependencies
  • Loading branch information
ashao committed Oct 21, 2022
1 parent 5626143 commit 3613592
Show file tree
Hide file tree
Showing 3 changed files with 26 additions and 34 deletions.
41 changes: 15 additions & 26 deletions doc/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -59,36 +59,25 @@ Supported Versions
to support Windows.



SmartSim supports multiple machine learning libraries through
the use of RedisAI_ 1.2.7. The following libraries are supported (based on .

.. list-table:: Supported ML Libraries
:widths: 50 50 50 50
:header-rows: 1
:align: center

* - Library
- Versions
- Python Versions
- Built By Default
* - PyTorch_
- 1.11.0
- 3.7 - 3.9
- Yes
* - Tensorflow_ / Keras_
- 2.8.0
- 3.7 - 3.9
- Yes
* - ONNX_ Runtime
- 1.11.1
- 3.7 - 3.9
- No
Native support for various machine learning libraries and their
versions is dictated by our dependency on RedisAI_ 1.2.7. Users
can also select RedisAI 1.2.3 or 1.2.5 (though that also limits
the version of the ML libraries).

+------------------+----------+-------------+---------------+
| RedisAI | PyTorch | Tensorflow | ONNX Runtime |
+==================+==========+=============+===============+
| 1.2.7 (default) | 1.11.0 | 2.8.0 | 1.11.1 |
| 1.2.5 | 1.9.0 | 2.6.0 | 1.9.0 |
| 1.2.3 | 1.7.0 | 2.5.2 | 1.9.0 |
+------------------+----------+-------------+---------------+

TensorFlow_ 2.0 and Keras_ are supported through graph freezing_.

ScikitLearn_ and Spark_ models are supported by SmartSim as well
through the use of the ONNX_ runtime.
through the use of the ONNX_ runtime (which is not built by
default due to issues with glibc on a variety of Linux
platforms and lack of support for Mac OS X).

.. _Spark: https://spark.apache.org/mllib/
.. _Keras: https://keras.io
Expand Down
6 changes: 4 additions & 2 deletions smartsim/_core/_cli/build.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,14 +30,16 @@ def _install_torch_from_pip(versions, device="cpu", verbose=False):
if sys.platform == "darwin":
if device == "gpu":
logger.warning("GPU support is not available on Mac OS X")
# The following is deliberately left blank as there is no
# alternative package available on Mac OS X
device_suffix = ""
end_point = None

# if we are on linux cpu, either CUDA or CPU must be installed
elif sys.platform == "linux":
end_point = "https://download.pytorch.org/whl/torch_stable.html"
if device == "gpu":
device_suffix = versions.TORCH_GPU_SUFFIX
if device in ["gpu","cuda"] :
device_suffix = versions.TORCH_CUDA_SUFFIX
elif device == "cpu":
device_suffix = versions.TORCH_CPU_SUFFIX

Expand Down
13 changes: 7 additions & 6 deletions smartsim/_core/_install/buildenv.py
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@ class RedisAIVersion(Version_):
"scikit-learn": "1.0.2",
"torch": "1.7.1",
"torch_cpu_suffix": "+cpu",
"torch_gpu_suffix": "+cu110",
"torch_cuda_suffix": "+cu110",
"torchvision": "0.8.2",
},
"1.2.5": {
Expand All @@ -141,7 +141,7 @@ class RedisAIVersion(Version_):
"scikit-learn": "1.0.2",
"torch": "1.9.1",
"torch_cpu_suffix": "+cpu",
"torch_gpu_suffix": "+cu111",
"torch_cuda_suffix": "+cu111",
"torchvision": "0.10.1",
},
"1.2.7": {
Expand All @@ -152,7 +152,7 @@ class RedisAIVersion(Version_):
"scikit-learn": "1.1.1",
"torch": "1.11.0",
"torch_cpu_suffix": "+cpu",
"torch_gpu_suffix": "+cu113",
"torch_cuda_suffix": "+cu113",
"torchvision": "0.12.0",
},
}
Expand Down Expand Up @@ -223,7 +223,7 @@ class Versioner:
TORCH = Version_(get_env("SMARTSIM_TORCH", REDISAI.torch))
TORCHVISION = Version_(get_env("SMARTSIM_TORCHVIS", REDISAI.torchvision))
TORCH_CPU_SUFFIX = Version_(get_env("TORCH_CPU_SUFFIX", REDISAI.torch_cpu_suffix))
TORCH_GPU_SUFFIX = Version_(get_env("TORCH_GPU_SUFFIX", REDISAI.torch_gpu_suffix))
TORCH_CUDA_SUFFIX = Version_(get_env("TORCH_CUDA_SUFFIX", REDISAI.torch_cuda_suffix))

# TensorFlow and ONNX only use the defaults, but these are not built into
# the RedisAI package and therefore the user is free to pick other versions.
Expand Down Expand Up @@ -263,12 +263,13 @@ def ml_extras_required(self):
# remove torch-related fields as they will be installed
# by the cli process for use in the RAI build. We don't install
# them here as the user needs to decide between GPU/CPU. All other
# libraries work on both devices
# libraries work on both devices. The correct versions and suffixes
# were scraped from https://pytorch.org/get-started/previous-versions/
_torch_fields = [
"torch",
"torchvision",
"torch_cpu_suffix",
"torch_gpu_suffix"
"torch_cuda_suffix"
]
for field in _torch_fields:
ml_defaults.pop(field)
Expand Down

0 comments on commit 3613592

Please sign in to comment.