Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update designer module notebook and conda.yaml #1286

Merged
merged 3 commits into from
Jan 21, 2021
Merged

update designer module notebook and conda.yaml #1286

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
eb24272
update designer module notebook and conda.yaml
likebupt Jan 21, 2021
dce6e43
fix links in notebook
likebupt Jan 21, 2021
688e332
fix copyright
likebupt Jan 21, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
134 changes: 66 additions & 68 deletions examples/00_quick_start/sar_movieratings_with_azureml_designer.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,5 @@
{
"cells": [
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hey Blanca, some minor comments:

  1. we need to leave the copyright
  2. We are transitioning to change the principal branch from master to main, so https://github.com/microsoft/recommenders/blob/master/reco_utils/azureml/azureml_designer_modules/module_specs/ should be https://github.com/microsoft/recommenders/blob/master/reco_utils/azureml/azureml_designer_modules/module_specs/
  3. In this line there is an error with the SETUP link:
The first step is to install Azure CLI and Component CLI extension. Assuming that you have installed the Recommenders environment reco_base as explained in the [SETUP.md].

should be

The first step is to install Azure CLI and Component CLI extension. Assuming that you have installed the Recommenders environment reco_base as explained in the [SETUP.md](../../SETUP.md).

{
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blanca see these cells, they need to be in the notebook:

<i>Copyright (c) Microsoft Corporation. All rights reserved.</i>
<i>Licensed under the MIT License.</i>

"cell_type": "markdown",
"metadata": {},
"source": [
"<i>Copyright (c) Microsoft Corporation. All rights reserved.</i>\n",
"\n",
"<i>Licensed under the MIT License.</i>"
]
},
{
"cell_type": "markdown",
"metadata": {},
Expand All @@ -17,31 +8,11 @@
"\n",
"This notebook shows how to integrate any algorithm in Recommenders library into AzureML Designer. \n",
"\n",
"[AzureML Designer](https://docs.microsoft.com/en-us/azure/machine-learning/concept-designer) is a low-code/no-code solution \n",
"to visually build, test, and deploy machine learning models. Itsdrag-and-drop canvas allows to perform end to end machine learning tasks, such as:\n",
"\n",
"* Connect to various data sources including Azure blob, Azure Data Lake, Azure SQL or upload from a local file. One-click to preview and visualize the data profile. Preprocess data using a rich set of built-in modules for data transformation and feature engineering.\n",
"* Build and train machine learning models with state-of-the art machine learning and deep learning algorithms. AzureML Designer comes with a suite of prebuilt modules but it also allows to include Python and R scripts.\n",
"* Validate and evaluate model performance by interactively running machine learning pipelines. Preview logs and outputs for debugging and troubleshooting.\n",
"* Deploy models for real-time and batch inferencing as REST endpoints to your environment, with a few clicks. Automatically generate scoring files and the deployment image. Models and other assets are stored in the central registry for MLOps tracking and lineage.\n",
"[AzureML Designer](https://docs.microsoft.com/en-us/azure/machine-learning/concept-designer) lets you visually connect datasets and modules on an interactive canvas to create machine learning models. \n",
"\n",
"One of the features of AzureML Designer is that it is possible for developers to integrate any python library to make it available as a module/component. In this notebook are are going to show how to integrate [SAR](sar_movielens.ipynb) and several other modules in Designer.\n",
"\n",
"Note that custom module is renamed to component.\n",
"\n",
"\n",
"## Installation\n",
"\n",
"The first step is to install [Azure CLI](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli) and Module CLI extension. Assuming that you have installed the Recommenders environment `reco_base` as explained in the [SETUP.md](../../SETUP.md), you need to install:\n",
"```bash\n",
"conda activate reco_base\n",
"pip install azure-cli\n",
"# Uninstall azure-cli-ml (the `az ml` commands)\n",
"az extension remove -n azure-cli-ml\n",
"# Install local version of azure-cli-ml (which includes `az ml module` commands)\n",
"CLI_SDK_VERSION=26005222\n",
"az extension add --source https://azuremlsdktestpypi.azureedge.net/CLI-SDK-Runners-Validation/$CLI_SDK_VERSION/azure_cli_ml-0.1.0.$CLI_SDK_VERSION-py3-none-any.whl --pip-extra-index-urls https://azuremlsdktestpypi.azureedge.net/CLI-SDK-Runners-Validation/$CLI_SDK_VERSION --yes --verbose\n",
"```\n",
"**Note that custom module is renamed to component.**\n",
"\n",
"## Component implementation\n",
"\n",
Expand Down Expand Up @@ -102,7 +73,7 @@
"```\n",
"\n",
"\n",
"### Define module specification yaml\n",
"### Define component specification yaml\n",
"\n",
"Once we have the python entry, we need to create the yaml file that will interact with Designer, [precision_at_k.yaml](../../reco_utils/azureml/azureml_designer_modules/module_specs/precision_at_k.yaml).\n",
"\n",
Expand Down Expand Up @@ -182,18 +153,64 @@
"In the yaml file we can see a number of sections. The heading defines attributes like name, version or description. In the section inputs, all inputs are defined. The two main dataframes have ports, which can be connected to other modules. The inputs without port appear in a canvas menu. The output is defined as a DataFrame as well. The last section, implementation, defines the conda environment, the associated python entry and the arguments to the python file.\n",
"\n",
"\n",
"## Module Registration\n",
"# Module Registration\n",
"\n",
"Once the code is implemented, we need to register it as an AzureML Designer custom module. The registration can be performed following these simple steps:"
]
},
{
"source": [
"## Register in studio UI\n",
"\n",
"You can directly register a custom module/component in the studio UI.\n",
"\n",
"Follow [this tutorial](https://aka.ms/component-tutorial1) to register the related components to your workspace."
],
"cell_type": "markdown",
"metadata": {}
},
{
"source": [
"## Register using CLI\n",
"\n",
"### CLI Installation\n",
"\n",
"The first step is to install [Azure CLI](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli) and Component CLI extension. Assuming that you have installed the Recommenders environment `reco_base` as explained in the [SETUP.md](../../SETUP.md).\n"
],
"cell_type": "markdown",
"metadata": {}
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"conda activate reco_base\n",
"pip install azure-cli"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Login\n",
"!az login -o none"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!az login"
"# Uninstall azure-cli-ml (the `az ml` commands)\n",
"!az extension remove -n azure-cli-ml \n",
"\n",
"# Install remote version of azure-cli-ml (which includes `az ml component` commands)\n",
"!az extension add --source https://azuremlsdktestpypi.blob.core.windows.net/wheels/modulesdkpreview/azure_cli_ml-0.1.0.29211468-py3-none-any.whl --pip-extra-index-urls https://azuremlsdktestpypi.azureedge.net/modulesdkpreview --yes --verbose"
]
},
{
Expand All @@ -220,37 +237,14 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Start to register module spec: map.yaml ...\n",
"Done.\n",
"Start to register module spec: ndcg.yaml ...\n",
"Done.\n",
"Start to register module spec: precision_at_k.yaml ...\n",
"Done.\n",
"Start to register module spec: recall_at_k.yaml ...\n",
"Done.\n",
"Start to register module spec: sar_conda.yaml ...\n",
"Done.\n",
"Start to register module spec: sar_score.yaml ...\n",
"Done.\n",
"Start to register module spec: sar_train.yaml ...\n",
"Done.\n",
"Start to register module spec: stratified_splitter.yaml ...\n",
"Done.\n"
]
}
],
"outputs": [],
"source": [
"# Regsiter components with spec via Azure CLI\n",
"root_path = os.path.abspath(os.path.join(os.getcwd(), \"../../\"))\n",
"specs_folder = os.path.join(root_path, \"reco_utils/azureml/azureml_designer_modules/module_specs\")\n",
"github_prefix = 'https://github.com/microsoft/recommenders/blob/master/reco_utils/azureml/azureml_designer_modules/module_specs/'\n",
"github_prefix = 'https://github.com/microsoft/recommenders/blob/main/reco_utils/azureml/azureml_designer_modules/module_specs/'\n",
"specs = os.listdir(specs_folder)\n",
"for spec in specs:\n",
" spec_path = github_prefix + spec\n",
Expand All @@ -274,17 +268,21 @@
"## References\n",
"\n",
"1. [AzureML Designer documentation](https://docs.microsoft.com/en-us/azure/machine-learning/concept-designer)\n",
"1. [Tutorial: Prediction of automobile price](https://docs.microsoft.com/en-us/azure/machine-learning/tutorial-designer-automobile-price-train-score)\n",
"1. [Tutorial: Classification of time flight delays](https://docs.microsoft.com/en-us/azure/machine-learning/how-to-designer-sample-classification-flight-delay)\n",
"1. [Tutorial: Text classification of company categories](https://docs.microsoft.com/en-us/azure/machine-learning/how-to-designer-sample-text-classification)"
"1. [Tutorial: Prediction of automobile price](https://aka.ms/designer-tutorial)\n",
"1. [Tutorial: Register component to workspace](https://aka.ms/component-tutorial1)\n",
"1. [Tutorial: Create a new component](https://aka.ms/component-tutorial2)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
"name": "python3",
"display_name": "Python 3.6.8 64-bit ('test': conda)",
"metadata": {
"interpreter": {
"hash": "ad1389e27ccf93b6cb9b27912fdce5bd72b7d47f7c4b29627ffa9bc4b1e3e5d1"
}
}
},
"language_info": {
"codemirror_mode": {
Expand All @@ -296,9 +294,9 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.0"
"version": "3.6.8-final"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
}
1 change: 1 addition & 0 deletions reco_utils/azureml/azureml_designer_modules/module_specs/sar_conda.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ channels:
- defaults
dependencies:
- python=3.7
- pip=20.2
- pip:
- pandas>=0.23.4,<1.0.0
- pyarrow>=0.13.0
Expand Down