Skip to content

Commit 628e527

Browse files
daveliepmannJose Luis Contreras
authored and
Jose Luis Contreras
committed
Improve clojure tutorial (apache#12974)
* Switch tutorial to dependency/ies that exist on Maven * Improve Clojure Module tutorial * Add namespace docstring * Bring verbiage up to date with https://mxnet.incubator.apache.org/api/clojure/module.html * Add newlines for readability and to keep line length <80 * Nix duplicated section in Clojure Symbol API docs "Multiple Outputs" is a (deprecated) repeat of "Group Multiple Symbols". * Improve Clojure Symbol tutorial * Add namespace docstring * Bring verbiage up to date with https://mxnet.incubator.apache.org/api/clojure/symbol.html * Add newlines for readability and to keep line length <80 * Fix missing end-code-block in Clojure NDArray API docs * Improve Clojure NDArray tutorial * Add namespace docstring * Bring verbiage up to date with https://mxnet.incubator.apache.org/api/clojure/ndarray.html * Add newlines for readability and to keep line length <80 * Improve Clojure KVStore tutorial * Add namespace docstring * Bring verbiage up to date with https://mxnet.incubator.apache.org/api/clojure/kvstore.html * Add newlines for readability and to keep line length <80 * [MXNET-1017] Updating the readme file for cpp-package and adding readme file for example directory. (apache#12773) * Updating the readme file for cpp-package and adding readme file for example directory. * Updating the readme file for cpp-package and adding readme file for example directory. * Addressed the review comments. * Addressed the review comments * Fail the broken link job when broken links are found (apache#12905) * Fix typo in formula in docstring for GRU cell and layer and add clarification to description (gluon.rnn) (apache#12896) * Fix typo in GRU cell and layers (gluon.rnn) docstring * empty * fix the paths issue for downloading script (apache#12913)
1 parent 93844d2 commit 628e527

File tree

7 files changed

+250
-179
lines changed

7 files changed

+250
-179
lines changed

contrib/clojure-package/examples/tutorial/project.clj

+4-1
Original file line numberDiff line numberDiff line change
@@ -19,4 +19,7 @@
1919
:description "MXNET tutorials"
2020
:plugins [[lein-cljfmt "0.5.7"]]
2121
:dependencies [[org.clojure/clojure "1.9.0"]
22-
[org.apache.mxnet.contrib.clojure/clojure-mxnet "1.3.1-SNAPSHOT"]])
22+
;; Uncomment the one appropriate for your machine & configuration:
23+
#_[org.apache.mxnet.contrib.clojure/clojure-mxnet-linux-cpu "1.3.0"]
24+
#_[org.apache.mxnet.contrib.clojure/clojure-mxnet-linux-gpu "1.3.0"]
25+
#_[org.apache.mxnet.contrib.clojure/clojure-mxnet-osx-cpu "1.3.0"]])

contrib/clojure-package/examples/tutorial/src/tutorial/kvstore.clj

+40-20
Original file line numberDiff line numberDiff line change
@@ -16,35 +16,44 @@
1616
;;
1717

1818
(ns tutorial.kvstore
19+
"A REPL tutorial of the MXNet Clojure API for KVStore, based on
20+
https://mxnet.incubator.apache.org/api/clojure/kvstore.html"
1921
(:require [org.apache.clojure-mxnet.kvstore :as kvstore]
2022
[org.apache.clojure-mxnet.ndarray :as ndarray]
2123
[org.apache.clojure-mxnet.context :as context]))
2224

23-
;;Basic Push and Pull
24-
;;Provides basic operation over multiple devices (GPUs or CPUs) on a single device.
2525

26-
;; Initialization
27-
;; Let’s consider a simple example. It initializes a (int, NDArray) pair into the store, and then pulls the value out.
26+
;;;; Basic Push and Pull
2827

29-
(def kv (kvstore/create "local")) ;; create a local kvstore
28+
;; Provides basic operation over multiple devices (GPUs or CPUs) on a
29+
;; single device.
30+
31+
;;; Initialization
32+
;; Let’s consider a simple example. It initializes a (`int`,
33+
;; `NDArray`) pair into the store, and then pulls the value out.
34+
35+
(def kv (kvstore/create "local")) ; create a local kvstore
3036
(def shape [2 3])
31-
;;; init the kvstore with a vector of keys (strings) and ndarrays
37+
;; init the kvstore with a vector of keys (strings) and ndarrays
3238
(kvstore/init kv ["3"] [(ndarray/* (ndarray/ones shape) 2)])
3339
(def a (ndarray/zeros shape))
3440
(kvstore/pull kv ["3"] [a])
3541
(ndarray/->vec a) ;=> [2.0 2.0 2.0 2.0 2.0 2.0]
3642

3743

38-
;;Push, Aggregation, and Updater
39-
;;For any key that’s been initialized, you can push a new value with the same shape to the key, as follows:
40-
44+
;;; Push, Aggregation, and Updater
45+
;; For any key that’s been initialized, you can push a new value with
46+
;; the same shape to the key, as follows:
4147
(kvstore/push kv ["3"] [(ndarray/* (ndarray/ones shape) 8)])
4248
(kvstore/pull kv ["3"] [a])
4349
(ndarray/->vec a);=>[8.0 8.0 8.0 8.0 8.0 8.0]
4450

45-
;;The data that you want to push can be stored on any device. Furthermore, you can push multiple values into the same key, where KVStore first sums all of these values, and then pushes the aggregated value, as follows:
51+
;; The data that you want to push can be stored on any
52+
;; device. Furthermore, you can push multiple values into the same
53+
;; key, where KVStore first sums all of these values, and then pushes
54+
;; the aggregated value, as follows:
4655

47-
;; using multiple cpus instead of gpus
56+
;; (Here we use multiple CPUs.)
4857
(def cpus [(context/cpu 0) (context/cpu 1) (context/cpu 2)])
4958
(def b [(ndarray/ones shape {:ctx (nth cpus 0)})
5059
(ndarray/ones shape {:ctx (nth cpus 1)})
@@ -53,22 +62,33 @@
5362
(kvstore/pull kv "3" a)
5463
(ndarray/->vec a) ;=> [3.0 3.0 3.0 3.0 3.0 3.0]
5564

56-
57-
;;Pull
58-
;;You’ve already seen how to pull a single key-value pair. Similar to the way that you use the push command, you can pull the value into several devices with a single call.
65+
;;; Pull
66+
;; You’ve already seen how to pull a single key-value pair. Similar to
67+
;; the way that you use the push command, you can pull the value into
68+
;; several devices with a single call.
5969
(def b [(ndarray/ones shape {:ctx (context/cpu 0)})
6070
(ndarray/ones shape {:ctx (context/cpu 1)})])
6171
(kvstore/pull kv ["3" "3"] b)
6272
(map ndarray/->vec b) ;=> ([3.0 3.0 3.0 3.0 3.0 3.0] [3.0 3.0 3.0 3.0 3.0 3.0])
6373

64-
;;List Key-Value Pairs
65-
;;All of the operations that we’ve discussed so far are performed on a single key. KVStore also provides the interface for generating a list of key-value pairs. For a single device, use the following:
74+
75+
;;;; List Key-Value Pairs
76+
77+
;; All of the operations that we’ve discussed so far are performed on
78+
;; a single key. KVStore also provides the interface for generating a
79+
;; list of key-value pairs. For a single device, use the following:
6680

6781
(def ks ["5" "7" "9"])
68-
(kvstore/init kv ks [(ndarray/ones shape) (ndarray/ones shape) (ndarray/ones shape)])
69-
(kvstore/push kv ks [(ndarray/ones shape) (ndarray/ones shape) (ndarray/ones shape)])
70-
(def b [(ndarray/zeros shape) (ndarray/zeros shape) (ndarray/zeros shape)])
82+
(kvstore/init kv ks [(ndarray/ones shape)
83+
(ndarray/ones shape)
84+
(ndarray/ones shape)])
85+
(kvstore/push kv ks [(ndarray/ones shape)
86+
(ndarray/ones shape)
87+
(ndarray/ones shape)])
88+
(def b [(ndarray/zeros shape)
89+
(ndarray/zeros shape)
90+
(ndarray/zeros shape)])
7191
(kvstore/pull kv ks b)
72-
(map ndarray/->vec b);=> ([1.0 1.0 1.0 1.0 1.0 1.0] [1.0 1.0 1.0 1.0 1.0 1.0] [1.0 1.0 1.0 1.0 1.0 1.0])
92+
(map ndarray/->vec b) ;=> ([1.0 1.0 1.0 1.0 1.0 1.0] [1.0 1.0 1.0 1.0 1.0 1.0] [1.0 1.0 1.0 1.0 1.0 1.0])
7393

7494

contrib/clojure-package/examples/tutorial/src/tutorial/module.clj

+96-40
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,8 @@
1616
;;
1717

1818
(ns tutorial.module
19+
"A REPL tutorial of the MXNet Clojure API for Module, based on
20+
https://mxnet.incubator.apache.org/api/clojure/module.html"
1921
(:require [clojure.java.io :as io]
2022
[clojure.java.shell :refer [sh]]
2123
[org.apache.clojure-mxnet.eval-metric :as eval-metric]
@@ -24,12 +26,26 @@
2426
[org.apache.clojure-mxnet.symbol :as sym]
2527
[org.apache.clojure-mxnet.ndarray :as ndarray]))
2628

29+
30+
;; The Module API provides an intermediate and high-level interface
31+
;; for performing computation with neural networks in MXNet. Module
32+
;; wraps a Symbol and one or more Executors. It has both a high level
33+
;; and intermediate level API.
34+
35+
36+
;;;; Prepare the Data
37+
38+
;; In this example, we are going to use the MNIST data set. If you
39+
;; start, we can run some helper scripts to download the data for us.
40+
2741
(def data-dir "data/")
2842

2943
(when-not (.exists (io/file (str data-dir "train-images-idx3-ubyte")))
3044
(sh "../../scripts/get_mnist_data.sh"))
3145

32-
;;; Load the MNIST datasets
46+
;; MXNet provides function in the `io` namespace to load the MNIST
47+
;; datasets into training and test data iterators that we can use with
48+
;; our module.
3349
(def train-data (mx-io/mnist-iter {:image (str data-dir "train-images-idx3-ubyte")
3450
:label (str data-dir "train-labels-idx1-ubyte")
3551
:label-name "softmax_label"
@@ -47,11 +63,13 @@
4763
:flat true
4864
:silent false}))
4965

50-
;; The module API provides an intermediate and high-level interface for performing computation with neural networks in MXNet. Module wraps a Symbol and one or more Executors. It has both a high level and intermediate level api
5166

52-
;; Preparing a module for Computation
67+
;;;; Preparing a module for Computation
5368

54-
;; construct a module
69+
;; To construct a module, we need to have a symbol as input. This
70+
;; symbol takes input data in the first layer and then has subsequent
71+
;; layers of fully connected and relu activation layers, ending up in
72+
;; a softmax layer for output.
5573

5674
(let [data (sym/variable "data")
5775
fc1 (sym/fully-connected "fc1" {:data data :num-hidden 128})
@@ -62,7 +80,7 @@
6280
out (sym/softmax-output "softmax" {:data fc3})]
6381
out) ;=>#object[org.apache.mxnet.Symbol 0x1f43a406 "org.apache.mxnet.Symbol@1f43a406"]
6482

65-
;; You can also use as-> for easier threading
83+
;; You can also write this with the `as->` threading macro.
6684

6785

6886
(def out (as-> (sym/variable "data") data
@@ -75,40 +93,62 @@
7593
;=> #'tutorial.module/out
7694

7795

78-
;; By default, context is the CPU. If you need data parallelization, you can specify a GPU context or an array of GPU contexts.
79-
;; like this (m/module out {:contexts [(context/gpu)]})
96+
;; By default, context is the CPU. If you need data parallelization,
97+
;; you can specify a GPU context or an array of GPU contexts, like
98+
;; this: `(m/module out {:contexts [(context/gpu)]})`
8099

81-
;; Before you can compute with a module, you need to call `bind` to allocate the device memory and `initParams` or `set-params` to initialize the parameters. If you simply want to fit a module, you don’t need to call `bind` and `init-params` explicitly, because the `fit` function automatically calls them if they are needed.
100+
;; Before you can compute with a module, you need to call `bind` to
101+
;; allocate the device memory and `initParams` or `set-params` to
102+
;; initialize the parameters. If you simply want to fit a module, you
103+
;; don’t need to call `bind` and `init-params` explicitly, because the
104+
;; `fit` function automatically calls them if they are needed.
82105

83106
(let [mod (m/module out)]
84107
(-> mod
85108
(m/bind {:data-shapes (mx-io/provide-data train-data)
86109
:label-shapes (mx-io/provide-label train-data)})
87110
(m/init-params)))
88111

89-
;; Now you can compute with the module using functions like `forward`, `backward`, etc.
112+
;; Now you can compute with the module using functions like `forward`,
113+
;; `backward`, etc.
90114

91115

92-
;; Training, Predicting, and Evaluating
116+
;;;; Training and Predicting
93117

94-
;;Modules provide high-level APIs for training, predicting, and evaluating. To fit a module, call the `fit` function with some DataIters:
118+
;; Modules provide high-level APIs for training, predicting, and
119+
;; evaluating. To fit a module, call the `fit` function with some data
120+
;; iterators:
95121

96-
(def mod (m/fit (m/module out) {:train-data train-data :eval-data test-data :num-epoch 1}))
122+
(def mod
123+
(m/fit (m/module out) {:train-data train-data
124+
:eval-data test-data
125+
:num-epoch 1}))
126+
;; =>
97127
;; Epoch 0 Train- [accuracy 0.12521666]
98128
;; Epoch 0 Time cost- 8392
99129
;; Epoch 0 Validation- [accuracy 0.2227]
100130

101131

102-
;; You can pass in batch-end callbacks using batch-end-callback and epoch-end callbacks using epoch-end-callback in the `fit-params`. You can also set parameters using functions like in the fit-params like optimizer and eval-metric. To learn more about the fit-params, see the fit-param function options. To predict with a module, call `predict` with a DataIter:
132+
;; You can pass in batch-end callbacks using batch-end-callback and
133+
;; epoch-end callbacks using epoch-end-callback in the
134+
;; `fit-params`. You can also set parameters using functions like in
135+
;; the fit-params like optimizer and eval-metric. To learn more about
136+
;; the fit-params, see the fit-param function options. To predict with
137+
;; a module, call `predict` with a DataIter:
138+
139+
(def results
140+
(m/predict mod {:eval-data test-data}))
103141

104-
(def results (m/predict mod {:eval-data test-data}))
105142
(first results) ;=>#object[org.apache.mxnet.NDArray 0x3540b6d3 "org.apache.mxnet.NDArray@a48686ec"]
106143

107144
(first (ndarray/->vec (first results))) ;=>0.08261358
108145

109-
;;The module collects and returns all of the prediction results. For more details about the format of the return values, see the documentation for the `predict` function.
146+
;; The module collects and returns all of the prediction results. For
147+
;; more details about the format of the return values, see the
148+
;; documentation for the `predict` function.
110149

111-
;;When prediction results might be too large to fit in memory, use the `predict-every-batch` API
150+
;; When prediction results might be too large to fit in memory, use
151+
;; the `predict-every-batch` API.
112152

113153
(let [preds (m/predict-every-batch mod {:eval-data test-data})]
114154
(mx-io/reduce-batches test-data
@@ -118,23 +158,33 @@
118158
;;; do something
119159
(inc i))))
120160

121-
;;If you need to evaluate on a test set and don’t need the prediction output, call the `score` function with a DataIter and an EvalMetric:
161+
;; If you need to evaluate on a test set and don’t need the prediction
162+
;; output, call the `score` function with a data iterator and an eval
163+
;; metric:
122164

123-
(m/score mod {:eval-data test-data :eval-metric (eval-metric/accuracy)}) ;=>["accuracy" 0.2227]
165+
(m/score mod {:eval-data test-data
166+
:eval-metric (eval-metric/accuracy)}) ;=>["accuracy" 0.2227]
124167

125-
;;This runs predictions on each batch in the provided DataIter and computes the evaluation score using the provided EvalMetric. The evaluation results are stored in metric so that you can query later.
168+
;; This runs predictions on each batch in the provided DataIter and
169+
;; computes the evaluation score using the provided EvalMetric. The
170+
;; evaluation results are stored in metric so that you can query
171+
;; later.
126172

127-
;;Saving and Loading Module Parameters
128173

129-
;;To save the module parameters in each training epoch, use a `checkpoint` function
130174

175+
;;;; Saving and Loading
176+
177+
;; To save the module parameters in each training epoch, use the
178+
;; `save-checkpoint` function:
131179

132180
(let [save-prefix "my-model"]
133181
(doseq [epoch-num (range 3)]
134182
(mx-io/do-batches train-data (fn [batch
135183
;; do something
136-
]))
137-
(m/save-checkpoint mod {:prefix save-prefix :epoch epoch-num :save-opt-states true})))
184+
]))
185+
(m/save-checkpoint mod {:prefix save-prefix
186+
:epoch epoch-num
187+
:save-opt-states true})))
138188

139189
;; INFO org.apache.mxnet.module.Module: Saved checkpoint to my-model-0000.params
140190
;; INFO org.apache.mxnet.module.Module: Saved optimizer state to my-model-0000.states
@@ -144,20 +194,22 @@
144194
;; INFO org.apache.mxnet.module.Module: Saved optimizer state to my-model-0002.states
145195

146196

147-
;;To load the saved module parameters, call the `load-checkpoint` function:
197+
;; To load the saved module parameters, call the `load-checkpoint`
198+
;; function:
148199

149200
(def new-mod (m/load-checkpoint {:prefix "my-model" :epoch 1 :load-optimizer-states true}))
150201

151202
new-mod ;=> #object[org.apache.mxnet.module.Module 0x5304d0f4 "org.apache.mxnet.module.Module@5304d0f4"]
152203

153-
;;To initialize parameters, Bind the symbols to construct executors first with bind function. Then, initialize the parameters and auxiliary states by calling `init-params` function.
154-
204+
;; To initialize parameters, bind the symbols to construct executors
205+
;; first with the `bind` function. Then, initialize the parameters and
206+
;; auxiliary states by calling the `init-params` function.\
155207
(-> new-mod
156-
(m/bind {:data-shapes (mx-io/provide-data train-data) :label-shapes (mx-io/provide-label train-data)})
208+
(m/bind {:data-shapes (mx-io/provide-data train-data)
209+
:label-shapes (mx-io/provide-label train-data)})
157210
(m/init-params))
158211

159-
;;To get current parameters, use `params`
160-
212+
;; To get current parameters, use `params`
161213
(let [[arg-params aux-params] (m/params new-mod)]
162214
{:arg-params arg-params
163215
:aux-params aux-params})
@@ -178,20 +230,24 @@ new-mod ;=> #object[org.apache.mxnet.module.Module 0x5304d0f4 "org.apache.mxnet.
178230
;; :aux-params {}}
179231

180232

181-
;;To assign parameter and aux state values, use `set-params` function.
233+
;; To assign parameter and aux state values, use the `set-params`
234+
;; function:
235+
(m/set-params new-mod {:arg-params (m/arg-params new-mod)
236+
:aux-params (m/aux-params new-mod)})
182237

183-
(m/set-params new-mod {:arg-params (m/arg-params new-mod) :aux-params (m/aux-params new-mod)})
184-
;=> #object[org.apache.mxnet.module.Module 0x5304d0f4 "org.apache.mxnet.module.Module@5304d0f4"]
185238

186-
;;To resume training from a saved checkpoint, instead of calling `set-params`, directly call `fit`, passing the loaded parameters, so that `fit` knows to start from those parameters instead of initializing randomly:
239+
;; To resume training from a saved checkpoint, pass the loaded
240+
;; parameters to the `fit` function. This will prevent `fit` from
241+
;; initializing randomly.
187242

188-
;; reset the training data before calling fit or you will get an error
243+
;; (First, reset the training data before calling `fit` or you will
244+
;; get an error)
189245
(mx-io/reset train-data)
190246
(mx-io/reset test-data)
191247

192-
(m/fit new-mod {:train-data train-data :eval-data test-data :num-epoch 2
193-
:fit-params (-> (m/fit-params {:begin-epoch 1}))})
194-
195-
;;Create fit-params, and then use it to set `begin-epoch` so that fit() knows to resume from a saved epoch.
196-
197-
248+
;; Create `fit-params` and then use it to set `begin-epoch` so that
249+
;; `fit` knows to resume from a saved epoch.
250+
(m/fit new-mod {:train-data train-data
251+
:eval-data test-data
252+
:num-epoch 2
253+
:fit-params (m/fit-params {:begin-epoch 1})})

0 commit comments

Comments
 (0)