@@ -2828,9 +2828,10 @@ class SoftMax(TransferFunction):
2828
2828
<SoftMax.gain>` parametrically based on the `variable <SoftMax.variable>`:
2829
2829
2830
2830
- *mask_threshold* -- setting the **mask_threshold** argument to a scalar value causes the `variable
2831
- <SoftMax.variable>` to be thresholded by that value before applying the SoftMax function; any elements of
2832
- `variable <SoftMax.variable>` with an absolute value below the threshold are set to 0; all others are scaled
2833
- by the specified `gain <SoftMax.gain>` and then passed through the SoftMax function. This only applies if the
2831
+ <SoftMax.variable>` to be thresholded by that value before applying the SoftMax function; Each element in
2832
+ variable <SoftMax.variable> is first scaled by gain <SoftMax.gain>. Then, any elements with an absolute
2833
+ value below *mask_threshold* are set to negative infinity (``-inf``), effectively masking them since
2834
+ ``exp(-inf) = 0``. The remaining values are then passed through the SoftMax function. This only applies if the
2834
2835
**gain** argument is specified as a scalar; if it is specified as *ADAPTIVE*, then the **mask_threshold**
2835
2836
argument is ignored.
2836
2837
@@ -2920,10 +2921,11 @@ class SoftMax(TransferFunction):
2920
2921
2921
2922
mask_threshold : scalar or None
2922
2923
determines whether the `variable <SoftMax.variable>` is thresholded before applying the SoftMax function;
2923
- if it is a scalar, only elements of `variable <SoftMax.variable>` with an absolute value greater than that
2924
- value are considered when applying the SoftMax function (which are then scaled by the `gain <SoftMax.gain>`
2925
- parameter; all other elements are assigned 0. This only applies if `gain <SoftMax.gain>` is specified as a
2926
- scalar; otherwise it is ignored (see `Thresholding and Adaptive Gain <SoftMax_AdaptGain>` for details).
2924
+ if it is a scalar, each elements of `variable <SoftMax.variable>` is first scaled by `<SoftMax.gain>`. Then,
2925
+ only elements with an absolute value greater than *mask_threshold* are considered when applying the SoftMax
2926
+ function, while all other elements are set to ``-inf`` effectively masking them since ``exp(-inf) = 0``.
2927
+ This only applies if `gain <SoftMax.gain>` is specified as a scalar; otherwise it is ignored
2928
+ (see `Thresholding and Adaptive Gain <SoftMax_AdaptGain>` for details).
2927
2929
2928
2930
adapt_scale : scalar
2929
2931
determines the *scale* parameter using by the `adapt_gain <SoftMax.adapt_gain>` method (see method for details).
0 commit comments