Skip to content

Commit eba49a9

Browse files
update documentation of softmax function
1 parent 69ee353 commit eba49a9

File tree

1 file changed

+9
-7
lines changed

1 file changed

+9
-7
lines changed

psyneulink/core/components/functions/nonstateful/transferfunctions.py

+9-7
Original file line numberDiff line numberDiff line change
@@ -2828,9 +2828,10 @@ class SoftMax(TransferFunction):
28282828
<SoftMax.gain>` parametrically based on the `variable <SoftMax.variable>`:
28292829
28302830
- *mask_threshold* -- setting the **mask_threshold** argument to a scalar value causes the `variable
2831-
<SoftMax.variable>` to be thresholded by that value before applying the SoftMax function; any elements of
2832-
`variable <SoftMax.variable>` with an absolute value below the threshold are set to 0; all others are scaled
2833-
by the specified `gain <SoftMax.gain>` and then passed through the SoftMax function. This only applies if the
2831+
<SoftMax.variable>` to be thresholded by that value before applying the SoftMax function; Each element in
2832+
variable <SoftMax.variable> is first scaled by gain <SoftMax.gain>. Then, any elements with an absolute
2833+
value below *mask_threshold* are set to negative infinity (``-inf``), effectively masking them since
2834+
``exp(-inf) = 0``. The remaining values are then passed through the SoftMax function. This only applies if the
28342835
**gain** argument is specified as a scalar; if it is specified as *ADAPTIVE*, then the **mask_threshold**
28352836
argument is ignored.
28362837
@@ -2920,10 +2921,11 @@ class SoftMax(TransferFunction):
29202921
29212922
mask_threshold : scalar or None
29222923
determines whether the `variable <SoftMax.variable>` is thresholded before applying the SoftMax function;
2923-
if it is a scalar, only elements of `variable <SoftMax.variable>` with an absolute value greater than that
2924-
value are considered when applying the SoftMax function (which are then scaled by the `gain <SoftMax.gain>`
2925-
parameter; all other elements are assigned 0. This only applies if `gain <SoftMax.gain>` is specified as a
2926-
scalar; otherwise it is ignored (see `Thresholding and Adaptive Gain <SoftMax_AdaptGain>` for details).
2924+
if it is a scalar, each elements of `variable <SoftMax.variable>` is first scaled by `<SoftMax.gain>`. Then,
2925+
only elements with an absolute value greater than *mask_threshold* are considered when applying the SoftMax
2926+
function, while all other elements are set to ``-inf`` effectively masking them since ``exp(-inf) = 0``.
2927+
This only applies if `gain <SoftMax.gain>` is specified as a scalar; otherwise it is ignored
2928+
(see `Thresholding and Adaptive Gain <SoftMax_AdaptGain>` for details).
29272929
29282930
adapt_scale : scalar
29292931
determines the *scale* parameter using by the `adapt_gain <SoftMax.adapt_gain>` method (see method for details).

0 commit comments

Comments
 (0)