SeparableRule#

class SeparableRule(alpha=None, zeta=None, weights=None, feedback=None, input=None, **kwargs)[source]#

Bases: object

A class representing a learning rule based on the separable error-term of Bush and Mosteller (1951).

Parameters:

alphafloat: The learning rate.
zetafloat, optional: The constant fraction of the magnitude of the prediction error, also called Weber’s scaling.
weightsarray_like: The value matrix, where rows are outcomes and columns are stimuli or features. The values can be anything; for example belief values, association weights, connection weights, Q-values.
feedbackarray_like, optional: The target values or feedback, sometimes referred to as teaching signals. These are the values that the algorithm should learn to predict.
inputarray_like, optional: The input value. The stimulus representation in the form of a 1D array, where each element can take a value of 0 and 1.
**kwargsdict, optional: Additional keyword arguments.

See also

cpm.models.learning.DeltaRule: An extension of the Rescorla and Wagner (1972) learning rule by Gluck and Bower (1988) to allow multi-outcome learning.

Notes

This type of learning rule was among the earliest formal models of associative learning (Le Pelley, 2004), which were based on standard linear operators (Bush & Mosteller, 1951; Estes, 1950; Kendler, 1971). It is used in a variety of reinforcement learning models. This learning rule is defined in cpm as

\[\Delta w_{ij} = \alpha \cdot (\lambda_i - w_{ij}) \cdot x_j\]

which is consistent with the modification of the Rescorla and Wagner (1972) learning rule by Sutton and Barto (2018). The current implementation generalises to any number of outcomes and stimuli, which means that it can be applied to both single- and multi-outcome learning paradigms.

References

Bush, R. R., & Mosteller, F. (1951). A mathematical model for simple learning. Psychological Review, 58, 313–323

Estes, W. K. (1950). Toward a statistical theory of learning. Psychological Review, 57, 94–107

Kendler, T. S. (1971). Continuity theory and cue dominance. In J. T. Spence (Ed.), Essays in neobehaviorism: A memorial volume to Kenneth W. Spence. New York: Appleton-Century-Crofts.

Le Pelley, M. E. (2004). The role of associative history in models of associative learning: A selective review and a hybrid model. Quarterly Journal of Experimental Psychology Section B, 57(3), 193-243.

compute()[source]#

Computes the prediction error using the learning rule.

Returns:

ndarray: The prediction error for each stimuli-outcome mapping. It has the same shape as the weights input argument.

Notes

The prediction error for each stimuli-outcome mapping before the update is stored in error, which has the same shape as the weights.

noisy_learning_rule()[source]#

Add random noise to the prediction error computed from the delta learning rule as specified Findling et al. (2019). It is inspired by Weber’s law of intensity sensation.

Returns:

ndarray: The prediction error for each stimuli-outcome mapping with learning noise. It has the same shape as the weights input argument.

References

Findling, C., Skvortsova, V., Dromnelle, R., Palminteri, S., and Wyart, V. (2019). Computational noise in reward-guided learning drives behavioral variability in volatile environments. Nature Neuroscience 22, 2066–2077

reset()[source]#: Resets the weights to zero.