GreedyRule

class GreedyRule(activations=None, epsilon=0, **kwargs)

Bases: object

A class representing an ε-greedy rule based on Daw et al. (2006).

Parameters:

activationsndarray

An array of activations for the greedy rule.

epsilonfloat

Exploration parameter. The probability of selecting a random action.

Attributes:

activationsndarray

An array of activations for the greedy rule.

epsilonfloat

Exploration parameter. The probability of selecting a random action.

policiesndarray

An array of outputs computed using the greedy rule.

shapetuple

The shape of the activations array.

Notes

The greedy rule is defined as follows:

\[\begin{split}P(x) = \begin{cases} 1 - (n - 1)\epsilon & \text{if } x = \text{argmax}(a) \\ \frac{\epsilon}{n} & \text{otherwise} \end{cases}\end{split}\]

where \(x\) is the action, \(a\) is the array of activations or the expected reward for each action, \(n\) is the number of actions, and \(\epsilon\) is the exploration parameter. Values are normalised to keep the probabilities valid. The greedy rule selects the action with the highest activation with probability \(1 - (n - 1)\epsilon\), and selects a random action with probability \(\frac{\epsilon}{n}\).

References

Daw, N. D., O’Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006). Cortical substrates for exploratory decisions in humans. Nature, 441(7095), Article 7095. https://doi.org/10.1038/nature04766

compute()

Computes the greedy rule.

Returns:

output: ndarray

A 2D array of outputs computed using the greedy rule.

choice()

Chooses the action based on the greedy rule.

Returns:

action: int

The chosen action based on the greedy rule.

config()

Returns the configuration of the greedy rule.

Returns:

config: dict

A dictionary containing the configuration of the greedy rule.

• activations (ndarray): An array of activations for the greedy rule.

• name (str): The name of the greedy rule.

• type (str): The class of function it belongs.