QLearningRule#
- class QLearningRule(alpha=0.5, gamma=0.1, values=None, reward=None, maximum=None, *args, **kwargs)[source]#
Bases:
objectQ-learning rule (Watkins, 1989) for a one-dimensional array of Q-values.
- Parameters:
- alpha
float The learning rate. Default is 0.5.
- gamma
float The discount factor. Default is 0.1.
- values
ndarray The values matrix. It is a 1D array of Q-values active for the current state, where each element corresponds to an action.
- reward
float The reward received on the current state.
- maximum
float The maximum estimated reward for the next state.
- alpha
Notes
The Q-learning rule is a model-free reinforcement learning algorithm that is used to learn the value of an action in a given state. It is defined as
\[\Delta \mathcal{Q}(s, a) = \alpha \cdot (r + \gamma \cdot \max_{a'} \mathcal{Q}(s', a') - \mathcal{Q}(s, a))\]where \(\Delta \mathcal{Q}(s, a)\) is the change in value of action \(a\) in state \(s\), \(r\) is the reward received on the current state, \(\gamma\) is the discount factor, and \(\max_{a'} \mathcal{Q}(s', a')\) is the maximum estimated reward for the next state.
References
Watkins, C. J. C. H. (1989). Learning from delayed rewards.
Watkins, C. J., & Dayan, P. (1992). Q-learning. Machine learning, 8, 279-292.
Examples
>>> import numpy as np >>> from cpm.models.learning import QLearningRule >>> values = np.array([1, 0.5, 0.99]) >>> component = QLearningRule(alpha=0.1, gamma=0.8, values=values, reward=1, maximum=10) >>> component.compute() array([1.8 , 1.35 , 1.791])