Emotional Reinforcement Learning: ACM's Novel Approach

Modern reinforcement learning (RL) algorithms typically optimize for accumulating rewards based on task performance alone. However, the ACM (Artificial Consciousness Module) introduces additional emotional signals, with the idea to simulate how living organisms learn through internal states.

Emotional State Dimensions

Our approach uses three core dimensions adapted from psychology’s Circumplex Model of Affect:

• Valence: Represents how positive or negative the state is.
• Arousal: Indicates emotional intensity, ranging from calm to excited.
• Dominance: Assesses perceived control, from submissive to empowered.

Integrating these dimensions helps guide agent behavior beyond external reward signals, reflecting an adaptive process more akin to real world learning.

Emotional Reward Shaping

By combining environment rewards with an emotional component, the RL algorithm internalizes experiences with a more “lifelike” feedback loop. For instance, an agent in a pleasant (positive valence) state might gain an incremental bonus, while high arousal without sufficient dominance might reduce net reward. This mechanism encourages balanced, context-aware decision-making.

Progressive Complexity

We begin with simple simulations focusing on valence and arousal before introducing dominance and social factors later. This staged approach prevents the agent from becoming overwhelmed by complex emotional interactions early on. The idea is that emotional bonds get deep into the AI agent with the ACM through the stratification of all the previous experiences through the simulations. This is based mimicking the premise that consciousness is developed through the stratification of emotional meta-memories that gets inherit from ancestors.

Example Pseudocode

def emotional_reward_shaper(base_reward, valence, arousal, dominance):
    # Higher valence or dominance yields bonus
    # Excessive arousal may penalize reward
    return base_reward + 0.1 * valence + 0.1 * dominance - 0.05 * arousal

Future Layers: Social & Empathy

Over time, wthe plan is to incorporate empathy-based signals where emotional states evolve from interactions with other agents. Such social emotional complexity will enable deeper explorations of group dynamics, collaboration, and conflict resolution within the ACM.

Nevertheless, this is a work in progress and things will continuously being readapted as research develops.