A paper currently under review by academics at the International Conference on Learning Representations (ICLR) explores the possibility of building the fear factor into AI machines.
Combating Reinforcement Learning's Sisyphean Curse with Intrinsic Fear — Zachary C. Lipton https://t.co/3IzY36Fea9
— Deep RL (@deep_rl) November 7, 2016
The paper's abstract says: "We might hope for an agent that would never make catastrophic mistakes. At the very least, we could hope that an agent would eventually learn to avoid old mistakes. Unfortunately, even in simple environments, modern deep reinforcement learning techniques are doomed by a Sisyphean curse."
The abstract continues to suggest that AI 'agents' are just as likely to forget new experiences as they are to understand and remember a new one. "Consequently, for as long as they continue to train, state-aggregating agents may periodically relive catastrophic mistakes."
The scientists attempted to induce fear in agents to train them to avoid dangerous situations. Their paper argues that if AI machines can be rewarded for making good decisions — they can be punished for making wrong ones — and therefore fear the consequences of those actions or decisions.
Using Deep Reinforcement Learning (DRL) AI machines are trained to make good decisions by chasing rewards.
"If you stray too close to the edge of a roof your fear system kicks in and you step away to stop yourself from falling over. But if the fear you feel is so powerful and it stays with you, you might develop an irrational fear where you never step foot on a roof ever again," Zachary Lipton, co-author of the paper and researcher at UC San Diego told The Register.
The academics suggest that DRL can be reversed, so if machines can be rewarded for making good decisions, so they can be for making wrong ones, and learn to perhaps fear the consequences.
