Open AI research shows that human feedback can speed up machine learning tasks

Research by Open AI has demonstrated the potential of human feedback to accelerate machine learning. The researchers have developed an algorithm that can take a guess at what humans want, based on human feedback on which of two behaviours is better. The researchers used the new algorithm to develop a backflip, by letting humans select the better option between two choices. This approach is better than specifying goals or writing a reward function can lead to undesirable behavior from the AI.The algorithm was used for a series of Atari games as well. The machines learnt to anticipate human preferences based on the feedback, instead of the goals in the game. In Seaquest, the agent learned to value oxygen, and worked out how to recover from crashes in the racing title Enduro. The performance of the agents matched the humans, and at times the agents even developed superhuman capabilities. The humans could provide the agents with feedback not aligned with the goal of the environment, so the agents could match the score of another car in Enduro instead of beating the score.The aim of the research is to improve the safety of AI solutions. This approach of human feedback can complement existing AI development approaches such as imitation learning and reinforcement learning. Using human feedback to train the machines allows for advancing the capabilities of the agent much faster than manually hand crafting the objectives.