Last week, I had the privilege of attending the 2014 AAAI Fall Symposium on Knowledge, Skill, and Behavior Transfer in Autonomous Robots. My own interest in this topic comes from my goal of being able to program a mobile robot by driving it around. I presented a paper describing an approach for doing this. It improves upon my previous work by automatically associating actions with the Growing Neural Gas nodes, rather than relying on human input for specifying an action for every node.
To demonstrate the versatility of what I had implemented, on the morning of the talk I drove my Lego Mindstorms EV3 robot (running leJOS) around part of my hotel room for a couple of minutes, teaching it to avoid obstacles. I then included in my presentation a nice video of the purely visual obstacle avoidance it had learned in this short time span.
Even more fun was the poster session. I had promised to bring the robot with me to the poster session at the end of my oral presentation. At the start of the poster session, I drove the robot around the poster area for about two or three minutes. I was careful to make sure I introduced it to numerous human legs, so that it would learn to avoid them. I then set it loose, and it demonstrated very nice visual obstacle avoidance for about the next hour or so. When learning, it hadn't seen any white sneakers, so it did run into a couple of people, but for the most part it did great!
I'm hoping to release the source code for my implementation once the semester ends and I have a chance to clean things up a bit.
There were some interesting trends in the presentations I saw. Several people, including Peter Stone and Stéphane Doncieux, presented work in which learning happened in simulation. Stone's work "closed the loop" by transferring the learned skills to a physical robot, and even learned from the physical robot to retrain the simulation. This was an aspect of transfer I hadn't really thought about very much.
Several other presenters, including Gabriel Barth-Maron, David Abel, Benjamin Rosman, and Manuela Veloso, were focused on Markov Decision Processes and reinforcement learning. That isn't really the focus of my current work, although I've explored it in the past and I may do so again in the future. Much of the presented work involved ways of extracting transferrable domain knowledge from an MDP policy that had been learned with a particular reward function. By transferring that knowledge to learning a policy for a new reward function but in the same domain, learning can be accelerated.
Finally, there were several presenters, including Tesca Fitzgerald, Andrea Thomaz, and Yiannis Demiris, who showcased work in demonstration learning. The first two are interested in humans demonstrating tasks for humanoid robots, by either showing start/stop states for arrangements of objects, or even by directly manipulating their arms. Of particular interest was Tesca Fitzgerald's efforts towards developing a spectrum of related tasks for which knowledge transfer is possible to varying degrees. Task knowledge that is not transferred is handled with a planner; she had a great video of a robot hitting a ping-pong ball to illustrate what she had in mind. Yiannis Demeris does a lot of work with assistive robotics for the disabled. He showed some fascinating work in which the robots learned to help their clients, but only to the degree that the clients wanted the help. He's also done some work with robots teaching humans various tasks. Particularly amusing were the humanoid robot dance instructors!
I received some great feedback on my research from Yiannis Demiris, Nathan Ratliff, Matteo Leonetti, Eric Eaton, Pooyan Fazli, Matthew Taylor, Gabriel Barth-Maron, David Abel, Benjamin Rosman, Tesca Fitzgerald, Bruce Johnson, Cynthia Matuszek, and Laurel Riek. I returned home with enough new ideas to keep me very busy for the next couple of years. Thanks again to everyone who helped make this a great symposium!
No comments:
Post a Comment