Sunday, December 8, 2013

Explorative learning of inverse models: a theoretical perspective

Optimization and exploratory learning
have a non-negative relation
Learning inverse models is an old story in motor learning literature. It is well known to work based on optimization approaches like "feedback-error" learning or learning with "distal-teacher". But these approaches require prior knowledge. The alternative is trying to learn from self-explored data and try to fit an inverse model to it. In some cases this works, it other cases it is known to fail, and some cases have just become tractable with goal babbling. Despite the very wide applicability (and actual application!) of inverse models, these aspects are heavily undertheorized. When does it work, and why? What solutions are selected? What is the relation between exploratory learning and optimization? And how does all of that work when not using a fixed data set, but dynamic exploration like goal babbling?
Convergence of learning towards the
Moore-Penrose pseudo-inverse
Our recent paper (see below) gives some answers to these questions. It only considers linear domains, but including arbitrarily high dimensions and redundancy. This obviously leaves questions for the non-linear case, but already gives some very non-trivial findings. Some highlights:
  • We prove that the gradients of optimization and exploratory learning satisfy a non-negative relation.
  • We prove that any fixpoint of exploratory learning for any data must be an actual inverse model.
  • We prove that exploratory learning with goal babbling not only works, but converges to the optimal least-squares solution.
  • We show that the basic learning dynamics of goal babbling resemble those of explosive combustion processes. This gives a neat view on the previous finding that goal babbling constitutes a positive feedback loop – and explains the S-shaped learning curves also observed in human learning.
Rolf, M., and J.J. Steil, "Explorative Learning of Inverse Models: a Theoretical Perspective", Neurocomputing, in press (available online).
Abstract — We investigate the role of redundancy for exploratory learning of inverse functions, where an agent learns to achieve goals by performing actions and observing outcomes. We present an analysis of linear redundancy and investigate goal-directed exploration approaches, which are empirically successful, but hardly theorized except negative results for special cases, and prove convergence to the optimal solution. We show that the learning curves of such processes are intrinsically low-dimensional and S-shaped, which explains previous empirical findings, and finally compare our results to non-linear domains.

Tuesday, November 26, 2013

Upcoming event: Advances in Machine Learning for Sensorimotor Control @ NIPS 2013

In December I will attend the NIPS 2013 conference held in Lake Tahoe, USA. On Dec. 9th I will give talk providing a review of my recent work on Goal Babbling in the workshop "Advances in Machine Learning for Sensorimotor Control". The workshop brings together a very illustrative set of motor-learning researchers, and the program looks very promising!

Tuesday, November 5, 2013

Goal Babbling Matlab code released

Finally, we managed (big "thank you" to Andre, Klaus and Felix!) to have some open source code for goal babbling. We initially prepared it for the CITEC summer school, and now released it here. The code contains a complete implementation of my ICDL 2011 paper and is written in Matlab, coming right away with some nice examples. Also, it contains an implementation of the BHA simulation based on my IROS 2012 paper, so you can even let the trunk robot babble a little bit. Check it out!

Monday, October 7, 2013

Observable Motor Synergies during Goal Babbling

At the IROS conference in Tokyo this November I will present a contribution in the workshop "Cognitive Neuroscience Robotics". It issues phenomena called "motor synergies", fixed low-dimensional patterns of motor activation that are often considered to facilitate motor learning by reducing the effective dimension of a coordination problem. I discuss the need, or not-need, for such mechanisms and show that, on a behavioural level, a system that actually does goal babbling might look like one that uses motor synergies. A quick preview...

Wednesday, September 18, 2013

Goal Babbling @ CITEC Summer School

Last week the CITEC Summer School 2013 took place at Bielefeld University. This year's topic was "Continuous learning in living and artificial systems". On Thursday, Jochen and I gave a joint keynote on goal babbling as applied on the Bionic Handling Assistant. A subsequent afternoon workshop allowed participants to experiment with the goal babbling algorithm of my ICDL 2011 paper, and to finally apply it on the Bionic Handling Assistant robot. We had a very adept audience, since several participants already had experience or exposure to goal babbling before the school. So, discussions were very interesting and quite deep. For all participants, and all other interested people, we will soon open the source code to the public, so stay tuned!

Friday, August 30, 2013

ICDL 2013 Conference Digest

As I predicted, this year's ICDL-EpiRob conference was really an interesting event. Although the number of submissions (77 submitted, 54 accepted) was not as high as usual - maybe because coming to faraway Japan can still be difficult (e.g. to fund) for many researchers in U.S. and Europe.
The conference this time nevertheless attracted a very international set of authors: there has been a total number of 124 attendees, 51 from Asia, 60 from Europe, and 13 from the U.S., who gave some really nice talks, keynotes, and came up with some quite interesting discussions. Let me give a (very subjective) selection of highlights...

Tuesday, August 13, 2013

See you at ICDL in Osaka!

It has been lots of work this year as author, reviewer and publication co-chair, so it's time for the conference to finally begin. The IEEE ICDL-EpiRob conference will take place here in Osaka, starting on Sunday, Aug. 18, and ending on next Thursday, Aug. 22.
The conference will be held in Osaka's beautiful Central Public Hall:
Besides me giving my oral presentation on "Goal Babbling with Unknown Ranges: A Direction Sampling Approach", Felix will present our work on the integration of goal babbling and associative neural memories. I will also give a short talk at the co-located event here.
In general, the progam looks promising so I think this will be an interesting scientific event. Plus, it will certainly be a nice social event. Nice to see again so many familiar faces in my new home town.
Finally, I want to share a tip that Japanese people keep giving me to those who come to ICDL: Try not to melt! (see the latest weather here)

Tuesday, July 30, 2013

Feature in BI.research

The newest issue of Bielefeld University's research magazine BI.research features researchers from Bielefeld going abroad. One article describes Lars Schillingmann's and my change to Osaka University, our research motivation, and some aspects of living in Japan.
See the article  "Teaching robots how to learn"/"Robotern das Lernen beibringen" available in English and German in the latest issue of BI.research.

Monday, July 1, 2013

Preview: Papers accepted at ICDL 2013

I will have two papers at this year's ICDL-EpiRob conference:
  • Rolf, M., "Goal Babbling with Unknown Ranges: A Direction-Sampling Approach", and
  • Reinhart, R.F., and M. Rolf, "Learning Versatile Sensorimotor Coordination with Goal Babbling and Neural Associative Dynamics"
A quick preview...

Monday, May 27, 2013

Blog articles about the Bionic Handling Assistant

In recent months I wrote a couple of blog articles together with Arne Nordmann about our work with the Bionic Handling Assistant (BHA). The articles do not only describe the technical results, but also provide a lot of "behind the scenes" information:
I hope you enjoy reading!

Monday, May 20, 2013

Upcoming Event: Japanese Baby Science Meeting

Fukuoka, JapanOn this weekend (May 25/26) the 13th meeting of the Japanese Baby-Science Society (JSBS 2013) will take place in Fukuoka, Japan's 8th biggest city. I will present a poster with some ideas on how goal babbling can account for several observations in humans' and particularly early infants' motor learning. Furthermore, the Asada-Lab contributes four other posters and a special session.

Thursday, April 11, 2013

I moved to Osaka

After finishing my Ph.D. at the CoR-Lab, Bielefeld University (Germany), I now moved to the Emergent Robotics Lab (a.k.a. Asada-Lab) at the Osaka University (Japan). The Osaka-region with its 16.8 million inhabitants is among the 20 biggest urban agglomerations worldwide. Besides that, of course, Osaka is known for its university, and the robots developed therein (like this, this, this and this one).
What is most important for me: Osaka University is clearly one of the best places for research on Developmental Robotics. I think these are going to be exciting times...