Our goal is to enable robots to understand and execute natural language commands from human agents. We develop algorithms that allow a robot to interpret, learn and reason about semantic concepts embedded in language in the context of low-level metric representations perceived from sensors.