Abstract

Large language models (LLMs) trained on code-completion have been shown to be capable of synthesizing simple Python programs from docstrings [1].
We find that these codewriting LLMs can be re-purposed to write robot policy code, given natural language commands.
Specifically, policy code can express functions or feedback loops that process perception outputs (e.g.,from object detectors [2], [3]) and parameterize control primitive APIs.
When provided as input several example language commands (formatted as comments) followed