Jérémy Scheurer

We introduce Imitation Learning from Language Feedback (ILF), demonstrate that large language models accurately incorporate natural language feedback and that finetuning with ILF scales well with the dataset size, even outperforming finetuning on human summaries.

We introduce Imitation Learning from Language Feedback (ILF) to improve code generation, demonstrating that a small amount of natural language feedback during training can lead to significant performance gains on program synthesis benchmarks.

We propose a three-step learning algorithm to learn from natural language feedback, which conveys more information per human evaluation than comparisons.

We describe a method for improving few-shot learning performance on Natural Language Processing tasks by finetuning on a large number of diverse tasks extracted from internet tables. We find that finetuning on narrow subsets of these tasks can lead to similar improvements, suggesting that the gains are not from domain adaptation but adapting to few-shot learning in general.