ABSTRACT:

Language models work well, but they are far from trustworthy. Major open questions remain on high-stakes issues such as detecting benchmark contamination, identifying LM-generated text, and reliably generating factually correct outputs. Addressing these challenges will require us to build more precise, reliable algorithms and evaluations that provide guarantees that we can trust.

Despite the complexity of these problems and the black-box nature of modern LLMs, I will discuss how in all three problems — benchmark contamination, watermarking, and factual correctness — there are surprising connections between classic statistical techniques and language modeling problems that lead to precise guarantees for identifying contamination, watermarking LM-generated text, and ensuring the correctness of LM outputs.

ZOOM LINK (if unable to attend in-person): https://upenn.zoom.us/j/94597712175