Large Language Models: Principles and Practice

The last few years have been marked by several breakthroughs in the domain of generative AI. Large language models such as GPT-4 are able to solve a plethora of tasks, ranging from text and code generation to multimodal data analysis, without task-specific training data. This tutorial, targeted at database researchers without prior background in language models, introduces language models as well as relevant use cases in the context of data management. The tutorial covers the fundamental principles enabling language models, including the Transformer architecture, pre-training, and alignment. Furthermore, the tutorial will show how to use language models in practice, leveraging OpenAI's GPT model to build a natural language query interface as a demonstration. Finally, the tutorial will discuss recent research exploiting language models in the context of data management.