Evaluating Top-k RAG-based approach for Game Review Generation

Having access to public opinion for a particular product can be a cumbersome task. There are multiple reviews for the same product. Some may be good or bad depending on the bias of the reviewer. Using LLMs for the interpretation of this data would make it easier to understand the overall perception of a product. This is a study regarding how well RAG+LLMs can be used as Game Review Generators, built using state-of-the-art open source LLM LLaMA 2 13b and the Retrieval Augmented Generation framework llamaindex. The goal here is to generate and evaluate game reviews that take elements from a set of game reviews regarding a particular game without using any form of fine-tuning. This is achieved by using a rudimentary ‘query engine’ over a subset of publicly available game reviews. Game reviews are converted to vector stores which allow us to use top-k semantic retrieval for inference. This technique of providing data to an LLM from a document is called Retrieval Augmented Generation or RAG. Upon experimenting, game-specific reviews were generated (without using any form of fine-tuning) with the help of RAG combined with a top-k semantic retrieval ranking system. The application of this technique goes beyond simple game review generation, it can be used to generate and query context-specific information on any product given enough base information.