2024-09-01

Lossless Compression using Large Language Models and Arithmetic Coding

Whimsy is a text compression tool that leverages LLMs’ probabilistic predictions to assign shorter codes to more likely tokens. By harnessing GPT’s ability to understand language context, it generates more efficient encodings that reduce file sizes.