This chapter covers
- Why you might want to build your own large language model
- Selecting an LLM model to serve as your base for a custom configuration
- How (in very general terms) model fine-tuning works
Build (or modify) your own LLM? But didn’t OpenAI (and its investors) spend billions of dollars optimizing and training their GPT? Is it possible to generate even remotely competitive results through a do-it-yourself project using local hardware?
Incredibly, at this point in the whirlwind evolution of LLM technologies, the answer to that question is yes. Due to the existence of Meta’s open source LLaMA model, an unauthorized leak of the model’s weights (which I’ll explain in just a moment), and a lot of remarkable public contributions, there are now hundreds of high-powered but resource-friendly LLMs available for anyone to download, optionally modify, and run. Having said that, if operating at this depth of technology tinkering isn’t your thing—and especially if you don’t have access to the right kind of hardware—feel free to skip to the next chapter.
Some background to building your own model
Before I explain how all that works, we should address the bigger question: Why would anyone want to build their own LLM? Here are some things worth considering: