Table of contents
Fine-Tuning A LLM Small Practical Guide With Resources
Fine-Tuning A LLM Small Practical Guide With Resources
I tried to fine-tune a DeepSeek model but failed, then I tried to do it with Llama and I couldn’t make it work, finally I managed to do it with a Mistral model, specifically the 7B parameter model.
I share with you some resources that I found useful during the process and some of my notes related to the topic. Still, don’t get confused by this, I still believe that we are in an AI bubble .
Where to learn the basics of AI and Neural networks?
If you have no idea about artificial intelligence or what a neural network is, the most complete and didactic videos belong to 3blue1brown.
However, it is important to clarify that these videos do not start from scratch, it is necessary that you have a background in linear algebra, linear regression and differential and integral calculus. I know it can be a little discouraging for those who come from non-technical backgrounds but it is not as difficult as it seems.
If you only speak english you probably don’t know DotCSV, but what you may not know is that DotCSV has an introduction to neural networks and their videos are excellent, the bad part of course is that his videos are in Spanish. Go look for your duolinguo App.
But if you already know about neural networks, then, let’s start with the fine-tuning concept.
What is fine-tuning?
Fine-tuning is the procedure of selecting an already trained model and continuing its training on a particular data set from the field. Like for example a LLM that has been trained specifically in scientific articles about diabetes.
Because, why use prompt engineering when you can fine-tune a model and get the same results?
Most current LLM models exhibit quite positive overall performance, but fail to solve particular problems focused on specific tasks. The fine-tuning procedure provides significant benefits, such as decreased computational costs and the opportunity to use advanced models without the need to build one from scratch. It may not replace programmers but fine-tuning can turn a model in the ultimate prediction tool in an area of knowledge.
Steps needed to fine-tune an LLM
The steps may vary according to the model but generally the process looks something like this:
- Load a dataset: Gather your dataset, either existing or original.
- Data preprocessing: Tokenize your dataset using model tokenizer and split it into training, validation and test sets.
- Model selection: Choose a pre-trained LLM.
- Parameter configuration: Configure the hyperparameters.
- Training or Fine-Tuning: Train the model on the customized dataset.
- Evaluation: Test it with the test set.
- Inference: Deploy the fine-tuned model for inference on new data, making sure it generalizes well to unseen inputs.
Here are some resources about fine-tuning:
I loved this resource for understanding the general steps to fine-tuning an LLM .
The notebook that I used to fine-tune the model is here and you can check it.
Where to get datasets for fine-tuning an LLM?
In the same way you need data to train an AI model, fine-tuning requires that you provide it with the best data can you find, the best source is your own data.
I used datasets from HuggingFace , however it is important to consider that pre-trained models that are freely available were probably trained using the same data, so if you decide to use them, don’t expect to have a substantial improvement over normal model performance, it would be best to use your own data or original content.
Make sure your dataset is in a proper format. Eg, I had to convert one using Python to JSONL, because each line was a separate file, which made the import incredibly slow.
Which models are best for fine-tuning an LLM?
To do fine-tuning, a good starting point is using a pre-trained LLM, the most parameters the better results but its performance will be worse.
As a rule of thumb, it’s a good idea to browse HuggingFace, and look for large models, among the most popular ones I could find are Llama, DeepSeek, Mistral, Falcon, among others.
I tested the following models because they were very light and did not require as much computing power, but you can use their larger versions with more parameters:
I got better results with the Mistral LLMs, probably because the DeepSeek ones were trained using Chinese sources.
I was actually able to train and deploy it but the model returned Chinese characters, just like Searle’s room , from time to time as part of the response, although it was incredibly fast.
Where to fine-tune an LLM for free?
It would be best to have your own GPUs, but in case you can’t, Google, through
Google Colab
has GPUs available for free for a limited amount of hours per day,
Kaggle
also offers you 30 hours of GPU usage per week to run your experiments. And create your furry images using the forbidden models.
In my fine-tuning adventure I also found a company called salad , which serves as an intermediary between users and gamers who want to rent their GPUs, the prices are incredibly competitive. VastAI is also another option. Digital Ocean also has very competitive prices and provices a wide variety of GPUs.
LLM Fine-tuning Tutorials
For Fine-tuning I found these videos quite enjoyable, and straight to the point, however I could not produce a working model, it is possible that the notebooks were outdated and probably my inexperience made it impossible to get it.
This is a good introduction but does not share the notebook, although I was able to find it on Google Colab, I put the link in the next section.
He is a bit more specific and also gets right to the point.
Notebooks for LLM fine-tuning.
This was the notebook that allowed me to fine-tuning the Mistral model , with a couple of modifications, of course.
Most of the notebooks just need you to change the model and adapt the dataset to the model input, so you can use them as a starting point for your particular case.
Others that I found particularly useful but did not work for me or produced unwanted results.
Where to deploy an LLM model?
After creating your model you may want to make it available to others.
Generally there are tools like Gradio that abstract the process of generating a chat server like OpenAI, you can set them up on your own or use the ones available from HuggingFace, although obviously for a monthly cost, the free plans only have CPU and RAM, so it is usually not enough to run your LLM, especially the big ones.
Since my project was for educational purposes and I don’t want to pay for a premium plan, I decided to put the code necessary to run my model in a GoogleColab notebook with the necessary code to run it.
