{"id":592844,"date":"2025-08-25T12:01:26","date_gmt":"2025-08-25T11:01:26","guid":{"rendered":"https:\/\/blog.jetbrains.com\/?post_type=pycharm&p=592844"},"modified":"2026-02-13T14:40:56","modified_gmt":"2026-02-13T13:40:56","slug":"fine-tuning-and-deploying-gpt-models-using-hugging-face-transformers","status":"publish","type":"pycharm","link":"https:\/\/blog.jetbrains.com\/pycharm\/2025\/08\/fine-tuning-and-deploying-gpt-models-using-hugging-face-transformers\/","title":{"rendered":"A Practical Guide to Fine-Tuning and Deploying GPT Models Using Hugging Face Transformers"},"content":{"rendered":"\n

Hugging Face is currently a household name for machine learning researchers and enthusiasts. One of their biggest successes is Transformers<\/a>, a model-definition framework for machine learning models in text, computer vision, audio, and video. Because of the vast repository of state-of-the-art machine learning models available on the Hugging Face Hub<\/a> and the compatibility of Transformers with the majority of training frameworks, it is widely used for inference and model training.<\/p>\n\n\n\n

Why fine-tuning GPT models matters<\/h2>\n\n\n\n

Fine-tuning AI models is crucial for tailoring their performance to specific tasks and datasets, enabling them to achieve higher accuracy and efficiency compared to using a general-purpose model. By adapting a pre-trained model, fine-tuning reduces the need for training from scratch, saving time and resources. It also allows for better handling of specific formats, nuances, and edge cases within a particular domain, leading to more reliable and tailored outputs.

In this blog post, we will fine-tune a GPT model using the Hugging Face Transformers library with mathematical reasoning so it better handles math questions.<\/p>\n\n\n\n

Using Hugging Face models in PyCharm<\/h2>\n\n\n\n

After downloading PyCharm<\/a>, we can easily browse and add any models from Hugging Face. In a new Python file, from the Code<\/em> menu at the top, select Insert HF Model<\/em>.<\/p>\n\n\n\n

\"Using<\/figure>\n\n\n\n

In the menu that opens, you can browse models by category or start typing in the search bar at the top. When you select a model, you can see its description on the right.<\/p>\n\n\n\n

\"Explore<\/figure>\n\n\n\n

When you click Use Model, <\/em>you will see a code snippet added to your file. And that’s it \u2013 You’re ready to start using your Hugging Face model.<\/p>\n\n\n\n

\"Use<\/figure>\n\n\n\n

Understanding GPT models<\/h2>\n\n\n\n

GPT models are very popular on the Hugging Face Hub<\/a>, but what are they? GPTs are trained models that understand natural language and generate high-quality text. They are mainly used in tasks related to textual entailment, question answering, semantic similarity, and document classification. The most famous example is ChatGPT, created by OpenAI<\/a>.<\/p>\n\n\n\n

A lot of OpenAI GPT models are available on the Hugging Face Hub<\/a>, and we will learn how to use these models with Transformers, <\/em>fine-tune them with our own data, and deploy them in an application.<\/em><\/p>\n\n\n\n

What are transformers and GPT models?<\/strong><\/h2>\n\n\n\n

Transformers are a class of deep learning models designed to understand and generate sequences of data, most commonly text. At the heart of a transformer is the attention mechanism, which allows the model to weigh the importance of different words in a sequence relative to each other.<\/p>\n\n\n\n

While decoder-only models such as GPTs process tokens autoregressively from left to right during both training and inference, they still use self-attention over all previously generated tokens to build context. This attention-based approach enables the model to consider relevant information from across the whole available input, making it far better at handling long-range dependencies and complex language patterns.<\/p>\n\n\n\n

GPT models are a specific type of transformer that use a decoder-only architecture. This means they are trained to predict the next token in a sequence based solely on what has come before. By stacking multiple attention layers and feed-forward networks, GPT models learn rich representations of language during pre-training on large text corpora.<\/p>\n\n\n\n

The core principle behind a GPT is to pre-train on vast amounts of general text, then fine-tune on more specific data to adapt it to particular tasks. This enables GPT models to perform text generation, summarization, and domain-specific reasoning after fine-tuning.<\/p>\n\n\n\n

Benefits of transformers for GPT fine-tuning<\/h2>\n\n\n\n

Transformers, together with other tools provided by Hugging Face, provides high-level tools for fine-tuning any sophisticated deep learning model. Instead of requiring you to fully understand a given model\u2019s architecture and tokenization method, these tools help make models \u201cplug and play\u201d with any compatible training data, while also providing a large amount of customization in tokenization and training.<\/p>\n\n\n\n

Why use Hugging Face transformers?<\/strong><\/h2>\n\n\n\n

Hugging Face transformers provide a unified ecosystem for working with modern machine learning models, removing much of the complexity involved in training and deploying large language models (LLMs). Instead of building training loops, evaluation logic, and model loading from scratch, developers can rely on high-level abstractions that still allow for deep customization when needed.<\/p>\n\n\n\n

One of the key components is the Trainer API<\/a>, which handles the full training lifecycle. Instead of writing custom training loops, you can define your TrainingArguments, pass your training and evaluation datasets into Trainer, and start training with just a few lines of code. This makes fine-tuning large models far more approachable, especially when experimenting with different hyperparameters or datasets. Hugging Face also offers a rich datasets library, which simplifies loading, preprocessing, and splitting large-scale datasets directly from the Hub.<\/p>\n\n\n\n

Tokenizers are tightly integrated with each model architecture, ensuring consistent preprocessing and efficient handling of text at scale. For performance and scalability, the Accelerate library<\/a> enables easy training across CPUs, GPUs, and multiple devices without significant code changes.<\/p>\n\n\n\n

Beyond the tooling, the Hugging Face community<\/a> is a major advantage. Thousands of open-source models, datasets, and examples are shared on the Hub, allowing teams to build on proven work, learn from others, and iterate faster.<\/p>\n\n\n\n

Running transformers with Hugging Face<\/h2>\n\n\n\n

To get a closer look at <\/em>Transformers in action, let\u2019s see how we can use it to interact with a GPT model.<\/p>\n\n\n\n

Inference using a GPT pretrained model with a pipeline<\/h3>\n\n\n\n

After selecting and adding the OpenAI GPT-2 model to the code, this is what we\u2019ve got:<\/p>\n\n\n\n

from transformers import pipeline\n\n\npipe = pipeline(\"text-generation\", model=\"openai-community\/gpt2\")<\/pre>\n\n\n\n
Before we can use it, we need to make a few preparations. First, we need to install a machine learning framework. In this example, we chose PyTorch<\/a>. You can install it easily via the Python Packages<\/em> window in PyCharm.<\/p>\n\n\n\n
\"Install<\/figure>\n\n\n\n
Then we need to install Transformers <\/em>using the `torch` option. You can do that by using the terminal \u2013 open it using the button on the left or use the \u2325 F12 <\/em>(macOS) or Alt + F12<\/em> (Windows) hotkey.<\/p>\n\n\n\n
\"Install<\/figure>\n\n\n\n
In the terminal, since we are using uv, we use the following commands to add it as a dependency and install it:<\/p>\n\n\n\n
uv add \u201ctransformers[torch]\u201d\nuv sync<\/pre>\n\n\n\n
If you are using pip:<\/p>\n\n\n\n
pip install \u201ctransformers[torch]\u201d<\/pre>\n\n\n\n
We will also install a couple more libraries that we will need later, including python-dotenv, datasets, <\/em>notebook, <\/em>and <\/em>ipywidgets. <\/em>You can use either of the methods above to install them.
After that, it may be best to add a GPU device to speed up the model. Depending on what you have on your machine, you can add it by setting the device parameter in pipeline. <\/em>Since I am using a Mac M2 machine, I can set device=\"mps\"<\/code> like this:<\/p>\n\n\n\n
pipe = pipeline(\"text-generation\", model=\"openai-community\/gpt2\", device=\"mps\")<\/pre>\n\n\n\n
If you have CUDA GPUs you can also set device=\"cuda\"<\/code>.<\/p>\n\n\n\n
Now that we\u2019ve set up our pipeline, let\u2019s try it out with a simple prompt:<\/p>\n\n\n\n
from transformers import pipeline\n\n\npipe = pipeline(\"text-generation\", model=\"openai-community\/gpt2\", device=\"mps\")\n\n\nprint(pipe(\"A rectangle has a perimeter of 20 cm. If the length is 6 cm, what is the width?\", max_new_tokens=200))<\/pre>\n\n\n\n
Run the script with the Run<\/em> button () at the top:<\/p>\n\n\n\n
\"Run<\/figure>\n\n\n\n
The result will look something like this:<\/p>\n\n\n\n
[{'generated_text': 'A rectangle has a perimeter of 20 cm. If the length is 6 cm, what is the width?nnA rectangle has a perimeter of 20 cm. If the length is 6 cm, what is the width? A rectangle has a perimeter of 20 cm. If the width is 6 cm, what is the width? A rectangle has a perimeter of 20 cm. If the width is 6 cm, what is the width? A rectangle has a perimeter of 20 cm. If the width is 6 cm, what is the width?nnA rectangle has a perimeter of 20 cm. If the width is 6 cm, what is the width? A rectangle has a perimeter of 20 cm. If the width is 6 cm, what is the width? A rectangle has a perimeter of 20 cm. If the width is 6 cm, what is the width? A rectangle has a perimeter of 20 cm. If the width is 6 cm, what is the width?nnA rectangle has a perimeter of 20 cm. If the width is 6 cm, what is the width? A rectangle has a perimeter'}]<\/pre>\n\n\n\n
There isn\u2019t much reasoning in this at all, only a bunch of nonsense. <\/p>\n\n\n\n
You may also see this warning:<\/p>\n\n\n\n
Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.<\/pre>\n\n\n\n
This is the default setting. You can also manually add it as below, so this warning disappears, but we don\u2019t have to worry about it too much at this stage.<\/p>\n\n\n\n
print(pipe(\"A rectangle has a perimeter of 20 cm. If the length is 6 cm, what is the width?\", max_new_tokens=200, pad_token_id=pipe.tokenizer.eos_token_id))<\/pre>\n\n\n\n
Now that we\u2019ve seen how GPT-2 behaves out of the box, let\u2019s see if we can make it better at math reasoning with some fine-tuning.<\/p>\n\n\n\n
Loading and preparing a dataset for GPT fine-tuning<\/h3>\n\n\n\n
Before we work on the GPT model, we first need training data. Let\u2019s see how to get a dataset from the Hugging Face Hub.<\/p>\n\n\n\n
If you haven’t already, sign up for a Hugging Face account and create an access token<\/a>. We only need a `read` token for now. Store your token in a `.env` file, like so:<\/p>\n\n\n\n
HF_TOKEN=your-hugging-face-access-token<\/pre>\n\n\n\n
We will use this Math Reasoning Dataset<\/a>, which has text describing some math reasoning. We will fine-tune our GPT model with this dataset so it can solve math problems more effectively.<\/p>\n\n\n\n
Let\u2019s create a new Jupyter notebook, which we\u2019ll use for fine-tuning because it lets us run different code snippets one by one and monitor the progress.<\/p>\n\n\n\n
In the first cell, we use this script to load the dataset from the Hugging Face Hub:<\/p>\n\n\n\n
from datasets import load_dataset\nfrom dotenv import load_dotenv\nimport os\n\n\nload_dotenv()\ndataset = load_dataset(\"Cheukting\/math-meta-reasoning-cleaned\", token=os.getenv(\"HF_TOKEN\"))\ndataset<\/pre>\n\n\n\n
Run this cell (it may take a while, depending on your internet speed), which will download the dataset. When it\u2019s done, we can have a look at the result:<\/p>\n\n\n\n
DatasetDict({\n    train: Dataset({\n        features: ['id', 'text', 'token_count'],\n        num_rows: 987485\n    })\n})\n<\/pre>\n\n\n\n
If you are curious and want to have a peek at the data, you can do so in PyCharm. Open the Jupyter Variables<\/em> window using the button on the right:<\/p>\n\n\n\n
\"Open<\/figure>\n\n\n\n
Expand dataset<\/em> and you will see the View as DataFrame<\/em> option next to dataset[\u2018train\u2019]<\/em>:<\/p>\n\n\n\n
\"Jupyter<\/figure>\n\n\n\n
Click on it to take a look at the data in the Data View<\/em> tool window:<\/p>\n\n\n\n
\"Data<\/figure>\n\n\n\n
Next, we will tokenize the text in the dataset:<\/p>\n\n\n\n
from transformers import GPT2Tokenizer\n\n\ntokenizer = GPT2Tokenizer.from_pretrained(\"openai-community\/gpt2\")\ntokenizer.pad_token = tokenizer.eos_token\n\n\ndef tokenize_function(examples):\n   return tokenizer(examples['text'], truncation=True, padding='max_length', max_length=512)\n\n\ntokenized_datasets = dataset.map(tokenize_function, batched=True)<\/pre>\n\n\n\n
Here we use the GPT-2 tokenizer and set the pad_token<\/code> to be the eos_token<\/code>, which is the token indicating the end of line. After that, we will tokenize the text with a function. It may take a while the first time you run it, but after that it will be cached and will be faster if you have to run the cell again.<\/p>\n\n\n\n
The dataset has almost 1 million rows for training. If you have enough computing power to process all of them, you can use them all. However, in this demonstration we\u2019re training locally on a laptop, so I’d better only use a small portion!<\/p>\n\n\n\n
tokenized_datasets_split = tokenized_datasets[\"train\"].shard(num_shards=100, index=0).train_test_split(test_size=0.2, shuffle=True)\ntokenized_datasets_split<\/pre>\n\n\n\n
Here I take only 1% of the data, and then perform train_test_split<\/code> <\/em>to split the dataset into two:<\/p>\n\n\n\n
DatasetDict({\n    train: Dataset({\n        features: ['id', 'text', 'token_count', 'input_ids', 'attention_mask'],\n        num_rows: 7900\n    })\n    test: Dataset({\n        features: ['id', 'text', 'token_count', 'input_ids', 'attention_mask'],\n        num_rows: 1975\n    })\n})\n<\/pre>\n\n\n\n
Now we are ready to fine-tune the GPT-2 model.<\/p>\n\n\n\n
Step-by-step: fine-tuning a GPT model<\/h3>\n\n\n\n
In the next empty cell, we will set our training arguments:<\/p>\n\n\n\n
from transformers import TrainingArguments\ntraining_args = TrainingArguments(\n   output_dir='.\/results',\n   num_train_epochs=5,\n   per_device_train_batch_size=8,\n   per_device_eval_batch_size=8,\n   warmup_steps=100,\n   weight_decay=0.01,\n   save_steps = 500,\n   logging_steps=100,\n   dataloader_pin_memory=False\n)<\/pre>\n\n\n\n
Most of them are pretty standard for fine-tuning a model. However, depending on your computer setup, you may want to tweak a few things:<\/p>\n\n\n\n