agents-course/notebooks/bonus-unit1/gemma-SFT-thinking-function_call.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "43b502c1-9548-4580-84ad-1cbac158edb8",
   "metadata": {
    "id": "43b502c1-9548-4580-84ad-1cbac158edb8"
   },
   "source": [
    "# Bonus Unit 1: Fine-Tuning a model for Function-Calling\n",
    "\n",
    "In this tutorial, **we're going to Fine-Tune an LLM for Function Calling.**\n",
    "\n",
    "This notebook is part of the <a href=\"https://www.hf.co/learn/agents-course/unit1/introduction\">Hugging Face Agents Course</a>, a free Course from beginner to expert, where you learn to build Agents.\n",
    "\n",
    "<img src=\"https://huggingface.co/datasets/agents-course/course-images/resolve/main/en/communication/share.png\" alt=\"Agent Course\"/>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "gWR4Rvpmjq5T",
   "metadata": {
    "id": "gWR4Rvpmjq5T"
   },
   "source": [
    "## Exercise Prerequisites 🏗️\n",
    "\n",
    "Before diving into the notebook, you need to:\n",
    "\n",
    "🔲 📚 **Study [What is Function-Calling](https://www.hf.co/learn/agents-course/bonus-unit1/what-is-function-calling) Section**\n",
    "\n",
    "🔲 📚 **Study [Fine-Tune your Model and what are LoRAs](https://www.hf.co/learn/agents-course/bonus-unit1/fine-tuning) Section**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1rZXU_1wkEPu",
   "metadata": {
    "id": "1rZXU_1wkEPu"
   },
   "source": [
    "# Step 0: Ask to Access Gemma on Hugging Face\n",
    "\n",
    "<img src=\"https://huggingface.co/datasets/agents-course/course-images/resolve/main/en/bonus-unit1/gemma.png\" alt=\"Gemma\"/>\n",
    "\n",
    "\n",
    "To access Gemma on Hugging Face:\n",
    "\n",
    "1. **Make sure you're signed in** to your Hugging Face Account\n",
    "\n",
    "2. Go to https://huggingface.co/google/gemma-2-2b-it\n",
    "\n",
    "3. Click on **Acknowledge license** and fill the form.\n",
    "\n",
    "Alternatively you can use another model, and modify the code accordingly (it can be a good exercise for you to be sure you know how to fine-tune for Function-Calling).\n",
    "\n",
    "You can use for instance:\n",
    "\n",
    "- [HuggingFaceTB/SmolLM2-1.7B-Instruct](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct)\n",
    "\n",
    "- [meta-llama/Llama-3.2-3B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5hjyx9nJlvKG",
   "metadata": {
    "id": "5hjyx9nJlvKG"
   },
   "source": [
    "## Step 1: Set the GPU 💪\n",
    "\n",
    "If you're on Colab:\n",
    "\n",
    "- To **accelerate the fine-tuning training, we'll use a GPU**. To do that, go to `Runtime > Change Runtime type`\n",
    "\n",
    "<img src=\"https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/notebooks/gpu-step1.jpg\" alt=\"GPU Step 1\"/>\n",
    "\n",
    "- `Hardware Accelerator > GPU`\n",
    "\n",
    "<img src=\"https://huggingface.co/datasets/huggingface-deep-rl-course/course-images/resolve/main/en/notebooks/gpu-step2.jpg\" alt=\"GPU Step 2\"/>\n",
    "\n",
    "\n",
    "### Important\n",
    "\n",
    "For this Unit, **with the free-tier of Colab** it will take around **6h to train**.\n",
    "\n",
    "You have three solutions if you want to make it faster:\n",
    "\n",
    "1. Train on your computer if you have GPUs. It might take time but you have less risks of timeout.\n",
    "\n",
    "2. Use a Google Colab Pro that allows you use to A100 GPU (15-20min training).\n",
    "\n",
    "3. Just follow the code to learn how to do it without training."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5Thjsc9fj6Ej",
   "metadata": {
    "id": "5Thjsc9fj6Ej"
   },
   "source": [
    "## Step 2: Install dependencies 📚\n",
    "\n",
    "We need multiple librairies:\n",
    "\n",
    "- `bitsandbytes` for quantization\n",
    "- `peft`for LoRA adapters\n",
    "- `Transformers`for loading the model\n",
    "- `datasets`for loading and using the fine-tuning dataset\n",
    "- `trl`for the trainer class"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "e63f4962-c644-491e-aa91-50e453e953a4",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "e63f4962-c644-491e-aa91-50e453e953a4",
    "outputId": "443077a6-7cff-4c46-90ac-bf279300f6ec",
    "tags": []
   },
   "outputs": [],
   "source": [
    "!pip install -q -U bitsandbytes\n",
    "!pip install -q -U peft\n",
    "!pip install -q -U trl\n",
    "!pip install -q -U tensorboardX\n",
    "!pip install -q wandb\n",
    "!pip install -q -U torchvision\n",
    "!pip install -q -U transformers"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "UWNoZzi1urSZ",
   "metadata": {
    "id": "UWNoZzi1urSZ"
   },
   "source": [
    "## Step 3: Create your Hugging Face Token to push your model to the Hub\n",
    "\n",
    "To be able to share your model with the community there are some more steps to follow:\n",
    "\n",
    "1️⃣ (If it's not already done) create an account to HF ➡ https://huggingface.co/join\n",
    "\n",
    "2️⃣ Sign in and then, you need to store your authentication token from the Hugging Face website.\n",
    "\n",
    "- Create a new token (https://huggingface.co/settings/tokens) **with write role**\n",
    "\n",
    "<img src=\"https://huggingface.co/datasets/agents-course/course-images/resolve/main/en/bonus-unit1/create_write_token.png\" alt=\"Create HF Token\" width=\"50%\">\n",
    "\n",
    "3️⃣ Store your token as an environment variable under the name \"HF_TOKEN\"\n",
    "- **Be very carefull not to share it with others** !"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "vBAkwg9zu6A1",
   "metadata": {
    "id": "vBAkwg9zu6A1"
   },
   "source": [
    "## Step 4: Import the librairies\n",
    "\n",
    "Don't forget to put your HF token."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "7ad2e4c2-593e-463e-9692-8d674c541d76",
   "metadata": {
    "id": "7ad2e4c2-593e-463e-9692-8d674c541d76",
    "tags": []
   },
   "outputs": [],
   "source": [
    "from enum import Enum\n",
    "from functools import partial\n",
    "import pandas as pd\n",
    "import torch\n",
    "import json\n",
    "\n",
    "from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments, BitsAndBytesConfig, set_seed\n",
    "from datasets import load_dataset\n",
    "from trl import SFTConfig, SFTTrainer\n",
    "from peft import LoraConfig, TaskType\n",
    "\n",
    "seed = 42\n",
    "set_seed(seed)\n",
    "\n",
    "import os\n",
    "\n",
    "# Put your HF Token here\n",
    "os.environ['HF_TOKEN']=\"hf_xxxxxxx\" # the token should have write access"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "44f30b2c-2cc0-48e0-91ca-4633e6444105",
   "metadata": {
    "id": "44f30b2c-2cc0-48e0-91ca-4633e6444105"
   },
   "source": [
    "## Step 5: Processing the dataset into inputs\n",
    "\n",
    "In order to train the model, we need to **format the inputs into what we want the model to learn**.\n",
    "\n",
    "For this tutorial, I enhanced a popular dataset for function calling \"NousResearch/hermes-function-calling-v1\" by adding some new **thinking** step computer from **deepseek-ai/DeepSeek-R1-Distill-Qwen-32B**.\n",
    "\n",
    "But in order for the model to learn, we need **to format the conversation correctly**. If you followed Unit 1, you know that going from a list of messages to a prompt is handled by the **chat_template**, or, the default chat_template of gemma-2-2B does not contain tool calls. So we will need to modify it !\n",
    "\n",
    "This is the role of our **preprocess** function. To go from a list of messages, to a prompt that the model can understand.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "29da85c8-33bf-4864-aed7-733cbe703512",
   "metadata": {
    "id": "29da85c8-33bf-4864-aed7-733cbe703512",
    "tags": []
   },
   "outputs": [],
   "source": [
    "model_name = \"google/gemma-2-2b-it\"\n",
    "dataset_name = \"Jofthomas/hermes-function-calling-thinking-V1\"\n",
    "tokenizer = AutoTokenizer.from_pretrained(model_name)\n",
    "\n",
    "tokenizer.chat_template = \"{{ bos_token }}{% if messages[0]['role'] == 'system' %}{{ raise_exception('System role not supported') }}{% endif %}{% for message in messages %}{{ '<start_of_turn>' + message['role'] + '\\n' + message['content'] | trim + '<end_of_turn><eos>\\n' }}{% endfor %}{% if add_generation_prompt %}{{'<start_of_turn>model\\n'}}{% endif %}\"\n",
    "\n",
    "\n",
    "def preprocess(sample):\n",
    "      messages = sample[\"messages\"]\n",
    "      first_message = messages[0]\n",
    "\n",
    "      # Instead of adding a system message, we merge the content into the first user message\n",
    "      if first_message[\"role\"] == \"system\":\n",
    "          system_message_content = first_message[\"content\"]\n",
    "          # Merge system content with the first user message\n",
    "          messages[1][\"content\"] = system_message_content + \"Also, before making a call to a function take the time to plan the function to take. Make that thinking process between <think>{your thoughts}</think>\\n\\n\" + messages[1][\"content\"]\n",
    "          # Remove the system message from the conversation\n",
    "          messages.pop(0)\n",
    "\n",
    "      return {\"text\": tokenizer.apply_chat_template(messages, tokenize=False)}\n",
    "\n",
    "\n",
    "\n",
    "dataset = load_dataset(dataset_name)\n",
    "dataset = dataset.rename_column(\"conversations\", \"messages\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dc8736d5-d64b-4c5c-9738-be08421d3f95",
   "metadata": {
    "id": "dc8736d5-d64b-4c5c-9738-be08421d3f95"
   },
   "source": [
    "## Step 6: A Dedicated Dataset for This Unit\n",
    "\n",
    "For this Bonus Unit, we created a custom dataset based on [NousResearch/hermes-function-calling-v1](https://huggingface.co/datasets/NousResearch/hermes-function-calling-v1), which is considered a **reference** when it comes to function-calling datasets.\n",
    "\n",
    "While the original dataset is excellent, it does **not** include a **“thinking”** step.\n",
    "\n",
    "In Function-Calling, such a step is optional, but recent work—like the **deepseek** model or the paper [\"Test-Time Compute\"](https://huggingface.co/papers/2408.03314)—suggests that giving an LLM time to “think” before it answers (or in this case, **before** taking an action) can **significantly improve** model performance.\n",
    "\n",
    "I, decided to then compute a subset of this dataset and to give it to [deepseek-ai/DeepSeek-R1-Distill-Qwen-32B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B) in order to compute some thinking tokens `<think>` before any function call. Which resulted in the following dataset :\n",
    "![Input Dataset](https://huggingface.co/datasets/agents-course/course-images/resolve/main/en/bonus-unit1/dataset_function_call.png)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "b63d4832-d92e-482d-9fe6-6e9dbfee377a",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "b63d4832-d92e-482d-9fe6-6e9dbfee377a",
    "outputId": "547d58fc-cf84-4878-f66e-ae817030a251",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b2ee958727314c8186af7ee5e5da64aa",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/3570 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "DatasetDict({\n",
      "    train: Dataset({\n",
      "        features: ['text'],\n",
      "        num_rows: 3213\n",
      "    })\n",
      "    test: Dataset({\n",
      "        features: ['text'],\n",
      "        num_rows: 357\n",
      "    })\n",
      "})\n"
     ]
    }
   ],
   "source": [
    "dataset = dataset.map(preprocess, remove_columns=\"messages\")\n",
    "dataset = dataset[\"train\"].train_test_split(0.1)\n",
    "print(dataset)\n",
    "\n",
    "dataset[\"train\"] = dataset[\"train\"].select(range(100))\n",
    "dataset[\"test\"] = dataset[\"test\"].select(range(10))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "67724a23-f298-4247-b002-2cf370b03897",
   "metadata": {
    "id": "67724a23-f298-4247-b002-2cf370b03897"
   },
   "source": [
    "## Step 7: Checking the inputs\n",
    "\n",
    "Let's manually look at what an input looks like !\n",
    "\n",
    "In this example we have :\n",
    "\n",
    "1. A *User message* containing the **necessary information with the list of available tools** inbetween `<tools></tools>` then the user query, here:  `\"Can you get me the latest news headlines for the United States?\"`\n",
    "\n",
    "2. An *Assistant message* here called \"model\" to fit the criterias from gemma models containing two new phases, a **\"thinking\"** phase contained in `<think></think>` and an **\"Act\"** phase contained in `<tool_call></<tool_call>`.\n",
    "\n",
    "3. If the model contains a `<tools_call>`, we will append the result of this action in a new **\"Tool\"** message containing a `<tool_response></tool_response>` with the answer from the tool."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "dc60da04-9411-487a-b629-2c59024a20c0",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "dc60da04-9411-487a-b629-2c59024a20c0",
    "outputId": "8af76a10-5a72-4401-cdf5-ee047fc2d850",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<bos><start_of_turn>human\n",
      "You are a function calling AI model. You are provided with function signatures within <tools></tools> XML tags.You may call one or more functions to assist with the user query. Don't make assumptions about what values to plug into functions.Here are the available tools:<tools> [{'type': 'function', 'function': {'name': 'get_news_headlines', 'description': 'Get the latest news headlines', 'parameters': {'type': 'object', 'properties': {'country': {'type': 'string', 'description': 'The country for which headlines are needed'}}, 'required': ['country']}}}, {'type': 'function', 'function': {'name': 'search_recipes', 'description': 'Search for recipes based on ingredients', 'parameters': {'type': 'object', 'properties': {'ingredients': {'type': 'array', 'items': {'type': 'string'}, 'description': 'The list of ingredients'}}, 'required': ['ingredients']}}}] </tools>Use the following pydantic model json schema for each tool call you will make: {'title': 'FunctionCall', 'type': 'object', 'properties': {'arguments': {'title': 'Arguments', 'type': 'object'}, 'name': {'title': 'Name', 'type': 'string'}}, 'required': ['arguments', 'name']}For each function call return a json object with function name and arguments within <tool_call></tool_call> XML tags as follows:\n",
      "<tool_call>\n",
      "{tool_call}\n",
      "</tool_call>Also, before making a call to a function take the time to plan the function to take. Make that thinking process between <think>{your thoughts}</think>\n",
      "\n",
      "Can you get me the latest news headlines for the United States?<end_of_turn><eos>\n",
      "<start_of_turn>model\n",
      "<think>Okay, so the user asked for the latest news headlines in the United States. I need to figure out how to respond appropriately using the tools available.\n",
      "\n",
      "Looking at the tools provided, there's a function called get_news_headlines. That seems perfect for this situation. It requires a country code, which the user specified as the United States.\n",
      "\n",
      "Since the user is asking specifically about the US, I'll use the country parameter as 'United States' in the function call. This should fetch the relevant news for them.\n",
      "\n",
      "I should make sure to format the response correctly as a JSON object with the function name and arguments.\n",
      "</think><tool_call>\n",
      "{'name': 'get_news_headlines', 'arguments': {'country': 'United States'}}\n",
      "</tool_call><end_of_turn><eos>\n",
      "<start_of_turn>tool\n",
      "<tool_response>\n",
      "{'headlines': ['US President signs new bill into law', 'Major earthquake hits California', 'NASA announces new Mars mission', 'Tech giant releases new smartphone', 'US economy shows signs of recovery']}\n",
      "</tool_response><end_of_turn><eos>\n",
      "<start_of_turn>model\n",
      "Here are the latest news headlines for the United States:\n",
      "1. US President signs new bill into law\n",
      "2. Major earthquake hits California\n",
      "3. NASA announces new Mars mission\n",
      "4. Tech giant releases new smartphone\n",
      "5. US economy shows signs of recovery<end_of_turn><eos>\n",
      "<start_of_turn>human\n",
      "That's interesting. Can you get me the news for France now?<end_of_turn><eos>\n",
      "<start_of_turn>model\n",
      "<think>Okay, the user initially asked for the latest US news, so I used get_news_headlines with 'United States'. Now the user is asking for France, so I'll call the same function with 'France' as the country. That makes sense because the function is designed for any country.\n",
      "</think><tool_call>\n",
      "{'name': 'get_news_headlines', 'arguments': {'country': 'France'}}\n",
      "</tool_call><end_of_turn><eos>\n",
      "<start_of_turn>tool\n",
      "<tool_response>\n",
      "{'headlines': ['French President announces new environmental policy', 'Paris Fashion Week highlights', 'France wins World Cup qualifier', 'New culinary trend sweeps across France', 'French tech startup raises millions in funding']}\n",
      "</tool_response><end_of_turn><eos>\n",
      "<start_of_turn>model\n",
      "Here are the latest news headlines for France:\n",
      "1. French President announces new environmental policy\n",
      "2. Paris Fashion Week highlights\n",
      "3. France wins World Cup qualifier\n",
      "4. New culinary trend sweeps across France\n",
      "5. French tech startup raises millions in funding<end_of_turn><eos>\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Let's look at how we formatted the dataset\n",
    "print(dataset[\"train\"][8][\"text\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "53a48281-2346-4dfb-ad60-cad85129ec9b",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "53a48281-2346-4dfb-ad60-cad85129ec9b",
    "outputId": "e16c07f9-8bbf-4c82-d2d3-41d96a95ab2e",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<pad>\n",
      "<eos>\n"
     ]
    }
   ],
   "source": [
    "# Sanity check\n",
    "print(tokenizer.pad_token)\n",
    "print(tokenizer.eos_token)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d6864b36-6033-445a-b6e2-b6bb02e38e26",
   "metadata": {
    "id": "d6864b36-6033-445a-b6e2-b6bb02e38e26"
   },
   "source": [
    "## Step 8: Let's Modify the Tokenizer\n",
    "\n",
    "Indeed, as we saw in Unit 1, the tokenizer splits text into sub-words by default. This is **not** what we want for our new special tokens!\n",
    "\n",
    "While we segmented our example using `<think>`, `<tool_call>`, and `<tool_response>`, the tokenizer does **not** yet treat them as whole tokens—it still tries to break them down into smaller pieces. To ensure the model correctly interprets our new format, we must **add these tokens** to our tokenizer.\n",
    "\n",
    "Additionally, since we changed the `chat_template` in our **preprocess** function to format conversations as messages within a prompt, we also need to modify the `chat_template` in the tokenizer to reflect these changes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "833ba5d6-4c1e-4689-9fed-22cc03d2a63a",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 754,
     "referenced_widgets": [
      "54fc2d9962de4ffda0500a01e112b58a",
      "8e70cccb0fb04a43b3c6ab234bcae9a8",
      "f2d87447fe8448c7baff656c540620e6",
      "5f438fbbfb60436faf79662e81092154",
      "197619158cd247649eac3722284c8d19",
      "1d104fe8612f45e4810bbed7e3540330",
      "d62425b0d0ea497ead2cd981eaa61c7e",
      "2828b2455ec44143a5739c8e8207fb9b",
      "03b839beea6a4f6b8d9005491146615d",
      "0db5d5c516d7479ebf7603093f9e1e99",
      "654fd18a9e59415e9dd314dc2f94a654",
      "deba87fc28db4f78b58fc1e456fea335",
      "f06414fa35b64ff588511796c9b51d1f",
      "21cf377576154ffeb028f4ae3b3503d5",
      "5c8fa13e559942849a61d9480be730bd",
      "fae24e4a14814c7ebef30b422c2d3652",
      "1981ff353e494735be4706a76053c24a",
      "9a8b33ef0f6c446496cc7dae07da321c",
      "2b8c9e43da4e4e008a2a2b06e6f3106e",
      "62f3a32c4f484dd6899ba5d6ffbaca11",
      "1e271fcb92a14680b84c33cb7c28f9c0",
      "da543c28b9e44723b019220de8427b5e",
      "ce3c69ec86224ffea8c46deb20831bb7",
      "3bb2805fcf0441089c0f4789e28b072d",
      "bc6e638133f545ff8d54f767de0cb35a",
      "6e1393f9f39a4b3cb94753c883b1bbf7",
      "a76145e51f7544fdabd23198baf318c4",
      "eedc1213dd604bbe827dcec617fdae34",
      "f73f91bb567a48f890ba0827b7ed321f",
      "a8edc3a5b0d9422b8e2e63d5564cfdc9",
      "e08c970bbf4540a89533a43f719f68e4",
      "fe30dac330db4fde94af1c54c84a1be8",
      "4cfce958cf5445129ab6281784664038",
      "71ba28670e8d4583b66b1c1587cdbce1",
      "4349fab6f16343cb8955367f6fbc4a43",
      "6296f0c3ac5647e9b9cbc7c8212a2b35",
      "c6d97ccbecb0407f89ef4469f3874e74",
      "ccabb079b6fd466fb877a018aebf793c",
      "0ff4d8414af94af59628b0417a40cc7d",
      "534a5dc9cd7e475db2be613849959fc7",
      "b821e13722e24feb941fa31a604a85e9",
      "91c21b2b127640e195f2e7c7256b5a0e",
      "1296a8a4baea4faabc62a42b2ca1a53a",
      "13b49790234c4d1aa8fe9b532fcf95bf",
      "e1ef7211c5554b2f8fdd39f2f846d2d9",
      "9ac674eeba0542e7b4e28a13e90ce652",
      "cc4993f1af6b4f2685ecbced238b8913",
      "5ec98ede3fbf4c64843b935ce3ad28a7",
      "bdbd9e7e1d674edeab47e3595f6af4f8",
      "2eb5230a43e84790b0392db58157b0f1",
      "26bd63db6a384c12b7d42bf95ca1d1bc",
      "6f4e106bb5b44f398a1fe4ac588f6cca",
      "0c881b9322624d70bb2c5147a5422633",
      "ec8ecb8bdee4435fbc97d73863ebd9f8",
      "1af481e6544a4fa386815218461cd5c3",
      "2f73c87217c84bd293a5ea5b4aba0ae5",
      "6883c5e46d3948649488d3e9f81ddb9c",
      "d0498872e00f4644847e073c6000a111",
      "482f477cc2354d9da6a8c2932bb73e98",
      "1c810f5364544762a3093e1a0a9c1e5b",
      "681019ca1d084d9692f5085d1df520ae",
      "b1bb9bcec30144a380fa4fe440c41d7c",
      "425fe267a38748ea9670a0996b802bdc",
      "95eebbfed6fc41d6a8c668abe1608756",
      "51b42c72bc2b428fbc0389bfd4d9e598",
      "611a6d951d2a42b9b650496612cfd484",
      "a893802af41d4400b35b9cb71add1387",
      "45e867ccde7d4b5bb4ed4861d723a758",
      "f5e196c908e7491982773d3231fff3e6",
      "68baf2700f614be288bf35e54d207096",
      "001b32600fdd418bb30c6b5ff85e269c",
      "2bcef869f1814a04bbbed73df7a9ab24",
      "e3bd52cd621e4d2196e94327324722c4",
      "1e9d8fccba9a468291ffe271b3497830",
      "d1d376228c334c5999143b905e234ffc",
      "fc9a32511862493e980682b6ff5044bb",
      "538f6a3c632f431fa9d16ab17383a602"
     ]
    },
    "id": "833ba5d6-4c1e-4689-9fed-22cc03d2a63a",
    "outputId": "3138107e-a2cd-447d-ee69-e07e401a1540",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "bedabbdf4cf44865a83fc9702391c298",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Gemma2ForCausalLM(\n",
       "  (model): Gemma2Model(\n",
       "    (embed_tokens): Embedding(256008, 2304, padding_idx=0)\n",
       "    (layers): ModuleList(\n",
       "      (0-25): 26 x Gemma2DecoderLayer(\n",
       "        (self_attn): Gemma2Attention(\n",
       "          (q_proj): Linear(in_features=2304, out_features=2048, bias=False)\n",
       "          (k_proj): Linear(in_features=2304, out_features=1024, bias=False)\n",
       "          (v_proj): Linear(in_features=2304, out_features=1024, bias=False)\n",
       "          (o_proj): Linear(in_features=2048, out_features=2304, bias=False)\n",
       "          (rotary_emb): Gemma2RotaryEmbedding()\n",
       "        )\n",
       "        (mlp): Gemma2MLP(\n",
       "          (gate_proj): Linear(in_features=2304, out_features=9216, bias=False)\n",
       "          (up_proj): Linear(in_features=2304, out_features=9216, bias=False)\n",
       "          (down_proj): Linear(in_features=9216, out_features=2304, bias=False)\n",
       "          (act_fn): PytorchGELUTanh()\n",
       "        )\n",
       "        (input_layernorm): Gemma2RMSNorm((2304,), eps=1e-06)\n",
       "        (pre_feedforward_layernorm): Gemma2RMSNorm((2304,), eps=1e-06)\n",
       "        (post_feedforward_layernorm): Gemma2RMSNorm((2304,), eps=1e-06)\n",
       "        (post_attention_layernorm): Gemma2RMSNorm((2304,), eps=1e-06)\n",
       "      )\n",
       "    )\n",
       "    (norm): Gemma2RMSNorm((2304,), eps=1e-06)\n",
       "  )\n",
       "  (lm_head): Linear(in_features=2304, out_features=256008, bias=False)\n",
       ")"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "class ChatmlSpecialTokens(str, Enum):\n",
    "    tools = \"<tools>\"\n",
    "    eotools = \"</tools>\"\n",
    "    think = \"<think>\"\n",
    "    eothink = \"</think>\"\n",
    "    tool_call=\"<tool_call>\"\n",
    "    eotool_call=\"</tool_call>\"\n",
    "    tool_response=\"<tool_reponse>\"\n",
    "    eotool_response=\"</tool_reponse>\"\n",
    "    pad_token = \"<pad>\"\n",
    "    eos_token = \"<eos>\"\n",
    "    @classmethod\n",
    "    def list(cls):\n",
    "        return [c.value for c in cls]\n",
    "\n",
    "tokenizer = AutoTokenizer.from_pretrained(\n",
    "        model_name,\n",
    "        pad_token=ChatmlSpecialTokens.pad_token.value,\n",
    "        additional_special_tokens=ChatmlSpecialTokens.list()\n",
    "    )\n",
    "tokenizer.chat_template = \"{{ bos_token }}{% if messages[0]['role'] == 'system' %}{{ raise_exception('System role not supported') }}{% endif %}{% for message in messages %}{{ '<start_of_turn>' + message['role'] + '\\n' + message['content'] | trim + '<end_of_turn><eos>\\n' }}{% endfor %}{% if add_generation_prompt %}{{'<start_of_turn>model\\n'}}{% endif %}\"\n",
    "\n",
    "model = AutoModelForCausalLM.from_pretrained(model_name,\n",
    "                                             attn_implementation='eager',\n",
    "                                             device_map=\"auto\")\n",
    "model.resize_token_embeddings(len(tokenizer))\n",
    "model.to(torch.bfloat16)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "X6DBY8AqxFLL",
   "metadata": {
    "id": "X6DBY8AqxFLL"
   },
   "source": [
    "## Step 9: Let's configure the LoRA\n",
    "\n",
    "This is we are going to define the parameter of our adapter. Those are the most important parameters in LoRA as they define the size and importance of the adapters we are training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "482d36ab-e326-4fd7-bc59-425abcca55e7",
   "metadata": {
    "id": "482d36ab-e326-4fd7-bc59-425abcca55e7",
    "tags": []
   },
   "outputs": [],
   "source": [
    "from peft import LoraConfig\n",
    "\n",
    "# TODO: Configure LoRA parameters\n",
    "# r: rank dimension for LoRA update matrices (smaller = more compression)\n",
    "rank_dimension = 16\n",
    "# lora_alpha: scaling factor for LoRA layers (higher = stronger adaptation)\n",
    "lora_alpha = 64\n",
    "# lora_dropout: dropout probability for LoRA layers (helps prevent overfitting)\n",
    "lora_dropout = 0.05\n",
    "\n",
    "peft_config = LoraConfig(r=rank_dimension,\n",
    "                         lora_alpha=lora_alpha,\n",
    "                         lora_dropout=lora_dropout,\n",
    "                         target_modules=[\"gate_proj\",\"q_proj\",\"lm_head\",\"o_proj\",\"k_proj\",\"embed_tokens\",\"down_proj\",\"up_proj\",\"v_proj\"], # wich layer in the transformers do we target ?\n",
    "                         task_type=TaskType.CAUSAL_LM)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "zdDR9hzgxPu2",
   "metadata": {
    "id": "zdDR9hzgxPu2"
   },
   "source": [
    "## Step 10: Let's define the Trainer and the Fine-Tuning hyperparameters\n",
    "\n",
    "In this step, we define the Trainer, the class that we use to fine-tune our model and the hyperparameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "3598b688-5a6f-437f-95ac-4794688cd38f",
   "metadata": {
    "id": "3598b688-5a6f-437f-95ac-4794688cd38f",
    "tags": []
   },
   "outputs": [],
   "source": [
    "username=\"Jofthomas\"# REPLCAE with your Hugging Face username\n",
    "output_dir = \"gemma-2-2B-it-thinking-function_calling-V0\" # The directory where the trained model checkpoints, logs, and other artifacts will be saved. It will also be the default name of the model when pushed to the hub if not redefined later.\n",
    "per_device_train_batch_size = 1\n",
    "per_device_eval_batch_size = 1\n",
    "gradient_accumulation_steps = 4\n",
    "logging_steps = 5\n",
    "learning_rate = 1e-4 # The initial learning rate for the optimizer.\n",
    "\n",
    "max_grad_norm = 1.0\n",
    "num_train_epochs=1\n",
    "warmup_ratio = 0.1\n",
    "lr_scheduler_type = \"cosine\"\n",
    "max_seq_length = 1500\n",
    "\n",
    "training_arguments = SFTConfig(\n",
    "    output_dir=output_dir,\n",
    "    per_device_train_batch_size=per_device_train_batch_size,\n",
    "    per_device_eval_batch_size=per_device_eval_batch_size,\n",
    "    gradient_accumulation_steps=gradient_accumulation_steps,\n",
    "    save_strategy=\"no\",\n",
    "    eval_strategy=\"epoch\",\n",
    "    logging_steps=logging_steps,\n",
    "    learning_rate=learning_rate,\n",
    "    max_grad_norm=max_grad_norm,\n",
    "    weight_decay=0.1,\n",
    "    warmup_ratio=warmup_ratio,\n",
    "    lr_scheduler_type=lr_scheduler_type,\n",
    "    report_to=\"tensorboard\",\n",
    "    bf16=True,\n",
    "    hub_private_repo=False,\n",
    "    push_to_hub=False,\n",
    "    num_train_epochs=num_train_epochs,\n",
    "    gradient_checkpointing=True,\n",
    "    gradient_checkpointing_kwargs={\"use_reentrant\": False},\n",
    "    packing=True,\n",
    "    max_seq_length=max_seq_length,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59TTqmW2xmV2",
   "metadata": {
    "id": "59TTqmW2xmV2"
   },
   "source": [
    "As Trainer, we use the `SFTTrainer` which is a Supervised Fine-Tuning Trainer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "ba0366b5-c9d0-4f7e-97e0-1f964cfad147",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 263,
     "referenced_widgets": [
      "e3fe61834c3e49a3895212a336776f9d",
      "472cbf91b24f46829960d68e2316c417",
      "bd746ea2e46a491e954ac6f32fb0e45b",
      "b1542891fc6243d98d51981dd0584bdf",
      "b2e16ad7540d4760b28f3a8c419905f8",
      "f2157c83879046a29b72613bce9de56e",
      "3b07ec7f4d024b61abe94c8adeebed55",
      "e6f810bd430245b190ee932554cca05c",
      "952893c941c346c2aedcd9358859a3b9",
      "d36e7dd4b9dc497faa6e8f63843a738f",
      "f469ec8c79ac476c82a5e228f347bffa",
      "e9de72aadc5743a2b56537b3ad035461",
      "57d137229091486dbf0e4b7dd6dce98a",
      "4d006dc72b2b45b58caf4d398c3756b8",
      "36bbd4bd563a4053a7af8532300253b7",
      "cacc3c3a10c64b338866a8e42201e44c",
      "b4c43d908bd64d9bbdb488dac46d2e45",
      "96691a6287ef401582a2a1744a4940c4",
      "328fa6d902bd46bbb0ecdc7404a13e8c",
      "f375fed157034dfcbd28744027d77eba",
      "33ec043a635f4e99b67cd6f7e6fc6193",
      "39fee7a249c146d1a166e58755c1cda8",
      "2ccddb85840c4981ae089ae4c74a2de6",
      "89eee480405a416ba0edf097423724b9",
      "56e8079c374a4f3f9af5ef96a73f2955",
      "3aeb28bc164444d7afb7bf7435a001f2",
      "1ead218e71614374909b92fda097fd42",
      "822bea0ac84d4ff29d984bf5f5d2c3a2",
      "ecef440c871b4daba34661a1ddba6b0c",
      "e633387b1640461e82617c1702ee82f5",
      "26484831a87a4b489e1288ea71ea7767",
      "9f8d631358e240f982e31171d1bd9f26",
      "a0c7029819414c5dacefed93194cd763",
      "f4a01e54ec53475585eaa88b3a272b4b",
      "63c269b37eed4d348f9ce24eef15fc15",
      "a89230859593424e960047a96977c6b8",
      "91163fcffc60438cb39b0eb586dac418",
      "89033e9c0dd249db9dc9a3b1e215dded",
      "73c9d510c8754f2ea21adf318e35bc8e",
      "0fe7751f55134695afb44bf8673dd4d9",
      "f7dd34e15348462297564f0e6e0b568d",
      "537e188c000041fea6adf26f2255d738",
      "39fff32b9756437581228465165a3115",
      "4f75329d3e8d4cc38a405c1c4cc51d70",
      "8bbe22288edf4a06b2c56952fd81d5e7",
      "298f092855f14af79ec2eda792732810",
      "2e6392b95f8a48568a89780adf76387f",
      "55e72b7f262b4f57a6abb1c9f01c8de2",
      "9fbca7fa0d6b4fff910b806a97fa7718",
      "a385bba49b514ab386cb5f4cbb01821f",
      "78a34eb9cd534c3d84c8f22d3c53c88f",
      "cee71fbbf8a04b3bb64b96e7fa2b0b0e",
      "0a0bc95f445948f486fbce865a4642f2",
      "b782092e2282488ba86f85eebe697603",
      "f7ba9e0f4e64484a82374bb5f1d12b15",
      "39c0963803c74ebab07cec20e10d0184",
      "46b69fb951af44f99232f459daa4f103",
      "3e23b4c5848843fcb44dc0ae2f157d66",
      "c9f86634a6bf4e49a902e3d42e67f1bd",
      "03b8a027c56f4b52a3e54f57bb5bb526",
      "58a18918bae34aca8ec73ae89fd5bc24",
      "fd6e1776cbcd4f7b96ec6d9754eb2c83",
      "b5121cada3514b67a9c533e7468b3058",
      "bf70daf78057419b8a78af75a093a3dd",
      "e0b3b3c072be44de8e0a2dae91598aa6",
      "4e21f1bd903443a89dea32bb3f3c26a9"
     ]
    },
    "id": "ba0366b5-c9d0-4f7e-97e0-1f964cfad147",
    "outputId": "5602ad04-17c4-431b-e20d-7f0d0c7fd24e",
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/user/miniconda/lib/python3.9/site-packages/peft/tuners/tuners_utils.py:543: UserWarning: Model with `tie_word_embeddings=True` and the tied_target_modules=['lm_head'] are part of the adapter. This can lead to complications, for example when merging the adapter or converting your model to formats other than safetensors. See for example https://github.com/huggingface/peft/issues/2018.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "be2be97d4aa64b23beaf316024229a3b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Applying chat template to train dataset:   0%|          | 0/100 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f7416500f850456792cacf42656df46a",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Tokenizing train dataset:   0%|          | 0/100 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "482ded5a74484b5f8d8907b4a886f946",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Packing train dataset:   0%|          | 0/100 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "3f59849c47ef42d0ac295296a4ad0a91",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Applying chat template to eval dataset:   0%|          | 0/10 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b1acbd0589c54b5887ea7b3e8b872057",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Tokenizing eval dataset:   0%|          | 0/10 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "24a119a8500146f28f3d866ed69f7161",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Packing eval dataset:   0%|          | 0/10 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "trainer = SFTTrainer(\n",
    "    model=model,\n",
    "    args=training_arguments,\n",
    "    train_dataset=dataset[\"train\"],\n",
    "    eval_dataset=dataset[\"test\"],\n",
    "    processing_class=tokenizer,\n",
    "    peft_config=peft_config,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "MtHjukK9xviB",
   "metadata": {
    "id": "MtHjukK9xviB"
   },
   "source": [
    "Here, we launch the training 🔥. Perfect time for you to pause and grab a coffee ☕."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "9e2df2e9-a82b-4540-aa89-1b40b70a7781",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 92
    },
    "id": "9e2df2e9-a82b-4540-aa89-1b40b70a7781",
    "outputId": "113322de-8f42-4285-d58f-8641e6daf295",
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='12' max='12' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [12/12 00:24, Epoch 1/1]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       " <tr style=\"text-align: left;\">\n",
       "      <th>Epoch</th>\n",
       "      <th>Training Loss</th>\n",
       "      <th>Validation Loss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>1.236800</td>\n",
       "      <td>1.240833</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/user/miniconda/lib/python3.9/site-packages/peft/utils/save_and_load.py:230: UserWarning: Setting `save_embedding_layers` to `True` as embedding layers found in `target_modules`.\n",
      "  warnings.warn(\"Setting `save_embedding_layers` to `True` as embedding layers found in `target_modules`.\")\n"
     ]
    }
   ],
   "source": [
    "trainer.train()\n",
    "trainer.save_model()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d7ea3ab-7c8c-47ad-acd2-99fbe5b68393",
   "metadata": {
    "id": "1d7ea3ab-7c8c-47ad-acd2-99fbe5b68393",
    "tags": []
   },
   "source": [
    "## Step 11: Let's push the Model and the Tokenizer to the Hub\n",
    "\n",
    "Let's push our model and out tokenizer to the Hub ! The model will be pushed under your username + the output_dir that we specified earlier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "370af020-9319-4ff7-bea1-2842a4847caa",
   "metadata": {
    "id": "370af020-9319-4ff7-bea1-2842a4847caa",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "00c6c786a8014635952d94bb505923f1",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "events.out.tfevents.1739887545.r-jofthomas-fttest-kff5bkw4-24c03-yhiku:   0%|          | 0.00/6.88k [00:00<?, …"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "6945a9a4ba05483e94061404480e2d4f",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "training_args.bin:   0%|          | 0.00/5.62k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d56b088f78144a3388749363ee944194",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "tokenizer.json:   0%|          | 0.00/34.4M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "eb6e24851a254a1b81a2032541453ec0",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "adapter_model.safetensors:   0%|          | 0.00/2.48G [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "6f268fc2255c46ac99cf15212035009c",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Upload 4 LFS files:   0%|          | 0/4 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "CommitInfo(commit_url='https://huggingface.co/Jofthomas/gemma-2-2B-it-thinking-function_calling-V0/commit/74db7b9fd8e5c6591db5851db70069bf3016fe50', commit_message='Jofthomas/gemma-2-2B-it-thinking-function_calling-V0', commit_description='', oid='74db7b9fd8e5c6591db5851db70069bf3016fe50', pr_url=None, repo_url=RepoUrl('https://huggingface.co/Jofthomas/gemma-2-2B-it-thinking-function_calling-V0', endpoint='https://huggingface.co', repo_type='model', repo_id='Jofthomas/gemma-2-2B-it-thinking-function_calling-V0'), pr_revision=None, pr_num=None)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainer.push_to_hub(f\"{username}/{output_dir}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "83a443ce-5072-4777-8621-cd4faf840410",
   "metadata": {
    "id": "83a443ce-5072-4777-8621-cd4faf840410"
   },
   "source": [
    "Since we also modified the **chat_template** Which is contained in the tokenizer, let's also push the tokenizer with the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "9d9a86b3-f23d-4060-a97f-b868a7c38c36",
   "metadata": {
    "id": "9d9a86b3-f23d-4060-a97f-b868a7c38c36",
    "outputId": "2726291c-5720-473e-ed92-e4f425f82bae",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1a84942726104c7a881b352df849997e",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "README.md:   0%|          | 0.00/1.53k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "CommitInfo(commit_url='https://huggingface.co/Jofthomas/gemma-2-2B-it-thinking-function_calling-V0/commit/5eae94697866df5f9f52bc7c7be677f100a7f339', commit_message='Upload tokenizer', commit_description='', oid='5eae94697866df5f9f52bc7c7be677f100a7f339', pr_url=None, repo_url=RepoUrl('https://huggingface.co/Jofthomas/gemma-2-2B-it-thinking-function_calling-V0', endpoint='https://huggingface.co', repo_type='model', repo_id='Jofthomas/gemma-2-2B-it-thinking-function_calling-V0'), pr_revision=None, pr_num=None)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokenizer.eos_token = \"<eos>\"\n",
    "# push the tokenizer to hub ( replace with your username and your previously specified\n",
    "tokenizer.push_to_hub(f\"{username}/{output_dir}\", token=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76d275ce-a3e6-4d30-8d8c-0ee274de5370",
   "metadata": {
    "id": "76d275ce-a3e6-4d30-8d8c-0ee274de5370"
   },
   "source": [
    "## Step 12: Let's now test our model !\n",
    "\n",
    "To so, we will :\n",
    "\n",
    "1. Load the adapter from the hub !\n",
    "2. Load the base model : **\"google/gemma-2-2b-it\"** from the hub\n",
    "3. Resize the model to with the new tokens we introduced !"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "56b89825-70ac-42c1-934c-26e2d54f3b7b",
   "metadata": {
    "colab": {
     "referenced_widgets": [
      "390c54434b6448b988ce015eeafe34c9",
      "35b2fe2d357b46488ccef710f2a9bfd7",
      "9c313149d4324bdaa9c8ddc373964d18"
     ]
    },
    "id": "56b89825-70ac-42c1-934c-26e2d54f3b7b",
    "outputId": "a4cd00b8-61fa-4522-d563-c4ef7e18807d",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "4c2546e08a424179af511d8abe3c1c7d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "adapter_config.json:   0%|          | 0.00/829 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e08b968b1648474e812283fa7c4358a9",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "bec06c857d234a7ba0593a92272ef764",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "tokenizer_config.json:   0%|          | 0.00/47.9k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "5608d64f66fa43ef8c162a6e50fb5359",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "tokenizer.json:   0%|          | 0.00/34.4M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8fd1767e3a8249f5becdab5ca1ccea89",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "special_tokens_map.json:   0%|          | 0.00/605 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a6173c485914437cb0147583215e8d84",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "adapter_model.safetensors:   0%|          | 0.00/2.48G [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "PeftModelForCausalLM(\n",
       "  (base_model): LoraModel(\n",
       "    (model): Gemma2ForCausalLM(\n",
       "      (model): Gemma2Model(\n",
       "        (embed_tokens): lora.Embedding(\n",
       "          (base_layer): Embedding(256006, 2304, padding_idx=0)\n",
       "          (lora_dropout): ModuleDict(\n",
       "            (default): Dropout(p=0.05, inplace=False)\n",
       "          )\n",
       "          (lora_A): ModuleDict()\n",
       "          (lora_B): ModuleDict()\n",
       "          (lora_embedding_A): ParameterDict(  (default): Parameter containing: [torch.cuda.BFloat16Tensor of size 16x256006 (cuda:0)])\n",
       "          (lora_embedding_B): ParameterDict(  (default): Parameter containing: [torch.cuda.BFloat16Tensor of size 2304x16 (cuda:0)])\n",
       "          (lora_magnitude_vector): ModuleDict()\n",
       "        )\n",
       "        (layers): ModuleList(\n",
       "          (0-25): 26 x Gemma2DecoderLayer(\n",
       "            (self_attn): Gemma2Attention(\n",
       "              (q_proj): lora.Linear(\n",
       "                (base_layer): Linear(in_features=2304, out_features=2048, bias=False)\n",
       "                (lora_dropout): ModuleDict(\n",
       "                  (default): Dropout(p=0.05, inplace=False)\n",
       "                )\n",
       "                (lora_A): ModuleDict(\n",
       "                  (default): Linear(in_features=2304, out_features=16, bias=False)\n",
       "                )\n",
       "                (lora_B): ModuleDict(\n",
       "                  (default): Linear(in_features=16, out_features=2048, bias=False)\n",
       "                )\n",
       "                (lora_embedding_A): ParameterDict()\n",
       "                (lora_embedding_B): ParameterDict()\n",
       "                (lora_magnitude_vector): ModuleDict()\n",
       "              )\n",
       "              (k_proj): lora.Linear(\n",
       "                (base_layer): Linear(in_features=2304, out_features=1024, bias=False)\n",
       "                (lora_dropout): ModuleDict(\n",
       "                  (default): Dropout(p=0.05, inplace=False)\n",
       "                )\n",
       "                (lora_A): ModuleDict(\n",
       "                  (default): Linear(in_features=2304, out_features=16, bias=False)\n",
       "                )\n",
       "                (lora_B): ModuleDict(\n",
       "                  (default): Linear(in_features=16, out_features=1024, bias=False)\n",
       "                )\n",
       "                (lora_embedding_A): ParameterDict()\n",
       "                (lora_embedding_B): ParameterDict()\n",
       "                (lora_magnitude_vector): ModuleDict()\n",
       "              )\n",
       "              (v_proj): lora.Linear(\n",
       "                (base_layer): Linear(in_features=2304, out_features=1024, bias=False)\n",
       "                (lora_dropout): ModuleDict(\n",
       "                  (default): Dropout(p=0.05, inplace=False)\n",
       "                )\n",
       "                (lora_A): ModuleDict(\n",
       "                  (default): Linear(in_features=2304, out_features=16, bias=False)\n",
       "                )\n",
       "                (lora_B): ModuleDict(\n",
       "                  (default): Linear(in_features=16, out_features=1024, bias=False)\n",
       "                )\n",
       "                (lora_embedding_A): ParameterDict()\n",
       "                (lora_embedding_B): ParameterDict()\n",
       "                (lora_magnitude_vector): ModuleDict()\n",
       "              )\n",
       "              (o_proj): lora.Linear(\n",
       "                (base_layer): Linear(in_features=2048, out_features=2304, bias=False)\n",
       "                (lora_dropout): ModuleDict(\n",
       "                  (default): Dropout(p=0.05, inplace=False)\n",
       "                )\n",
       "                (lora_A): ModuleDict(\n",
       "                  (default): Linear(in_features=2048, out_features=16, bias=False)\n",
       "                )\n",
       "                (lora_B): ModuleDict(\n",
       "                  (default): Linear(in_features=16, out_features=2304, bias=False)\n",
       "                )\n",
       "                (lora_embedding_A): ParameterDict()\n",
       "                (lora_embedding_B): ParameterDict()\n",
       "                (lora_magnitude_vector): ModuleDict()\n",
       "              )\n",
       "              (rotary_emb): Gemma2RotaryEmbedding()\n",
       "            )\n",
       "            (mlp): Gemma2MLP(\n",
       "              (gate_proj): lora.Linear(\n",
       "                (base_layer): Linear(in_features=2304, out_features=9216, bias=False)\n",
       "                (lora_dropout): ModuleDict(\n",
       "                  (default): Dropout(p=0.05, inplace=False)\n",
       "                )\n",
       "                (lora_A): ModuleDict(\n",
       "                  (default): Linear(in_features=2304, out_features=16, bias=False)\n",
       "                )\n",
       "                (lora_B): ModuleDict(\n",
       "                  (default): Linear(in_features=16, out_features=9216, bias=False)\n",
       "                )\n",
       "                (lora_embedding_A): ParameterDict()\n",
       "                (lora_embedding_B): ParameterDict()\n",
       "                (lora_magnitude_vector): ModuleDict()\n",
       "              )\n",
       "              (up_proj): lora.Linear(\n",
       "                (base_layer): Linear(in_features=2304, out_features=9216, bias=False)\n",
       "                (lora_dropout): ModuleDict(\n",
       "                  (default): Dropout(p=0.05, inplace=False)\n",
       "                )\n",
       "                (lora_A): ModuleDict(\n",
       "                  (default): Linear(in_features=2304, out_features=16, bias=False)\n",
       "                )\n",
       "                (lora_B): ModuleDict(\n",
       "                  (default): Linear(in_features=16, out_features=9216, bias=False)\n",
       "                )\n",
       "                (lora_embedding_A): ParameterDict()\n",
       "                (lora_embedding_B): ParameterDict()\n",
       "                (lora_magnitude_vector): ModuleDict()\n",
       "              )\n",
       "              (down_proj): lora.Linear(\n",
       "                (base_layer): Linear(in_features=9216, out_features=2304, bias=False)\n",
       "                (lora_dropout): ModuleDict(\n",
       "                  (default): Dropout(p=0.05, inplace=False)\n",
       "                )\n",
       "                (lora_A): ModuleDict(\n",
       "                  (default): Linear(in_features=9216, out_features=16, bias=False)\n",
       "                )\n",
       "                (lora_B): ModuleDict(\n",
       "                  (default): Linear(in_features=16, out_features=2304, bias=False)\n",
       "                )\n",
       "                (lora_embedding_A): ParameterDict()\n",
       "                (lora_embedding_B): ParameterDict()\n",
       "                (lora_magnitude_vector): ModuleDict()\n",
       "              )\n",
       "              (act_fn): PytorchGELUTanh()\n",
       "            )\n",
       "            (input_layernorm): Gemma2RMSNorm((2304,), eps=1e-06)\n",
       "            (pre_feedforward_layernorm): Gemma2RMSNorm((2304,), eps=1e-06)\n",
       "            (post_feedforward_layernorm): Gemma2RMSNorm((2304,), eps=1e-06)\n",
       "            (post_attention_layernorm): Gemma2RMSNorm((2304,), eps=1e-06)\n",
       "          )\n",
       "        )\n",
       "        (norm): Gemma2RMSNorm((2304,), eps=1e-06)\n",
       "      )\n",
       "      (lm_head): lora.Linear(\n",
       "        (base_layer): Linear(in_features=2304, out_features=256006, bias=False)\n",
       "        (lora_dropout): ModuleDict(\n",
       "          (default): Dropout(p=0.05, inplace=False)\n",
       "        )\n",
       "        (lora_A): ModuleDict(\n",
       "          (default): Linear(in_features=2304, out_features=16, bias=False)\n",
       "        )\n",
       "        (lora_B): ModuleDict(\n",
       "          (default): Linear(in_features=16, out_features=256006, bias=False)\n",
       "        )\n",
       "        (lora_embedding_A): ParameterDict()\n",
       "        (lora_embedding_B): ParameterDict()\n",
       "        (lora_magnitude_vector): ModuleDict()\n",
       "      )\n",
       "    )\n",
       "  )\n",
       ")"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from peft import PeftModel, PeftConfig\n",
    "from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig\n",
    "from datasets import load_dataset\n",
    "import torch\n",
    "\n",
    "bnb_config = BitsAndBytesConfig(\n",
    "            load_in_4bit=True,\n",
    "            bnb_4bit_quant_type=\"nf4\",\n",
    "            bnb_4bit_compute_dtype=torch.bfloat16,\n",
    "            bnb_4bit_use_double_quant=True,\n",
    "        )\n",
    "\n",
    "peft_model_id = f\"{username}/{output_dir}\" # replace with your newly trained adapter\n",
    "device = \"auto\"\n",
    "config = PeftConfig.from_pretrained(peft_model_id)\n",
    "model = AutoModelForCausalLM.from_pretrained(config.base_model_name_or_path,\n",
    "                                             device_map=device,\n",
    "                                             )\n",
    "tokenizer = AutoTokenizer.from_pretrained(peft_model_id)\n",
    "model.resize_token_embeddings(len(tokenizer))\n",
    "model = PeftModel.from_pretrained(model, peft_model_id)\n",
    "model.to(torch.bfloat16)\n",
    "model.eval()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "69e83af9-f967-4e5a-842b-0daed13f7957",
   "metadata": {
    "id": "69e83af9-f967-4e5a-842b-0daed13f7957",
    "outputId": "979b2ee9-fe5b-49b1-aed5-e28f0239a709",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<bos><start_of_turn>human\n",
      "You are a function calling AI model. You are provided with function signatures within <tools></tools> XML tags.You may call one or more functions to assist with the user query. Don't make assumptions about what values to plug into functions.Here are the available tools:<tools> [{'type': 'function', 'function': {'name': 'convert_currency', 'description': 'Convert from one currency to another', 'parameters': {'type': 'object', 'properties': {'amount': {'type': 'number', 'description': 'The amount to convert'}, 'from_currency': {'type': 'string', 'description': 'The currency to convert from'}, 'to_currency': {'type': 'string', 'description': 'The currency to convert to'}}, 'required': ['amount', 'from_currency', 'to_currency']}}}, {'type': 'function', 'function': {'name': 'calculate_distance', 'description': 'Calculate the distance between two locations', 'parameters': {'type': 'object', 'properties': {'start_location': {'type': 'string', 'description': 'The starting location'}, 'end_location': {'type': 'string', 'description': 'The ending location'}}, 'required': ['start_location', 'end_location']}}}] </tools>Use the following pydantic model json schema for each tool call you will make: {'title': 'FunctionCall', 'type': 'object', 'properties': {'arguments': {'title': 'Arguments', 'type': 'object'}, 'name': {'title': 'Name', 'type': 'string'}}, 'required': ['arguments', 'name']}For each function call return a json object with function name and arguments within <tool_call></tool_call> XML tags as follows:\n",
      "<tool_call>\n",
      "{tool_call}\n",
      "</tool_call>Also, before making a call to a function take the time to plan the function to take. Make that thinking process between <think>{your thoughts}</think>\n",
      "\n",
      "Hi, I need to convert 500 USD to Euros. Can you help me with that?<end_of_turn><eos>\n",
      "<start_of_turn>model\n",
      "<think>Alright, so I'm looking at this scenario where the user needs help converting 500 USD to Euros. Let's break it down step by step to understand how to approach this.\n",
      "\n",
      "First, the user has asked for conversion from USD to EUR. I know there's a specific tool available for currency conversion called `convert_currency`. This tool requires three parameters: amount, from_currency, and to_currency. In the user's message, they've provided all necessary details—500 USD and the desire to convert to Euros. \n",
      "\n",
      "I don't need to make any assumptions here because the user has already provided the exact amount and the target currency. My next step is to prepare the function call with the correct parameters. So, I'll structure the request with the amount as 500, from_currency as USD, and to_currency as EUR. \n",
      "\n",
      "This will be sent in JSON format as specified, ensuring that the tool receives all the necessary information to perform the conversion accurately. Once the function is called, it should return the converted amount, which I can then present back to the user.\n",
      "</think><tool_call>\n",
      "{'name': 'convert_currency', 'arguments': {'amount': 500, 'from_currency': 'USD', 'to_currency': 'EUR'}}\n",
      "</tool_call><end_of_turn><eos>\n",
      "<start_of_turn>tool\n",
      "<tool_response>\n",
      "{'converted_amount': 420.5, 'from_currency': 'USD', 'to_currency': 'EUR'}\n",
      "</tool_response><end_of_turn><eos>\n",
      "<start_of_turn>model\n",
      "Sure, 500 USD is approximately 420.50 Euros.<end_of_turn><eos>\n",
      "<start_of_turn>human\n",
      "That's great! Now, can you convert 1000 Euros back to USD?<end_of_turn><eos>\n",
      "<start_of_turn>model\n",
      "<think>Okay, so I'm trying to figure out how to handle the next function call based on the conversation and the previous steps. Let me see...\n",
      "\n",
      "In the conversation, the user first asked to convert 500 USD to Euros, and the model used the `convert_currency` function with the parameters amount=500, from_currency=USD, to_currency=EUR. Then, after getting the response, the model provided the converted amount. Now, the user is asking to convert 1000 Euros back to USD. \n",
      "\n",
      "So, the user's request is clear: convert 1000 EUR to USD. Since the function `convert_currency` is designed to handle currency conversions, and it requires amount, from_currency, and to_currency as parameters, it makes sense to call this function again with the updated values. \n",
      "\n",
      "The amount is 1000, from_currency should be EUR because the user is converting from Euros, and to_currency should be USD because they want the result in US Dollars. I don't need to make any assumptions here because all the required information is provided by the user. \n",
      "\n",
      "I'll structure the function call with these parameters. This should give the user the accurate converted amount from Euros to USD. Once the tool processes this, the model can present the result back to the user.\n",
      "</think><tool_call>\n",
      "{'name': 'convert_currency', 'arguments': {'amount': 1000, 'from_currency': 'EUR', 'to_currency': 'USD'}}\n",
      "</tool_call><end_of_turn><eos>\n",
      "<start_of_turn>tool\n",
      "<tool_response>\n",
      "{'converted_amount': 1190.0, 'from_currency': 'EUR', 'to_currency': 'USD'}\n",
      "</tool_response><end_of_turn><eos>\n",
      "<start_of_turn>model\n",
      "1000 Euros is approximately 1190.00 USD.<end_of_turn><eos>\n",
      "<start_of_turn>human\n",
      "Thank you for your help!<end_of_turn><eos>\n",
      "<start_of_turn>model\n",
      "You're welcome! If you have any other questions, feel free to ask.<end_of_turn><eos>\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(dataset[\"test\"][8][\"text\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b47fd511-ea00-47ce-8618-6e78e25672b2",
   "metadata": {
    "id": "b47fd511-ea00-47ce-8618-6e78e25672b2"
   },
   "source": [
    "### Testing the model 🚀\n",
    "\n",
    "In that case, we will take the start of one of the samples from the test set and hope that it will generate the expected output.\n",
    "\n",
    "Since we want to test the function-calling capacities of our newly fine-tuned model, the input will be a user message with the available tools, a\n",
    "\n",
    "\n",
    "### Disclaimer ⚠️\n",
    "\n",
    "The dataset we’re using **does not contain sufficient training data** and is purely for **educational purposes**. As a result, **your trained model’s outputs may differ** from the examples shown in this course. **Don’t be discouraged** if your results vary—our primary goal here is to illustrate the core concepts rather than produce a fully optimized or production-ready model.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "37bf938d-08fa-4577-9966-0238339afcdb",
   "metadata": {
    "id": "37bf938d-08fa-4577-9966-0238339afcdb",
    "outputId": "e97e7a1e-5ab2-46a2-dc3a-f436964fe004",
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<bos><start_of_turn>human\n",
      "You are a function calling AI model. You are provided with function signatures within <tools></tools> XML tags.You may call one or more functions to assist with the user query. Don't make assumptions about what values to plug into functions.Here are the available tools:<tools> [{'type': 'function', 'function': {'name': 'convert_currency', 'description': 'Convert from one currency to another', 'parameters': {'type': 'object', 'properties': {'amount': {'type': 'number', 'description': 'The amount to convert'}, 'from_currency': {'type': 'string', 'description': 'The currency to convert from'}, 'to_currency': {'type': 'string', 'description': 'The currency to convert to'}}, 'required': ['amount', 'from_currency', 'to_currency']}}}, {'type': 'function', 'function': {'name': 'calculate_distance', 'description': 'Calculate the distance between two locations', 'parameters': {'type': 'object', 'properties': {'start_location': {'type': 'string', 'description': 'The starting location'}, 'end_location': {'type': 'string', 'description': 'The ending location'}}, 'required': ['start_location', 'end_location']}}}] </tools>Use the following pydantic model json schema for each tool call you will make: {'title': 'FunctionCall', 'type': 'object', 'properties': {'arguments': {'title': 'Arguments', 'type': 'object'}, 'name': {'title': 'Name', 'type': 'string'}}, 'required': ['arguments', 'name']}For each function call return a json object with function name and arguments within <tool_call></tool_call> XML tags as follows:\n",
      "<tool_call>\n",
      "{tool_call}\n",
      "</tool_call>Also, before making a call to a function take the time to plan the function to take. Make that thinking process between <think>{your thoughts}</think>\n",
      "\n",
      "Hi, I need to convert 500 USD to Euros. Can you help me with that?<end_of_turn><eos>\n",
      "<start_of_turn>model\n",
      "<think>Okay, so the user is asking to convert 500 USD to Euros. I need to figure out how to respond using the available functions. Let me look at the tools provided. There's a function called convert_currency which does exactly that—it converts one currency to another. The parameters required are amount, from_currency, and to_currency. \n",
      "\n",
      "The user provided the amount as 500, the source currency as USD, and the target currency as EUR. That fits perfectly with the function's parameters. I don't need to make any assumptions here because the user has given all the necessary details. \n",
      "\n",
      "So, I should call the convert_currency function with these arguments. That should give the user the converted amount they need.\n",
      "</think><tool_call>\n",
      "{'name': 'convert_currency', 'arguments': {'amount': 500, 'from_currency': 'USD', 'to_currency': 'EUR'}}\n",
      "</tool_call><end_of_turn><eos>\n"
     ]
    }
   ],
   "source": [
    "#this prompt is a sub-sample of one of the test set examples. In this example we start the generation after the model generation starts.\n",
    "prompt=\"\"\"<bos><start_of_turn>human\n",
    "You are a function calling AI model. You are provided with function signatures within <tools></tools> XML tags.You may call one or more functions to assist with the user query. Don't make assumptions about what values to plug into functions.Here are the available tools:<tools> [{'type': 'function', 'function': {'name': 'convert_currency', 'description': 'Convert from one currency to another', 'parameters': {'type': 'object', 'properties': {'amount': {'type': 'number', 'description': 'The amount to convert'}, 'from_currency': {'type': 'string', 'description': 'The currency to convert from'}, 'to_currency': {'type': 'string', 'description': 'The currency to convert to'}}, 'required': ['amount', 'from_currency', 'to_currency']}}}, {'type': 'function', 'function': {'name': 'calculate_distance', 'description': 'Calculate the distance between two locations', 'parameters': {'type': 'object', 'properties': {'start_location': {'type': 'string', 'description': 'The starting location'}, 'end_location': {'type': 'string', 'description': 'The ending location'}}, 'required': ['start_location', 'end_location']}}}] </tools>Use the following pydantic model json schema for each tool call you will make: {'title': 'FunctionCall', 'type': 'object', 'properties': {'arguments': {'title': 'Arguments', 'type': 'object'}, 'name': {'title': 'Name', 'type': 'string'}}, 'required': ['arguments', 'name']}For each function call return a json object with function name and arguments within <tool_call></tool_call> XML tags as follows:\n",
    "<tool_call>\n",
    "{tool_call}\n",
    "</tool_call>Also, before making a call to a function take the time to plan the function to take. Make that thinking process between <think>{your thoughts}</think>\n",
    "\n",
    "Hi, I need to convert 500 USD to Euros. Can you help me with that?<end_of_turn><eos>\n",
    "<start_of_turn>model\n",
    "<think>\"\"\"\n",
    "\n",
    "inputs = tokenizer(prompt, return_tensors=\"pt\", add_special_tokens=False)\n",
    "inputs = {k: v.to(\"cuda\") for k,v in inputs.items()}\n",
    "outputs = model.generate(**inputs,\n",
    "                         max_new_tokens=300,# Adapt as necessary\n",
    "                         do_sample=True,\n",
    "                         top_p=0.95,\n",
    "                         temperature=0.01,\n",
    "                         repetition_penalty=1.0,\n",
    "                         eos_token_id=tokenizer.eos_token_id)\n",
    "print(tokenizer.decode(outputs[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "xWewPCZOyfJQ",
   "metadata": {
    "id": "xWewPCZOyfJQ"
   },
   "source": [
    "## Congratulations\n",
    "Congratulations on finishing this first Bonus Unit 🥳\n",
    "\n",
    "You've just **mastered what Function-Calling is and how to fine-tune your model to do Function-Calling**!\n",
    "\n",
    "If it's the first time you do this, it's normal that you're feeling puzzled. Take time to check the documentation and understand each part of the code and why we did it this way.\n",
    "\n",
    "Also, don't hesitate to try to **fine-tune different models**. The **best way to learn is by trying.**\n",
    "\n",
    "### Keep Learning, Stay Awesome 🤗"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "gpuType": "T4",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.5"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "001b32600fdd418bb30c6b5ff85e269c": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "03b839beea6a4f6b8d9005491146615d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "03b8a027c56f4b52a3e54f57bb5bb526": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0a0bc95f445948f486fbce865a4642f2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "0c881b9322624d70bb2c5147a5422633": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "0db5d5c516d7479ebf7603093f9e1e99": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0fe7751f55134695afb44bf8673dd4d9": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "0ff4d8414af94af59628b0417a40cc7d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1296a8a4baea4faabc62a42b2ca1a53a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "13b49790234c4d1aa8fe9b532fcf95bf": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "197619158cd247649eac3722284c8d19": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1981ff353e494735be4706a76053c24a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1af481e6544a4fa386815218461cd5c3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "1c810f5364544762a3093e1a0a9c1e5b": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1d104fe8612f45e4810bbed7e3540330": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1e271fcb92a14680b84c33cb7c28f9c0": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1e9d8fccba9a468291ffe271b3497830": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1ead218e71614374909b92fda097fd42": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "21cf377576154ffeb028f4ae3b3503d5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_2b8c9e43da4e4e008a2a2b06e6f3106e",
      "max": 24223,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_62f3a32c4f484dd6899ba5d6ffbaca11",
      "value": 24223
     }
    },
    "26484831a87a4b489e1288ea71ea7767": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "26bd63db6a384c12b7d42bf95ca1d1bc": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "2828b2455ec44143a5739c8e8207fb9b": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "298f092855f14af79ec2eda792732810": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a385bba49b514ab386cb5f4cbb01821f",
      "placeholder": "​",
      "style": "IPY_MODEL_78a34eb9cd534c3d84c8f22d3c53c88f",
      "value": "Tokenizing eval dataset: 100%"
     }
    },
    "2b8c9e43da4e4e008a2a2b06e6f3106e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2bcef869f1814a04bbbed73df7a9ab24": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2ccddb85840c4981ae089ae4c74a2de6": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_89eee480405a416ba0edf097423724b9",
       "IPY_MODEL_56e8079c374a4f3f9af5ef96a73f2955",
       "IPY_MODEL_3aeb28bc164444d7afb7bf7435a001f2"
      ],
      "layout": "IPY_MODEL_1ead218e71614374909b92fda097fd42"
     }
    },
    "2e6392b95f8a48568a89780adf76387f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_cee71fbbf8a04b3bb64b96e7fa2b0b0e",
      "max": 10,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_0a0bc95f445948f486fbce865a4642f2",
      "value": 10
     }
    },
    "2eb5230a43e84790b0392db58157b0f1": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2f73c87217c84bd293a5ea5b4aba0ae5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_6883c5e46d3948649488d3e9f81ddb9c",
       "IPY_MODEL_d0498872e00f4644847e073c6000a111",
       "IPY_MODEL_482f477cc2354d9da6a8c2932bb73e98"
      ],
      "layout": "IPY_MODEL_1c810f5364544762a3093e1a0a9c1e5b"
     }
    },
    "328fa6d902bd46bbb0ecdc7404a13e8c": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "33ec043a635f4e99b67cd6f7e6fc6193": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "36bbd4bd563a4053a7af8532300253b7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_33ec043a635f4e99b67cd6f7e6fc6193",
      "placeholder": "​",
      "style": "IPY_MODEL_39fee7a249c146d1a166e58755c1cda8",
      "value": " 100/100 [00:00&lt;00:00, 277.78 examples/s]"
     }
    },
    "39c0963803c74ebab07cec20e10d0184": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_46b69fb951af44f99232f459daa4f103",
       "IPY_MODEL_3e23b4c5848843fcb44dc0ae2f157d66",
       "IPY_MODEL_c9f86634a6bf4e49a902e3d42e67f1bd"
      ],
      "layout": "IPY_MODEL_03b8a027c56f4b52a3e54f57bb5bb526"
     }
    },
    "39fee7a249c146d1a166e58755c1cda8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "39fff32b9756437581228465165a3115": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "3aeb28bc164444d7afb7bf7435a001f2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_9f8d631358e240f982e31171d1bd9f26",
      "placeholder": "​",
      "style": "IPY_MODEL_a0c7029819414c5dacefed93194cd763",
      "value": " 100/100 [00:00&lt;00:00, 648.06 examples/s]"
     }
    },
    "3b07ec7f4d024b61abe94c8adeebed55": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "3bb2805fcf0441089c0f4789e28b072d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_eedc1213dd604bbe827dcec617fdae34",
      "placeholder": "​",
      "style": "IPY_MODEL_f73f91bb567a48f890ba0827b7ed321f",
      "value": "Downloading shards: 100%"
     }
    },
    "3e23b4c5848843fcb44dc0ae2f157d66": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b5121cada3514b67a9c533e7468b3058",
      "max": 10,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_bf70daf78057419b8a78af75a093a3dd",
      "value": 10
     }
    },
    "425fe267a38748ea9670a0996b802bdc": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4349fab6f16343cb8955367f6fbc4a43": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_0ff4d8414af94af59628b0417a40cc7d",
      "placeholder": "​",
      "style": "IPY_MODEL_534a5dc9cd7e475db2be613849959fc7",
      "value": "model-00001-of-00002.safetensors: 100%"
     }
    },
    "45e867ccde7d4b5bb4ed4861d723a758": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_2bcef869f1814a04bbbed73df7a9ab24",
      "placeholder": "​",
      "style": "IPY_MODEL_e3bd52cd621e4d2196e94327324722c4",
      "value": "generation_config.json: 100%"
     }
    },
    "46b69fb951af44f99232f459daa4f103": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_58a18918bae34aca8ec73ae89fd5bc24",
      "placeholder": "​",
      "style": "IPY_MODEL_fd6e1776cbcd4f7b96ec6d9754eb2c83",
      "value": "Packing eval dataset: 100%"
     }
    },
    "472cbf91b24f46829960d68e2316c417": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f2157c83879046a29b72613bce9de56e",
      "placeholder": "​",
      "style": "IPY_MODEL_3b07ec7f4d024b61abe94c8adeebed55",
      "value": "Applying chat template to train dataset: 100%"
     }
    },
    "482f477cc2354d9da6a8c2932bb73e98": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_51b42c72bc2b428fbc0389bfd4d9e598",
      "placeholder": "​",
      "style": "IPY_MODEL_611a6d951d2a42b9b650496612cfd484",
      "value": " 2/2 [00:25&lt;00:00, 10.98s/it]"
     }
    },
    "4cfce958cf5445129ab6281784664038": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "4d006dc72b2b45b58caf4d398c3756b8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_328fa6d902bd46bbb0ecdc7404a13e8c",
      "max": 100,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_f375fed157034dfcbd28744027d77eba",
      "value": 100
     }
    },
    "4e21f1bd903443a89dea32bb3f3c26a9": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "4f75329d3e8d4cc38a405c1c4cc51d70": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "51b42c72bc2b428fbc0389bfd4d9e598": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "534a5dc9cd7e475db2be613849959fc7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "537e188c000041fea6adf26f2255d738": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "538f6a3c632f431fa9d16ab17383a602": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "54fc2d9962de4ffda0500a01e112b58a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_8e70cccb0fb04a43b3c6ab234bcae9a8",
       "IPY_MODEL_f2d87447fe8448c7baff656c540620e6",
       "IPY_MODEL_5f438fbbfb60436faf79662e81092154"
      ],
      "layout": "IPY_MODEL_197619158cd247649eac3722284c8d19"
     }
    },
    "55e72b7f262b4f57a6abb1c9f01c8de2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b782092e2282488ba86f85eebe697603",
      "placeholder": "​",
      "style": "IPY_MODEL_f7ba9e0f4e64484a82374bb5f1d12b15",
      "value": " 10/10 [00:00&lt;00:00, 154.34 examples/s]"
     }
    },
    "56e8079c374a4f3f9af5ef96a73f2955": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e633387b1640461e82617c1702ee82f5",
      "max": 100,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_26484831a87a4b489e1288ea71ea7767",
      "value": 100
     }
    },
    "57d137229091486dbf0e4b7dd6dce98a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b4c43d908bd64d9bbdb488dac46d2e45",
      "placeholder": "​",
      "style": "IPY_MODEL_96691a6287ef401582a2a1744a4940c4",
      "value": "Tokenizing train dataset: 100%"
     }
    },
    "58a18918bae34aca8ec73ae89fd5bc24": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "5c8fa13e559942849a61d9480be730bd": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_1e271fcb92a14680b84c33cb7c28f9c0",
      "placeholder": "​",
      "style": "IPY_MODEL_da543c28b9e44723b019220de8427b5e",
      "value": " 24.2k/24.2k [00:00&lt;00:00, 1.86MB/s]"
     }
    },
    "5ec98ede3fbf4c64843b935ce3ad28a7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ec8ecb8bdee4435fbc97d73863ebd9f8",
      "placeholder": "​",
      "style": "IPY_MODEL_1af481e6544a4fa386815218461cd5c3",
      "value": " 241M/241M [00:05&lt;00:00, 41.5MB/s]"
     }
    },
    "5f438fbbfb60436faf79662e81092154": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_0db5d5c516d7479ebf7603093f9e1e99",
      "placeholder": "​",
      "style": "IPY_MODEL_654fd18a9e59415e9dd314dc2f94a654",
      "value": " 838/838 [00:00&lt;00:00, 38.3kB/s]"
     }
    },
    "611a6d951d2a42b9b650496612cfd484": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "6296f0c3ac5647e9b9cbc7c8212a2b35": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b821e13722e24feb941fa31a604a85e9",
      "max": 4988025760,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_91c21b2b127640e195f2e7c7256b5a0e",
      "value": 4988025760
     }
    },
    "62f3a32c4f484dd6899ba5d6ffbaca11": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "63c269b37eed4d348f9ce24eef15fc15": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_73c9d510c8754f2ea21adf318e35bc8e",
      "placeholder": "​",
      "style": "IPY_MODEL_0fe7751f55134695afb44bf8673dd4d9",
      "value": "Applying chat template to eval dataset: 100%"
     }
    },
    "654fd18a9e59415e9dd314dc2f94a654": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "681019ca1d084d9692f5085d1df520ae": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "6883c5e46d3948649488d3e9f81ddb9c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_681019ca1d084d9692f5085d1df520ae",
      "placeholder": "​",
      "style": "IPY_MODEL_b1bb9bcec30144a380fa4fe440c41d7c",
      "value": "Loading checkpoint shards: 100%"
     }
    },
    "68baf2700f614be288bf35e54d207096": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_fc9a32511862493e980682b6ff5044bb",
      "placeholder": "​",
      "style": "IPY_MODEL_538f6a3c632f431fa9d16ab17383a602",
      "value": " 187/187 [00:00&lt;00:00, 12.9kB/s]"
     }
    },
    "6e1393f9f39a4b3cb94753c883b1bbf7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_fe30dac330db4fde94af1c54c84a1be8",
      "placeholder": "​",
      "style": "IPY_MODEL_4cfce958cf5445129ab6281784664038",
      "value": " 2/2 [02:06&lt;00:00, 53.19s/it]"
     }
    },
    "6f4e106bb5b44f398a1fe4ac588f6cca": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "71ba28670e8d4583b66b1c1587cdbce1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_4349fab6f16343cb8955367f6fbc4a43",
       "IPY_MODEL_6296f0c3ac5647e9b9cbc7c8212a2b35",
       "IPY_MODEL_c6d97ccbecb0407f89ef4469f3874e74"
      ],
      "layout": "IPY_MODEL_ccabb079b6fd466fb877a018aebf793c"
     }
    },
    "73c9d510c8754f2ea21adf318e35bc8e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "78a34eb9cd534c3d84c8f22d3c53c88f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "822bea0ac84d4ff29d984bf5f5d2c3a2": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "89033e9c0dd249db9dc9a3b1e215dded": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "89eee480405a416ba0edf097423724b9": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_822bea0ac84d4ff29d984bf5f5d2c3a2",
      "placeholder": "​",
      "style": "IPY_MODEL_ecef440c871b4daba34661a1ddba6b0c",
      "value": "Packing train dataset: 100%"
     }
    },
    "8bbe22288edf4a06b2c56952fd81d5e7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_298f092855f14af79ec2eda792732810",
       "IPY_MODEL_2e6392b95f8a48568a89780adf76387f",
       "IPY_MODEL_55e72b7f262b4f57a6abb1c9f01c8de2"
      ],
      "layout": "IPY_MODEL_9fbca7fa0d6b4fff910b806a97fa7718"
     }
    },
    "8e70cccb0fb04a43b3c6ab234bcae9a8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_1d104fe8612f45e4810bbed7e3540330",
      "placeholder": "​",
      "style": "IPY_MODEL_d62425b0d0ea497ead2cd981eaa61c7e",
      "value": "config.json: 100%"
     }
    },
    "91163fcffc60438cb39b0eb586dac418": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_39fff32b9756437581228465165a3115",
      "placeholder": "​",
      "style": "IPY_MODEL_4f75329d3e8d4cc38a405c1c4cc51d70",
      "value": " 10/10 [00:00&lt;00:00, 298.91 examples/s]"
     }
    },
    "91c21b2b127640e195f2e7c7256b5a0e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "952893c941c346c2aedcd9358859a3b9": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "95eebbfed6fc41d6a8c668abe1608756": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "96691a6287ef401582a2a1744a4940c4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "9a8b33ef0f6c446496cc7dae07da321c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "9ac674eeba0542e7b4e28a13e90ce652": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_2eb5230a43e84790b0392db58157b0f1",
      "placeholder": "​",
      "style": "IPY_MODEL_26bd63db6a384c12b7d42bf95ca1d1bc",
      "value": "model-00002-of-00002.safetensors: 100%"
     }
    },
    "9f8d631358e240f982e31171d1bd9f26": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9fbca7fa0d6b4fff910b806a97fa7718": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a0c7029819414c5dacefed93194cd763": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "a385bba49b514ab386cb5f4cbb01821f": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a76145e51f7544fdabd23198baf318c4": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a89230859593424e960047a96977c6b8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f7dd34e15348462297564f0e6e0b568d",
      "max": 10,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_537e188c000041fea6adf26f2255d738",
      "value": 10
     }
    },
    "a893802af41d4400b35b9cb71add1387": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_45e867ccde7d4b5bb4ed4861d723a758",
       "IPY_MODEL_f5e196c908e7491982773d3231fff3e6",
       "IPY_MODEL_68baf2700f614be288bf35e54d207096"
      ],
      "layout": "IPY_MODEL_001b32600fdd418bb30c6b5ff85e269c"
     }
    },
    "a8edc3a5b0d9422b8e2e63d5564cfdc9": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b1542891fc6243d98d51981dd0584bdf": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d36e7dd4b9dc497faa6e8f63843a738f",
      "placeholder": "​",
      "style": "IPY_MODEL_f469ec8c79ac476c82a5e228f347bffa",
      "value": " 100/100 [00:00&lt;00:00, 690.13 examples/s]"
     }
    },
    "b1bb9bcec30144a380fa4fe440c41d7c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "b2e16ad7540d4760b28f3a8c419905f8": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b4c43d908bd64d9bbdb488dac46d2e45": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b5121cada3514b67a9c533e7468b3058": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b782092e2282488ba86f85eebe697603": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b821e13722e24feb941fa31a604a85e9": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "bc6e638133f545ff8d54f767de0cb35a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a8edc3a5b0d9422b8e2e63d5564cfdc9",
      "max": 2,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_e08c970bbf4540a89533a43f719f68e4",
      "value": 2
     }
    },
    "bd746ea2e46a491e954ac6f32fb0e45b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e6f810bd430245b190ee932554cca05c",
      "max": 100,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_952893c941c346c2aedcd9358859a3b9",
      "value": 100
     }
    },
    "bdbd9e7e1d674edeab47e3595f6af4f8": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "bf70daf78057419b8a78af75a093a3dd": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "c6d97ccbecb0407f89ef4469f3874e74": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_1296a8a4baea4faabc62a42b2ca1a53a",
      "placeholder": "​",
      "style": "IPY_MODEL_13b49790234c4d1aa8fe9b532fcf95bf",
      "value": " 4.99G/4.99G [01:59&lt;00:00, 42.4MB/s]"
     }
    },
    "c9f86634a6bf4e49a902e3d42e67f1bd": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e0b3b3c072be44de8e0a2dae91598aa6",
      "placeholder": "​",
      "style": "IPY_MODEL_4e21f1bd903443a89dea32bb3f3c26a9",
      "value": " 10/10 [00:00&lt;00:00, 312.22 examples/s]"
     }
    },
    "cacc3c3a10c64b338866a8e42201e44c": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "cc4993f1af6b4f2685ecbced238b8913": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_6f4e106bb5b44f398a1fe4ac588f6cca",
      "max": 240691728,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_0c881b9322624d70bb2c5147a5422633",
      "value": 240691728
     }
    },
    "ccabb079b6fd466fb877a018aebf793c": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ce3c69ec86224ffea8c46deb20831bb7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_3bb2805fcf0441089c0f4789e28b072d",
       "IPY_MODEL_bc6e638133f545ff8d54f767de0cb35a",
       "IPY_MODEL_6e1393f9f39a4b3cb94753c883b1bbf7"
      ],
      "layout": "IPY_MODEL_a76145e51f7544fdabd23198baf318c4"
     }
    },
    "cee71fbbf8a04b3bb64b96e7fa2b0b0e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d0498872e00f4644847e073c6000a111": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_425fe267a38748ea9670a0996b802bdc",
      "max": 2,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_95eebbfed6fc41d6a8c668abe1608756",
      "value": 2
     }
    },
    "d1d376228c334c5999143b905e234ffc": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "d36e7dd4b9dc497faa6e8f63843a738f": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d62425b0d0ea497ead2cd981eaa61c7e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "da543c28b9e44723b019220de8427b5e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "deba87fc28db4f78b58fc1e456fea335": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_f06414fa35b64ff588511796c9b51d1f",
       "IPY_MODEL_21cf377576154ffeb028f4ae3b3503d5",
       "IPY_MODEL_5c8fa13e559942849a61d9480be730bd"
      ],
      "layout": "IPY_MODEL_fae24e4a14814c7ebef30b422c2d3652"
     }
    },
    "e08c970bbf4540a89533a43f719f68e4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "e0b3b3c072be44de8e0a2dae91598aa6": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e1ef7211c5554b2f8fdd39f2f846d2d9": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_9ac674eeba0542e7b4e28a13e90ce652",
       "IPY_MODEL_cc4993f1af6b4f2685ecbced238b8913",
       "IPY_MODEL_5ec98ede3fbf4c64843b935ce3ad28a7"
      ],
      "layout": "IPY_MODEL_bdbd9e7e1d674edeab47e3595f6af4f8"
     }
    },
    "e3bd52cd621e4d2196e94327324722c4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "e3fe61834c3e49a3895212a336776f9d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_472cbf91b24f46829960d68e2316c417",
       "IPY_MODEL_bd746ea2e46a491e954ac6f32fb0e45b",
       "IPY_MODEL_b1542891fc6243d98d51981dd0584bdf"
      ],
      "layout": "IPY_MODEL_b2e16ad7540d4760b28f3a8c419905f8"
     }
    },
    "e633387b1640461e82617c1702ee82f5": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e6f810bd430245b190ee932554cca05c": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e9de72aadc5743a2b56537b3ad035461": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_57d137229091486dbf0e4b7dd6dce98a",
       "IPY_MODEL_4d006dc72b2b45b58caf4d398c3756b8",
       "IPY_MODEL_36bbd4bd563a4053a7af8532300253b7"
      ],
      "layout": "IPY_MODEL_cacc3c3a10c64b338866a8e42201e44c"
     }
    },
    "ec8ecb8bdee4435fbc97d73863ebd9f8": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ecef440c871b4daba34661a1ddba6b0c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "eedc1213dd604bbe827dcec617fdae34": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f06414fa35b64ff588511796c9b51d1f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_1981ff353e494735be4706a76053c24a",
      "placeholder": "​",
      "style": "IPY_MODEL_9a8b33ef0f6c446496cc7dae07da321c",
      "value": "model.safetensors.index.json: 100%"
     }
    },
    "f2157c83879046a29b72613bce9de56e": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f2d87447fe8448c7baff656c540620e6": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_2828b2455ec44143a5739c8e8207fb9b",
      "max": 838,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_03b839beea6a4f6b8d9005491146615d",
      "value": 838
     }
    },
    "f375fed157034dfcbd28744027d77eba": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "f469ec8c79ac476c82a5e228f347bffa": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "f4a01e54ec53475585eaa88b3a272b4b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_63c269b37eed4d348f9ce24eef15fc15",
       "IPY_MODEL_a89230859593424e960047a96977c6b8",
       "IPY_MODEL_91163fcffc60438cb39b0eb586dac418"
      ],
      "layout": "IPY_MODEL_89033e9c0dd249db9dc9a3b1e215dded"
     }
    },
    "f5e196c908e7491982773d3231fff3e6": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_1e9d8fccba9a468291ffe271b3497830",
      "max": 187,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_d1d376228c334c5999143b905e234ffc",
      "value": 187
     }
    },
    "f73f91bb567a48f890ba0827b7ed321f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "f7ba9e0f4e64484a82374bb5f1d12b15": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "f7dd34e15348462297564f0e6e0b568d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "fae24e4a14814c7ebef30b422c2d3652": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "fc9a32511862493e980682b6ff5044bb": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "fd6e1776cbcd4f7b96ec6d9754eb2c83": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "fe30dac330db4fde94af1c54c84a1be8": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}