Take it easy!

Chapter 0: Introduction — How to Use This Book

0.1 Characters and Roles

Reimu
Hey Marisa, what is this book? The cover says Hugging Face — that's the logo with the hugging face, right?

Marisa
Exactly, ZE. Hugging Face (nicknamed HF) is a platform for sharing machine learning models and datasets. If GitHub is where you put code, think of HF as "where pretrained AI lives."

Reimu
I know GitHub. The place where people yell at you for git push.

Marisa
This book is in yukkuri Reimu and yukkuri Marisa dialogue format. Roles are set like this.

Role	Who	What they do
Fellow learner	Reimu	Plain questions, stumbles, reactions to results
Guide	Marisa	Concept explanations, commands, "that's why you write it like this, ZE"
Reader (you)	—	Run the code in your own environment

Reimu
…Oh, you mean me.
So I voice the "I don't get it" parts and Marisa answers. Readers just sit quietly and type?

Marisa
If you only sit quietly, nothing sticks. Every chapter follows read → run → tweak a little → reflect — that's the rule, ZE. We'll do an environment check at the end of Chapter 0, so open a terminal once here.

Marisa
Also, each chapter's "Full Chapter Scripts" section has the same code as the repo's scripts/. Even if you're reading without GitHub or a clone, you can create files from there and run them.

0.2 The Big Picture of Hugging Face in This Book

Reimu
Is HF just a website? Or is there a Python library too?

Marisa
Both. Five things are enough to remember, ZE.

┌─────────────────────────────────────────────────────────┐
│  Hugging Face Hub (Web)                                  │
│  Where models, datasets, and demo apps (Spaces) live     │
└──────────────────────────┬──────────────────────────────┘
                           │  download / upload
┌──────────────────────────▼──────────────────────────────┐
│  Python libraries                                        │
│  ├─ transformers … load models, inference, training      │
│  ├─ datasets       … load and preprocess datasets        │
│  ├─ accelerate     … multi-GPU / efficiency (advanced)   │
│  ├─ peft           … lightweight FT such as LoRA (Ch. 7) │
│  └─ gradio         … Web UI demos (Ch. 9)                │
└─────────────────────────────────────────────────────────┘

Reimu
Looks like a wiring diagram…

Marisa
Chapter order follows this flow.

Ch.	Theme	What you touch
1	Peek at the Hub	Browser, `huggingface-cli`
2	Run something fast	`pipeline`
3–4	Inside	Tokenizer, Model
5	Data	`datasets`
6–7	Training	`Trainer`, LoRA
8–9	Ops & sharing	Inference optimization, Gradio, Spaces
10	Capstone	Yukkuri live-commentary bot (planned)

Reimu
The TOC said Chapter 2 runs in 3 lines — really?

Marisa
Really, ZE. But first set up the environment in Chapter 0. Some readers jump straight to Chapter 2; we structured things so you can come back here when errors hit.

0.3 Required Environment

Reimu
What do I need? Bullet list. Does it cost money?

Marisa
Mostly free. Here's the list.

Item	Required?	Notes
Python 3.10+	Required	3.11 recommended
Internet	Required	First run may download several GB of models
GPU (NVIDIA)	Optional	CPU / Colab works for learning
Hugging Face account	Recommended	Created in Ch. 1; download-only can wait
Google Colab	Optional	For people without a local GPU

0.3.1 Checking Python version

Marisa
First check Python in the terminal.

python3 --version

Example expected output:

Python 3.11.8

Reimu
What if it says 3.9.6?

Marisa
This book does not recommend below 3.10, ZE. Upgrade with pyenv or the official installer before continuing.

# Example when using pyenv
pyenv install 3.11.8
pyenv local 3.11.8
python --version

0.3.2 Virtual environment (venv)

Marisa
Global pip install fights with other projects. Always use venv.

# At repo root (sample for this book)
cd /path/to/yukkuri-hugging-face

python3 -m venv .venv

Activate:

# macOS / Linux
source .venv/bin/activate

# Windows (PowerShell)
# .venv\Scripts\Activate.ps1

You know it's active when (.venv) appears at the prompt, or check with:

which python
# e.g.: /path/to/yukkuri-hugging-face/.venv/bin/python

0.3.3 Bulk install of packages used in this book

Marisa
Chapter 0 uses the minimum. We add more as chapters progress, but installing everything upfront is fine.

pip install --upgrade pip

pip install \
  "transformers>=4.40" \
  "datasets>=2.18" \
  "accelerate>=0.28" \
  "huggingface_hub>=0.22" \
  "torch" \
  "sentencepiece" \
  "protobuf"

Reimu
Isn't torch huge?

Marisa
Huge, ZE. If CPU-only is enough, follow the official guide and pick a CPU wheel. With a GPU, install the CUDA build from PyTorch Get Started.

Example for CUDA 12.x (URL varies by environment):

pip install torch --index-url https://download.pytorch.org/whl/cu124

0.3.4 Environment check script

Marisa
Save the following as scripts/check_env.py and run it.
(The same content is in "Full Chapter Scripts" at the end of this chapter.)

# scripts/check_env.py
import sys

def main() -> None:
    print("Python:", sys.version.replace("\n", " "))

    import torch
    print("torch:", torch.__version__)
    print("CUDA available:", torch.cuda.is_available())
    if torch.cuda.is_available():
        print("CUDA device:", torch.cuda.get_device_name(0))

    import transformers
    import datasets
    import huggingface_hub

    print("transformers:", transformers.__version__)
    print("datasets:", datasets.__version__)
    print("huggingface_hub:", huggingface_hub.__version__)

    print("\nOK: Chapter 0 environment check complete")

if __name__ == "__main__":
    main()

Run:

python scripts/check_env.py

Example output (no GPU):

Python: 3.11.8 (main, ...) 
torch: 2.2.2
CUDA available: False
transformers: 4.41.2
datasets: 2.19.1
huggingface_hub: 0.23.2

OK: Chapter 0 environment check complete

Reimu
It says CUDA available: False. Am I doomed?

Marisa
You're not doomed. Small models in Chapter 2 run on CPU, ZE. Heavy training can wait for Colab or LoRA in later chapters.

0.3.5 Using Google Colab

Reimu
You said it's for people without a local GPU.

Marisa
On Colab you can do almost the same check in the first notebook cell.

# First Colab cell example
!pip install -q "transformers>=4.40" "datasets>=2.18" accelerate

import torch
print(torch.__version__, "CUDA:", torch.cuda.is_available())

Free tier sometimes has no GPU. "When Colab gives you a GPU, tackle Chapter 6 onward" is a fine pace for this book.

0.4 How to Run the Hands-ons

Reimu
You keep saying read, run, tweak, reflect every time — but concretely?

Marisa
The template is fixed, ZE.

Step 1: Read

Don't skip the dialogue — skim the code blocks first.
Note what you import and what you call.

Step 2: Run

Don't paste blindly — type by hand or paste line by line.
If you get an error, keep the full message.

# Optional: save run output to a log file
python scripts/check_env.py 2>&1 | tee log/ch00_env.txt

Step 3: Tweak

Try this chapter's tweak challenges. Chapter 0's are below.

Step 4: Reflect

Read Reimu's Notebook and Marisa's One More Thing at chapter end, then summarize in your own words in 3 lines.

Reimu
Tell me the tweak challenges already.

Marisa
Chapter 0's Hands-on 0.

Hands-on 0 — Environment check + one-line inference

Challenge A Add lines to check_env.py showing platform and free memory (if psutil is available).

pip install psutil

# Example addition to check_env.py
import platform
import psutil

print("Platform:", platform.platform())
print("RAM (GB):", round(psutil.virtual_memory().total / 1e9, 1))

Challenge B Run a minimal pipeline once from the Hub (first run takes time to download).

from transformers import pipeline

# Light English sentiment analysis (first-run download)
clf = pipeline("sentiment-analysis", model="distilbert-base-uncased-finetuned-sst-2-english")
print(clf("I love Hugging Face!"))

Example expected output:

[{'label': 'POSITIVE', 'score': 0.9998}]

Reimu
Challenge B is Chapter 2 stuff, isn't it?

Marisa
A preview, ZE. One line is enough to prove the environment works. Details come in Chapter 2.

Challenge C Check where the cache lives (prep for disk pressure).

import os
from huggingface_hub import constants

print(constants.HF_HOME)
# or
print(os.environ.get("HF_HOME", "(default ~/.cache/huggingface)"))

0.5 Mini Glossary

Reimu
A dictionary… do I have to memorize it?

Marisa
No memorization. Here we only cover words that show up when reading model pages on the Hub.

Term	In one line
Model	A trained neural net that maps input to output; includes weight files
Tokenizer	Converts strings to ID sequences for the model; used as a set with the model
Inference	Feed data to a trained model and get predictions; in this book via `pipeline` or `model.generate`
Fine-tuning (FT)	Further training an existing model on your data
Hub	Web service to publish and search models and datasets
Model Card	README describing the model, training data, limits, and license
Checkpoint	Saved weights mid-training or at completion
Token	A unit the tokenizer splits on; may be word fragments or symbols
Pipeline	High-level API bundling preprocessing → inference → postprocessing

Reimu
I'm still fuzzy on Model vs Checkpoint.

Marisa
Analogy: model = the game itself, checkpoint = save data, ZE. Same architecture (the game) with different weights (the save) swapped in.

In code, the relationship looks like this:

from transformers import AutoModel, AutoTokenizer

model_id = "distilbert-base-uncased-finetuned-sst-2-english"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModel.from_pretrained(model_id)

# tokenizer … characters ↔ IDs
# model … ID sequence → internal representation (Ch. 3+)

0.6 Repository Layout (Bundled with This Book)

Marisa
Scripts grow chapter by chapter. Rough directory layout:

yukkuri-hugging-face/
├── docs_en/       # Book Markdown (this file is docs_en/00.md)
├── scripts/        # Runnable Python
├── notebooks/      # For Colab (optional)
├── log/            # Run logs (keep out of git)
└── .venv/          # Virtual environment (not in git)

.gitignore example:

.venv/
__pycache__/
*.pyc
.cache/
log/
.env

Reimu
.env — that's where you put the HF token?

Marisa
You'll use it later, ZE. Chapter 1 creates account and token. Not required yet in Chapter 0. Challenge B running is enough.

Full Chapter Scripts

Marisa
So you can work without the repo, this chapter's finished scripts are included as-is.
Save them under scripts/ with the same filenames and run, ZE.
(Where it says python scripts/xxx.py, adjust the path to match where you saved.)

`scripts/check_env.py`

import sys


def main() -> None:
    print("Python:", sys.version.replace("\n", " "))

    import torch

    print("torch:", torch.__version__)
    print("CUDA available:", torch.cuda.is_available())
    if torch.cuda.is_available():
        print("CUDA device:", torch.cuda.get_device_name(0))

    import transformers
    import datasets
    import huggingface_hub

    print("transformers:", transformers.__version__)
    print("datasets:", datasets.__version__)
    print("huggingface_hub:", huggingface_hub.__version__)

    print("\nOK: Chapter 0 environment check complete")


if __name__ == "__main__":
    main()

Reimu's Notebook

HF is Hub (sharing) + Python libraries (execution) together.
Isolate with venv and check versions and CUDA with check_env.py.
Hands-ons are read → run → tweak → reflect. Challenge B's one-line pipeline proves it works.

Marisa's One More Thing

Environment variables control cache location and offline behavior. Handy early if disk space is tight, ZE.

# Example: move cache to another drive (bash)
export HF_HOME="/path/to/large-disk/huggingface"
export TRANSFORMERS_CACHE="$HF_HOME/hub"

# Offline only (after models are already downloaded)
import os
os.environ["HF_HUB_OFFLINE"] = "1"

On to the Next Chapter

Marisa
Once the environment is ready, Chapter 1 covers creating a Hub account, reading model cards, and huggingface-cli login.

Reimu
…I'll be careful not to leak the login token.

Marisa
That mindset matters, ZE. Next up: Chapter 1: Let's Peek at the Hugging Face Hub.

Chapter 1: Hugging Face Hub — Let's Take a Peek

1.1 What Is the Hub? Why Does Everyone Use It?

Reimu
In Chapter 0 you called it "something like GitHub," but what can the Hub actually do?

Marisa
Hugging Face Hub is a web service for publishing, searching, and downloading trained models, datasets, and demo apps (Spaces). The URL is https://huggingface.co.

Reimu
Why does everyone use it?

Marisa
Roughly four reasons.

Reason	Explanation
Easy to find	Filter by task (translation, classification, etc.) and language
Easy to reproduce	Model cards document training conditions and limitations
DL from CLI / Python	Fetch with `huggingface-cli` or `from_pretrained`, not just the browser
Sharing culture	A common place where paper implementations and community models gather

The big picture matches the diagram in Chapter 0. This chapter focuses on the top layer (the Hub) on the web.

┌──────────────────────────────────────────────────────────┐
│  huggingface.co (Hub)                                    │
│  ├─ Models      … weights + config + README (Model Card) │
│  ├─ Datasets    … data for training and evaluation       │
│  └─ Spaces      … demos with Gradio, etc. (Chapter 9)    │
└───────────────────────────┬──────────────────────────────┘
                            │  hf download / snapshot_download
┌───────────────────────────▼──────────────────────────────┐
│  Local cache (~/.cache/huggingface, etc.)                │
└──────────────────────────────────────────────────────────┘

Reimu
In Chapter 0's Exercise B, pipeline downloaded stuff on its own — was that via the Hub too?

Marisa
Exactly, ZE. Just writing model="distilbert-base-uncased-finetuned-sst-2-english" fetches from the Hub and puts it in the cache. In Chapter 1 we build the habit of deliberately browsing the Hub before downloading.

1.2 Creating an Account and Issuing a Token

Reimu
…In Chapter 0 you said not to leak the login token. How do you actually create one?

Marisa
Follow these steps. Sign up on the Hub in your browser.

Create an account at https://huggingface.co/join (email or GitHub / Google sign-in)
Top-right icon → Settings → Access Tokens
Create a New token. Choose permissions according to use

Token type	Use
Read	Download public models; download gated models after access approval
Write	Push to your own repositories (Chapter 6 onward)

Reimu
Is it bad to copy the token into Notepad?

Marisa
Pretty much, ZE. Pushing to GitHub or taking screenshots can leak it. Put it in .env or the OS credential store and never hard-code it in source.

Example .env (repository root. Chapter 0's .gitignore includes .env):

# Always include in .gitignore
.env

# .env (example — issue the real value in Settings)
HF_TOKEN=hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

To read from Python (optional):

pip install python-dotenv

from dotenv import load_dotenv
import os

load_dotenv()
token = os.environ.get("HF_TOKEN")  # None if unset

Logging in with the CLI

Marisa
You installed huggingface_hub in Chapter 0. Use huggingface-cli login to save the token locally.

cd /path/to/yukkuri-hugging-face
source .venv/bin/activate

huggingface-cli login

When prompted, paste the Read token issued in Settings (input is hidden while typing).

Expected flow:

    _|    _|  _|    _|    _|_|_|    _|_|_|  _|_|_|  _|      _|    _|_|_|      _|_|_|_|    _|_|      _|_|_|  _|_|_|_|
    _|    _|  _|    _|  _|        _|          _|    _|_|    _|  _|            _|        _|    _|  _|        _|
    _|_|_|_|  _|    _|  _|  _|_|  _|  _|_|    _|    _|  _|  _|  _|  _|_|      _|_|_|    _|_|_|_|  _|        _|_|_|
    _|    _|  _|    _|  _|    _|  _|    _|    _|    _|    _|_|  _|    _|      _|        _|    _|  _|        _|
    _|    _|    _|_|      _|_|_|    _|_|_|  _|_|_|  _|      _|    _|_|_|      _|        _|    _|    _|_|_|  _|_|_|_|

    A token is already saved on your machine. Run `huggingface-cli whoami` to get more information or `huggingface-cli logout` if you want to log out.
    Enter your token (input will not be visible):
    Add token as git credential? (Y/n) n
    Token is valid (permission: read).
    Your token has been saved to /Users/you/.cache/huggingface/token
    Login successful

Reimu
Can I check with whoami?

Marisa
You can, ZE.

huggingface-cli whoami

user_name

To log out:

huggingface-cli logout

Reimu
For downloads only, it worked without logging in, right?

Marisa
Public models often download without login. But gated models (models that require agreeing to terms of use) and private repositories need a Read token. Logging in early makes life easier later.

1.3 How to Read Model Cards and Dataset Cards

Reimu
Model pages are overwhelming — so many files…

Marisa
Start by reading the README (Model Card) from the top. That's the right move, ZE. As an example, let's look at the sentiment analysis model we use often in this book.

Model ID: distilbert-base-uncased-finetuned-sst-2-english
URL: https://huggingface.co/distilbert-base-uncased/distilbert-base-uncased-finetuned-sst-2-english

Items to grasp in the Model Card:

Section	What to look at
Model description	What task it's for, what the base model is
Intended uses	Intended use cases and situations where you should not use it
Training data	What it was trained on (clues about bias)
Evaluation	Benchmark scores
Limitations	Weaknesses, language, domain constraints
How to use	Sample code for `pipeline` or `AutoModel`

On the Files and versions tab, check the actual files.

File	Role
`config.json`	Architecture settings such as vocabulary size
`tokenizer.json` / `vocab.txt`	Tokenizer definitions
`model.safetensors` or `pytorch_model.bin`	Trained weights
`README.md`	Model Card body

Reimu
Are datasets the same?

Marisa
Dataset Cards also center on the README. Example: imdb movie review sentiment data.

https://huggingface.co/datasets/imdb

On a Dataset Card, always check data provenance, license, and presence of personal information. When you load_dataset("imdb") in Chapter 5, you'll be in the state of having read this card.

1.4 The Habit of Checking Licenses and Terms of Use

Reimu
"Just download it" doesn't mean commercial use is OK, right?

Marisa
Exactly, ZE. Build the habit of checking the License badge on the Hub together with terms of use in the README.

License example	Rough meaning
apache-2.0	Open-source style, relatively easy for commercial use
mit	Permissive conditions (copyright notice, etc.)
cc-by-4.0	Attribution often required
cc-by-nc-4.0	Non-commercial (NC) — watch out for commercial products
other / custom	Read the full README. Gated models have an agreement screen

Gated models won't let you download weights until you agree to terms of use on the Hub. A logged-in token is required.

# Typical when a gated model returns 403
# → Open the model page in a browser and click Agree and access
huggingface-cli login

Reimu
I want something like a checklist.

Marisa
Before downloading, at least check these.

Does the License badge allow commercial use and redistribution?
Do Intended uses / Limitations include your use case?
If gated, have you agreed?
For datasets, is there mention of personal data or bias?

Hands-on 1 — Browse the Hub and Download

Marisa
Following Chapter 0's "read → run → modify → reflect," we'll do three exercises, ZE.

Hands-on 1-A — Find a model on the Hub and read the README

Exercise A On the Hub Models page, use filters to pick one model with task Text Classification and language Japanese (or English), and read the Model Card for three minutes.

Browser steps as a guide:

1. https://huggingface.co/models
2. Left sidebar Task → Text Classification
3. Language → japanese (english is fine if japanese isn't available)
4. Open a model you like and note Intended uses / Limitations from the README

Reimu
For Japanese models, should I pick by star count?

Marisa
Download counts and likes are useful references, but task match and license come first. Picking by stars alone can land you an LLM that's wrong for your task.

Hands-on 1-B — Log in with `huggingface-cli login`

Exercise B Log in as in Section 1.2 and confirm your username appears with whoami.

cd /path/to/yukkuri-hugging-face
source .venv/bin/activate

huggingface-cli login
huggingface-cli whoami

If you haven't already, put HF_TOKEN in .env for future scripts (switch to a Write token before pushing in Chapter 6).

Hands-on 1-C — Download a model locally and peek inside

Exercise C Use the book's bundled script to explicitly download a model and list files in the cache. On first run expect a download on the order of hundreds of MB; depending on your connection it may take several minutes.

python scripts/download_model.py \
  --model-id distilbert-base-uncased-finetuned-sst-2-english

Example expected output:

Downloading snapshot for: distilbert-base-uncased-finetuned-sst-2-english
Local path: /Users/you/.cache/huggingface/hub/models--distilbert-base-uncased-finetuned-sst-2-english/snapshots/xxxxxxxx
Done. Run inspect_model.py with the same --model-id to list files.

Inspect the contents:

python scripts/inspect_model.py \
  --model-id distilbert-base-uncased-finetuned-sst-2-english

Model ID: distilbert-base-uncased-finetuned-sst-2-english
Cache root: /Users/you/.cache/huggingface/hub/...

Files (N):
  config.json                                    1.2 KB
  model.safetensors                            256.3 MB
  tokenizer.json                               456.1 KB
  tokenizer_config.json                          1.1 KB
  vocab.txt                                     226.0 KB
  README.md                                     12.4 KB

config.json (excerpt):
  model_type: distilbert
  num_labels: 2
  ...

Reimu
How is snapshot_download different from Chapter 0's pipeline?

Marisa
pipeline lazily fetches only the files needed for inference. snapshot_download drops a full snapshot of the repository into a specified directory (or the cache). It's good for peeking inside and practicing offline distribution.

Core part of download_model.py:

from huggingface_hub import snapshot_download

path = snapshot_download(repo_id=model_id)
print("Local path:", path)

Modification exercise (optional) Try copying into the project with --output-dir ./models/my-model.

python scripts/download_model.py \
  --model-id distilbert-base-uncased-finetuned-sst-2-english \
  --output-dir ./models/sst2

Full Chapter Scripts

Marisa
So you can practice even without the repository, this chapter's finished scripts are included as-is. Save them under scripts/ with the same filenames and run them, ZE. (Where it says python scripts/xxx.py, adjust the path to match where you saved them.)

`scripts/download_model.py`

#!/usr/bin/env python3
"""Download a model snapshot from Hugging Face Hub."""

from __future__ import annotations

import argparse
from pathlib import Path

from huggingface_hub import snapshot_download


def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(
        description="Download a model repository snapshot from the Hub.",
    )
    parser.add_argument(
        "--model-id",
        required=True,
        help='Hub model ID, e.g. "distilbert-base-uncased-finetuned-sst-2-english"',
    )
    parser.add_argument(
        "--output-dir",
        default=None,
        help="Optional local directory. Default: Hub cache only.",
    )
    parser.add_argument(
        "--revision",
        default=None,
        help="Git revision (branch, tag, or commit). Default: main.",
    )
    return parser.parse_args()


def main() -> None:
    args = parse_args()
    kwargs: dict = {"repo_id": args.model_id, "repo_type": "model"}
    if args.revision:
        kwargs["revision"] = args.revision
    if args.output_dir:
        kwargs["local_dir"] = args.output_dir
        kwargs["local_dir_use_symlinks"] = False

    print(f"Downloading snapshot for: {args.model_id}")
    if args.output_dir:
        print(f"Output directory: {Path(args.output_dir).resolve()}")
    print("(First download may take several minutes depending on model size.)")

    path = snapshot_download(**kwargs)
    print(f"Local path: {path}")
    print("Done. Run inspect_model.py with the same --model-id to list files.")


if __name__ == "__main__":
    main()

`scripts/inspect_model.py`

#!/usr/bin/env python3
"""Inspect cached Hub model files and show config excerpt."""

from __future__ import annotations

import argparse
import json
from pathlib import Path

from huggingface_hub import scan_cache_dir, snapshot_download


def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(
        description="List files for a cached Hub model and show config.json excerpt.",
    )
    parser.add_argument(
        "--model-id",
        required=True,
        help='Hub model ID, e.g. "distilbert-base-uncased-finetuned-sst-2-english"',
    )
    return parser.parse_args()


def human_size(num_bytes: int) -> str:
    if num_bytes < 1024:
        return f"{num_bytes} B"
    if num_bytes < 1024**2:
        return f"{num_bytes / 1024:.1f} KB"
    if num_bytes < 1024**3:
        return f"{num_bytes / 1024**2:.1f} MB"
    return f"{num_bytes / 1024**3:.2f} GB"


def find_repo_in_cache(model_id: str) -> Path | None:
    cache_info = scan_cache_dir()
    needle = model_id.replace("/", "--")
    for repo in cache_info.repos:
        if needle in repo.repo_id.replace("/", "--") or repo.repo_id.endswith(model_id):
            if repo.revisions:
                return Path(repo.revisions[0].snapshot_path)
    return None


def main() -> None:
    args = parse_args()
    print(f"Model ID: {args.model_id}")

    local_path = find_repo_in_cache(args.model_id)
    if local_path is None:
        print("Not found in cache. Downloading snapshot first...")
        print("(First download may take several minutes.)")
        local_path = Path(snapshot_download(repo_id=args.model_id))

    print(f"Snapshot path: {local_path}")

    files = sorted(local_path.iterdir(), key=lambda p: p.name)
    print(f"\nFiles ({len(files)}):")
    for file_path in files:
        if file_path.is_file():
            size = human_size(file_path.stat().st_size)
            print(f"  {file_path.name:<40} {size:>10}")

    config_path = local_path / "config.json"
    if config_path.exists():
        with config_path.open(encoding="utf-8") as f:
            config = json.load(f)
        print("\nconfig.json (excerpt):")
        excerpt = {
            k: config[k]
            for k in (
                "model_type",
                "architectures",
                "num_labels",
                "vocab_size",
                "hidden_size",
            )
            if k in config
        }
        print(json.dumps(excerpt, indent=2, ensure_ascii=False))
    else:
        print("\nconfig.json not found in snapshot.")


if __name__ == "__main__":
    main()

Reimu's Notebook

Hub is the shared place for models, datasets, and Spaces. Read Model Cards / Dataset Cards before downloading.
Save a Read token with huggingface-cli login. Manage tokens in .env; never hard-code them.
Build the habit of pulling locally with snapshot_download and checking config.json and weight files.

Marisa's One More Thing

Models on the Hub use IDs in the form organization/model-name. Copying the ID from the browser URL reduces typos, ZE.

# Fetch metadata only from the CLI (no download)
huggingface-cli repo info distilbert-base-uncased-finetuned-sst-2-english

# Display the start of the README in Python
from huggingface_hub import hf_hub_download

readme_path = hf_hub_download(
    repo_id="distilbert-base-uncased-finetuned-sst-2-english",
    filename="README.md",
)
print(open(readme_path, encoding="utf-8").read()[:500])

On to the Next Chapter

Reimu
I know where models live on the Hub now. But won't it get tiring to download and peek inside every time?

Marisa
For day-to-day use, Chapter 2's pipeline is enough, ZE. Three lines to inference. We'll peel back Tokenizer and weight details from Chapter 3 onward.

Reimu
Chapter 0's Exercise B — the main story's here.

Marisa
Exactly. Move on to Chapter 2: Experience "3-Line Inference" with Pipeline. From sentiment analysis to translation, images, and speech — experience the same API pattern, ZE.

Chapter 2: Pipeline — Experience "3-Line Inference"

2.1 Installing the Transformers Library

Reimu
In Chapter 1 we downloaded a model from the Hub. So how do we run inference?

Marisa
transformers from Chapter 0 is the core library, ZE. Just double-check the version.

cd /path/to/yukkuri-hugging-face
source .venv/bin/activate

python -c "import transformers; print(transformers.__version__)"

4.41.2

If something's missing:

pip install "transformers>=4.40" torch sentencepiece

Reimu
If we already installed it in Chapter 0, skip this section?

Marisa
If check_env.py passes, skip is fine. From Chapter 2 onward extra packages per task may be needed (pillow for images, librosa for audio, etc.). When errors appear, pip install as needed.

2.2 What Is `pipeline`? — Preprocessing Through Inference in One Place

Reimu
Chapter 0's Exercise B — the thing that ran in about three lines. What's inside?

Marisa
pipeline is a high-level interface that bundles Tokenizer → Model → post-processing into one API, ZE.

  Input text
       │
       ▼
┌──────────────┐
│  Tokenizer   │  string → ID sequence
└──────┬───────┘
       ▼
┌──────────────┐
│    Model     │  inference (we look inside in Chapter 4)
└──────┬───────┘
       ▼
┌──────────────┐
│  Post-process│  label names, score formatting
└──────┬───────┘
       ▼
  [{'label': 'POSITIVE', 'score': 0.99}]

Minimal example (Chapter 0 Exercise B again):

from transformers import pipeline

clf = pipeline(
    "sentiment-analysis",
    model="distilbert-base-uncased-finetuned-sst-2-english",
)
print(clf("I love Hugging Face!"))

[{'label': 'POSITIVE', 'score': 0.9998}]

Reimu
Same model ID as the one we downloaded in Chapter 1.

Marisa
Same, ZE. On first run it fetches from the Hub. From the second run onward it reads from cache, so it's fast. Chapter 1's download_model.py is practice for "download everything first"; pipeline is the practical route of "fetch when needed."

Main arguments when creating a pipeline:

Argument	Meaning
1st argument (task name)	`"sentiment-analysis"`, `"translation"`, `"automatic-speech-recognition"`, etc.
`model`	Hub model ID. If omitted, a task default is chosen
`device`	`-1` = CPU, `0` = first GPU (when CUDA is available)

2.3 Try Text Classification and Sentiment Analysis

Marisa
English sentiment analysis is fine with the model from earlier. For Japanese, swap in a Japanese fine-tuned model, ZE.

from transformers import pipeline

# First download (may be hundreds of MB)
clf = pipeline(
    "sentiment-analysis",
    model="distilbert-base-uncased-finetuned-sst-2-english",
)
print(clf("What a beautiful day today"))
print(clf("It was the worst day ever"))

Example expected output (label names vary by model):

[{'label': 'POSITIVE', 'score': 0.92}]
[{'label': 'NEGATIVE', 'score': 0.88}]

Pass multiple sentences at once:

texts = ["Awesome!", "Meh…", "Hugging Face is fun"]
print(clf(texts))

Reimu
What if I feed Japanese into an English model?

Marisa
It may run but you often get meaningless results, ZE. Trust the Model Card's supported languages. Hands-on 2-A tries this with the bundled script.

python scripts/ch02_pipeline_sentiment.py \
  --text "Let's take it easy"

2.4 Try Translation, Summarization, and Question Answering

Marisa
Change the task name and you can use the same pipeline pattern.

Translation (EN → FR)

from transformers import pipeline

translator = pipeline(
    "translation_en_to_fr",
    model="Helsinki-NLP/opus-mt-en-fr",
)
print(translator("Hello, how are you?"))

[{'translation_text': 'Bonjour, comment allez-vous ?'}]

Summarization (English)

summarizer = pipeline(
    "summarization",
    model="facebook/bart-large-cnn",
)
article = (
    "The Hugging Face Hub is a platform for sharing machine learning models. "
    "Researchers and developers upload models, datasets, and demo applications."
)
print(summarizer(article, max_length=30, min_length=10, do_sample=False))

First download for BART-large can exceed 1 GB. Run when you have time to spare.

Question answering (extractive QA)

qa = pipeline(
    "question-answering",
    model="distilbert-base-cased-distilled-squad",
)
context = "Hugging Face was founded in 2016. The Hub hosts models and datasets."
print(qa(question="When was Hugging Face founded?", context=context))

{'score': 0.45, 'start': 25, 'end': 29, 'answer': '2016'}

Reimu
For summarization, can I paste a Japanese article as-is?

Marisa
facebook/bart-large-cnn is for English. Japanese summarization needs a different model from the Hub. In Chapter 2 the goal is to learn the shape of the API, ZE.

Script for translation experiments:

python scripts/ch02_pipeline_translation.py \
  --text "I will learn Hugging Face step by step."

2.5 Image Classification and Object Detection (Vision)

Reimu
Can pipeline handle non-text too?

Marisa
It can, ZE. Here's image classification (first download takes time).

pip install pillow

from transformers import pipeline

classifier = pipeline(
    "image-classification",
    model="google/vit-base-patch16-224",
)
result = classifier(
    "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/pipeline-cat-chonk.jpeg"
)
print(result[:3])

[{'label': 'Egyptian cat', 'score': 0.85}, ...]

Object detection (with bounding boxes):

detector = pipeline(
    "object-detection",
    model="facebook/detr-resnet-50",
)
# Local image path or URL both work
results = detector("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/coco_sample.png")
for obj in results[:3]:
    print(obj["label"], round(obj["score"], 3), obj["box"])

Reimu
Sounds rough without a GPU.

Marisa
ViT-base often runs in a few to tens of seconds on CPU. DETR is heavier. Trying on Colab GPU another day is fine too.

python scripts/ch02_pipeline_vision.py \
  --image-url "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/pipeline-cat-chonk.jpeg"

2.6 Speech Recognition (Whisper and Others)

Marisa
Whisper models use the automatic-speech-recognition task. Start with the smaller openai/whisper-tiny, ZE.

pip install librosa soundfile

from transformers import pipeline

asr = pipeline(
    "automatic-speech-recognition",
    model="openai/whisper-tiny",
)
# Sample audio URL (English)
sample = "https://huggingface.co/datasets/Narsil/asr_dummy/resolve/main/1.flac"
print(asr(sample))

{'text': ' He hoped there would be stew for dinner, turnips and carrots and bruised potatoes and fat mutton pieces to be ladled out in thick, peppered flour-fatten sauce.'}

To try Japanese, pass a short audio file with --audio:

python scripts/ch02_pipeline_whisper.py --audio /path/to/your/sample.wav

Whisper is multilingual, but accuracy drops on noisy recordings. Read the Model Card Limitations too.

Hands-on 2 — Pipeline Practice

Hands-on 2-A — Japanese text sentiment analysis

Exercise A Use the bundled script to classify sentiment for Japanese text.

python scripts/ch02_pipeline_sentiment.py \
  --text "This book is so easy to follow" \
  --model distilbert-base-uncased-finetuned-sst-2-english

Modification Add three sentences with --text and redirect results to log/ch02_sentiment.txt.

mkdir -p log
python scripts/ch02_pipeline_sentiment.py \
  --text "Awesome" --text "Terrible" --text "So-so" \
  2>&1 | tee log/ch02_sentiment.txt

Hands-on 2-B — Swap English–Japanese translation models

Exercise B Translate the same English sentence while changing the model ID.

python scripts/ch02_pipeline_translation.py \
  --text "The weather is nice today." \
  --model Helsinki-NLP/opus-mt-en-ja

Swap --model to another model (confirm it exists on the Hub) and note differences in the translation.

Reimu
I'm gonna typo model names…

Marisa
Build the habit of copying IDs from Hub URLs from Chapter 1. 404 or Repository Not Found is usually a typo, ZE.

Hands-on 2-C — Compare accuracy and speed by swapping models on the same task

Exercise C For English sentiment analysis, compare lightweight vs. somewhat larger.

python scripts/ch02_pipeline_compare.py \
  --task sentiment-analysis \
  --text "I love Hugging Face!" \
  --models distilbert-base-uncased-finetuned-sst-2-english \
           bert-base-uncased \
  --runs 3

Example expected output:

Model: distilbert-base-uncased-finetuned-sst-2-english
  Result: [{'label': 'POSITIVE', 'score': 0.9998}]
  Avg time (3 runs, excl. 1st load): 0.042 s

Model: bert-base-uncased
  ...

Reimu
bert-base-uncased isn't fine-tuned, is it?

Marisa
Sharp, ZE. Hands-on 2-C is practice for "same API, only change model=". Comparing fine-tuned vs. non-fine-tuned can make labels and scores meaningless — remember that as a teaching point. In production, pick task fine-tuned models per the Model Card.

End of Chapter: Common Errors and Fixes

Reimu
I want an error collection.

Marisa
Here's a table of frequent Chapter 2 issues, ZE.

Symptom	Example cause	Fix
`CUDA out of memory`	Insufficient GPU memory	Smaller model, CPU with `device=-1`, `pipeline(..., model_kwargs={"torch_dtype": ...})`
`Repository Not Found`	Model ID typo / private repo	Copy ID from Hub URL. `login` for private
`401 / 403`	Gated not agreed / missing token	Agree in browser, `huggingface-cli login`
`Connection error`	Offline / proxy	Check network. Corporate proxy may need `HF_HUB_ENABLE_HF_TRANSFER`, etc.
Slow only on first run	Normal (downloading)	Wait. Check cache location from Chapter 1
Japanese is gibberish	Language mismatch	Switch to a Japanese fine-tuned model

Check CUDA:

import torch
print(torch.cuda.is_available())

Create a pipeline pinned to CPU:

clf = pipeline("sentiment-analysis", model="distilbert-base-uncased-finetuned-sst-2-english", device=-1)

Reduce memory (advanced, GPU environment):

import torch
clf = pipeline(
    "summarization",
    model="facebook/bart-large-cnn",
    device=0,
    model_kwargs={"torch_dtype": torch.float16},
)

End of Chapter: Review Quiz (Reimu vs Marisa)

Marisa
Three questions only. Reimu, try answering.

Q1. What do you pass as the first argument to pipeline?

Reimu
The task name! Like "sentiment-analysis".

Marisa
Correct, ZE.

Q2. For the same sentiment-analysis, can you use a Japanese fine-tuned model on English text?

Reimu
Pretty much no. Check the Model Card language.

Marisa
Exactly.

Q3. Why is pipeline faster from the second run onward?

Reimu
Chapter 1's cache. It's not downloading from the Hub every time.

Marisa
Nearly perfect. Next chapter we peel off the Tokenizer and watch how strings become IDs.

Full Chapter Scripts

`scripts/ch02_pipeline_sentiment.py`

#!/usr/bin/env python3
"""Japanese sentiment analysis via transformers pipeline."""

from __future__ import annotations

import argparse

from transformers import pipeline


def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(description="Run sentiment-analysis pipeline on Japanese text.")
    parser.add_argument(
        "--model",
        default="daigo/bert-base-japanese-sentiment",
        help="Hub model ID for Japanese sentiment.",
    )
    parser.add_argument(
        "--text",
        action="append",
        required=True,
        help="Input text (repeatable). Example: --text 'Awesome' --text 'Terrible'",
    )
    parser.add_argument(
        "--device",
        type=int,
        default=-1,
        help="Device index (-1 for CPU, 0 for first CUDA GPU).",
    )
    return parser.parse_args()


def main() -> None:
    args = parse_args()
    print(f"Loading pipeline: sentiment-analysis / {args.model}")
    print("(First run downloads weights from the Hub; may take a few minutes.)")

    clf = pipeline(
        "sentiment-analysis",
        model=args.model,
        device=args.device,
    )

    for text in args.text:
        result = clf(text)[0]
        label = result.get("label", result)
        score = result.get("score", 0.0)
        print(f"\nText: {text}")
        print(f"  -> {label} (score={score:.4f})")


if __name__ == "__main__":
    main()

`scripts/ch02_pipeline_translation.py`

#!/usr/bin/env python3
"""English to Japanese translation via transformers pipeline."""

from __future__ import annotations

import argparse

from transformers import pipeline


def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(description="Translate English text to Japanese.")
    parser.add_argument(
        "--text",
        required=True,
        help="English source sentence.",
    )
    parser.add_argument(
        "--model",
        default="Helsinki-NLP/opus-mt-en-ja",
        help="Hub translation model ID.",
    )
    parser.add_argument(
        "--device",
        type=int,
        default=-1,
        help="Device index (-1 for CPU, 0 for first CUDA GPU).",
    )
    return parser.parse_args()


def main() -> None:
    args = parse_args()
    print(f"Loading pipeline: translation_en_to_ja / {args.model}")
    print("(First run downloads weights from the Hub; may take a few minutes.)")

    translator = pipeline(
        "translation_en_to_ja",
        model=args.model,
        device=args.device,
    )

    result = translator(args.text)[0]
    translation = result.get("translation_text", result)
    print(f"\nEN: {args.text}")
    print(f"JA: {translation}")


if __name__ == "__main__":
    main()

`scripts/ch02_pipeline_compare.py`

#!/usr/bin/env python3
"""Compare pipeline models on the same task (latency + output)."""

from __future__ import annotations

import argparse
import time

from transformers import pipeline


def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(
        description="Run the same pipeline task with multiple models and compare timing.",
    )
    parser.add_argument(
        "--task",
        default="sentiment-analysis",
        help='Pipeline task name, e.g. "sentiment-analysis".',
    )
    parser.add_argument(
        "--text",
        required=True,
        help="Input text for inference.",
    )
    parser.add_argument(
        "--models",
        nargs="+",
        required=True,
        help="One or more Hub model IDs.",
    )
    parser.add_argument(
        "--runs",
        type=int,
        default=3,
        help="Timed runs per model (after warmup).",
    )
    parser.add_argument(
        "--device",
        type=int,
        default=-1,
        help="Device index (-1 for CPU, 0 for first CUDA GPU).",
    )
    return parser.parse_args()


def main() -> None:
    args = parse_args()

    for model_id in args.models:
        print(f"\nModel: {model_id}")
        print("(First load may download from the Hub and is excluded from avg timing.)")

        pipe = pipeline(args.task, model=model_id, device=args.device)

        # Warmup + first result display
        first = pipe(args.text)
        print(f"  Result: {first}")

        if args.runs < 1:
            continue

        start = time.perf_counter()
        for _ in range(args.runs):
            pipe(args.text)
        elapsed = time.perf_counter() - start
        avg = elapsed / args.runs
        print(f"  Avg time ({args.runs} runs, excl. 1st load): {avg:.3f} s")


if __name__ == "__main__":
    main()

`scripts/ch02_pipeline_vision.py`

#!/usr/bin/env python3
"""Image classification demo via transformers pipeline."""

from __future__ import annotations

import argparse

from transformers import pipeline


def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(description="Classify an image from URL or local path.")
    parser.add_argument(
        "--image-url",
        default=(
            "https://huggingface.co/datasets/huggingface/documentation-images/"
            "resolve/main/pipeline-cat-chonk.jpeg"
        ),
        help="Image URL or local file path.",
    )
    parser.add_argument(
        "--model",
        default="google/vit-base-patch16-224",
        help="Hub image-classification model ID.",
    )
    parser.add_argument(
        "--top-k",
        type=int,
        default=3,
        help="Number of top labels to print.",
    )
    parser.add_argument(
        "--device",
        type=int,
        default=-1,
        help="Device index (-1 for CPU, 0 for first CUDA GPU).",
    )
    return parser.parse_args()


def main() -> None:
    args = parse_args()
    print(f"Loading pipeline: image-classification / {args.model}")
    print("(First run downloads weights from the Hub; may take several minutes.)")

    classifier = pipeline(
        "image-classification",
        model=args.model,
        device=args.device,
    )

    results = classifier(args.image_url)
    print(f"\nImage: {args.image_url}")
    for rank, item in enumerate(results[: args.top_k], start=1):
        print(f"  {rank}. {item['label']}: {item['score']:.4f}")


if __name__ == "__main__":
    main()

`scripts/ch02_pipeline_whisper.py`

#!/usr/bin/env python3
"""Speech recognition demo via Whisper pipeline."""

from __future__ import annotations

import argparse

from transformers import pipeline


DEFAULT_SAMPLE_URL = (
    "https://huggingface.co/datasets/Narsil/asr_dummy/resolve/main/1.flac"
)


def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(description="Transcribe audio with Whisper pipeline.")
    parser.add_argument(
        "--audio",
        default=DEFAULT_SAMPLE_URL,
        help="Path to local audio file or URL (default: Hub sample FLAC).",
    )
    parser.add_argument(
        "--model",
        default="openai/whisper-tiny",
        help="Hub ASR model ID.",
    )
    parser.add_argument(
        "--device",
        type=int,
        default=-1,
        help="Device index (-1 for CPU, 0 for first CUDA GPU).",
    )
    return parser.parse_args()


def main() -> None:
    args = parse_args()
    print(f"Loading pipeline: automatic-speech-recognition / {args.model}")
    print("(First run downloads weights from the Hub; may take several minutes.)")

    asr = pipeline(
        "automatic-speech-recognition",
        model=args.model,
        device=args.device,
    )

    result = asr(args.audio)
    text = result.get("text", result) if isinstance(result, dict) else result
    print(f"\nAudio: {args.audio}")
    print(f"Text: {text}")


if __name__ == "__main__":
    main()

Reimu's Notebook

pipeline(task, model=ID) bundles preprocessing through inference and post-processing. That's the real Chapter 0 three-line inference.
Pick models from Hub Model Cards with matching language and task. Don't reuse English models for Japanese.
First run is slow due to download — that's normal. Later runs use the cache. Suspect typos, gated access, or CUDA OOM for errors.

Marisa's One More Thing

Internally, pipeline runs in inference mode equivalent to torch.no_grad(). When you want to batch yourself, that's prep for using the Tokenizer directly in Chapter 3, ZE.

from transformers import pipeline

# Format return value JSON-style (for logging)
clf = pipeline("sentiment-analysis", model="distilbert-base-uncased-finetuned-sst-2-english")
out = clf("Hello")[0]
print(f"{out['label']} ({out['score']:.4f})")

On to the Next Chapter

Reimu
pipeline is handy, but it feels like a black box inside.

Marisa
Next we peel from the Tokenizer. Chapter 3: Tokenizer — Turning Strings into Forms Models Understand. You'll see with your own eyes how the same English sentence splits into different tokens per model, ZE.

Reimu
I'm curious whether "love" is one token or two.

Marisa
Hands-on 3-A lines them up side by side. It becomes the foundation leading to the Model in Chapter 4, ZE.

Chapter 3　Tokenizer — Turning Strings into a Form Models Can Understand

3.1 Why Tokenization Is Needed

Reimu
In Chapter 2's pipeline, the raw characters don't go straight into the model, right?

Marisa
Exactly, ZE. Neural nets only handle sequences of numbers. The Tokenizer converts string → token ID sequence.

  "I love HF"
       │
       ▼  Tokenizer
  tokens: ["I", "love", "H", "##F"]   ← splitting differs by model / method
  ids:    [101, 1045, 2293, 123, 456, 102]
       │
       ▼  Model
  logits / hidden states

Reimu
So one word doesn't always equal one token?

Marisa
Not necessarily. With English WordPiece, you get subword splitting like playing → play + ##ing. Words not in the vocabulary get split into smaller pieces to reduce unknown tokens.

Method	Example models	Characteristics
WordPiece	BERT family	Subwords with `##` prefix
BPE	GPT-2, RoBERTa	Vocabulary built via byte pairs
SentencePiece	Multilingual, Japanese	Less dependent on whitespace

Reimu
Is that why the Japanese and English models in Chapter 2 gave different results — the tokenizer?

Marisa
It's both the Tokenizer and the training data, ZE. Even the same sentence gets split differently per model — we'll see that in code next.

3.2 Basics of `AutoTokenizer`

Marisa
AutoTokenizer automatically picks the matching Tokenizer from a Hub model ID. It comes down from the Hub together with config.json from Chapter 1.

from transformers import AutoTokenizer

model_id = "distilbert-base-uncased-finetuned-sst-2-english"
tokenizer = AutoTokenizer.from_pretrained(model_id)

print(type(tokenizer).__name__)
print("Vocab size:", tokenizer.vocab_size)

DistilBertTokenizerFast
Vocab size: 30522

Reimu
from_pretrained — same vibe as the download in Chapter 1.

Marisa
Same from_pretrained(model_id) pattern. We'll load the Model the same way in Chapter 4. On first run, tokenizer files download from the Hub (a few seconds to tens of seconds).

To use only the local cache (offline):

tokenizer = AutoTokenizer.from_pretrained(model_id, local_files_only=True)

Chapter 2's pipeline uses the same Tokenizer internally.

3.3 Encode, Decode, and Special Tokens

Marisa
The basic operations are encode (text → IDs) and decode (IDs → text), ZE.

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained(
    "distilbert-base-uncased-finetuned-sst-2-english"
)

text = "I love Hugging Face!"
encoded = tokenizer.encode(text)
print("IDs:", encoded)
print("Decoded:", tokenizer.decode(encoded))

IDs: [101, 1045, 2293, 9259, 3563, 999, 102]
Decoded: [CLS] i love hugging face! [SEP]

Special tokens are defined per model.

Token	BERT-family example	Role
`[CLS]`	Start of sequence	Representative token for classification
`[SEP]`	End / separator	Boundary between sentence pairs
`<pad>`	Padding for short sequences	Batch length alignment
`<unk>`	Out-of-vocabulary piece	Unknown token (rare with modern tokenizers)

Check via attributes:

print(tokenizer.cls_token, tokenizer.sep_token, tokenizer.pad_token)
print(tokenizer.cls_token_id, tokenizer.sep_token_id)

Reimu
After decode, everything turned lowercase.

Marisa
distilbert-base-uncased assumes lowercase normalization, ZE. Uppercase information is discarded. For models that distinguish case, pick a cased version.

See details as a dict:

batch = tokenizer(text, return_tensors="pt")
print(batch.keys())       # input_ids, attention_mask
print(batch["input_ids"])
print(batch["attention_mask"])

Keys: dict_keys(['input_ids', 'attention_mask'])
tensor([[101, 1045, 2293, 9259, 3563, 999, 102]])
tensor([[1, 1, 1, 1, 1, 1, 1]])

attention_mask marks real tokens as 1 and padding as 0. You pass it together with inputs to the Model in Chapter 4.

3.4 Padding and Truncation

Reimu
For batch processing, sentences can't all be different lengths, right?

Marisa
Exactly, ZE. Tensors need to be rectangular. padding extends short sequences; truncation cuts long ones.

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained(
    "distilbert-base-uncased-finetuned-sst-2-english"
)

long_text = "word " * 200
short_text = "hello"

# Truncate individually to max length 512
enc = tokenizer(long_text, truncation=True, max_length=512)
print(len(enc["input_ids"]))

# Batch: pad to the longest in the batch (standalone tokenizer may need pad_token set)
tokenizer.pad_token = tokenizer.eos_token if tokenizer.pad_token is None else tokenizer.pad_token

batch = tokenizer(
    [short_text, long_text],
    padding=True,
    truncation=True,
    max_length=128,
    return_tensors="pt",
)
print(batch["input_ids"].shape)
print(batch["attention_mask"])

512
torch.Size([2, 128])
tensor([[101, 7592, 102, 0, 0, ...],
        [101, ..., 102, ...]])

Keep max_length at or below max_position_embeddings from the Model Card or config.json to stay safe.

Reimu
I've seen pad_token come back as None.

Marisa
Some tokenizers like GPT-family ones have no pad token by default. Borrow an existing special token with something like tokenizer.pad_token = tokenizer.eos_token, or add to the vocabulary as in Hands-on 3-C.

3.5 Batching and `DataCollator`

Marisa
In the training loop (Chapter 6), DataCollatorWithPadding handles dynamic padding. It's the reference pattern for Dataset → batch shaping before inference, ZE.

from transformers import AutoTokenizer, DataCollatorWithPadding

tokenizer = AutoTokenizer.from_pretrained(
    "distilbert-base-uncased-finetuned-sst-2-english"
)
if tokenizer.pad_token is None:
    tokenizer.pad_token = tokenizer.sep_token

collator = DataCollatorWithPadding(tokenizer=tokenizer)

features = [
    tokenizer("First sentence."),
    tokenizer("Second sentence is a bit longer than the first."),
    tokenizer("Third."),
]
batch = collator(features)
print(batch["input_ids"].shape)
print(batch["attention_mask"])

Benefits of DataCollator:

Item	Description
Dynamic padding	Pad to the longest in the batch (less wasted padding)
Trainer integration	Pass directly as `data_collator=collator` in Chapter 6
Consistency	Shared tokenizer settings and padding policy

Reimu
Did Chapter 2's pipeline do all of this automatically?

Marisa
At inference time it handles internal tokenizer calls + pad/trunc for you. Only when you want control yourself (chunking long text manually, adding custom vocabulary) do you use the Chapter 3+ APIs directly.

Hands-on 3 — Tokenizer in Practice

Hands-on 3-A — Compare the Same Sentence Across Multiple Model Tokenizers

Exercise A Take one English sentence and compare token sequences from two Tokenizers side by side.

python scripts/ch03_tokenizer_compare.py \
  --text "Hugging Face makes NLP easy!" \
  --models distilbert-base-uncased-finetuned-sst-2-english \
           bert-base-uncased

Example expected output:

Text: Hugging Face makes NLP easy!

--- distilbert-base-uncased-finetuned-sst-2-english ---
Tokenizer: DistilBertTokenizerFast
Tokens: ['hugging', 'face', 'makes', 'nl', '##p', 'easy', '!']
IDs (first 12): [101, 9238, 3563, 7594, 17953, 7861, 3739, 999, 102]

--- bert-base-uncased ---
Tokenizer: BertTokenizerFast
Tokens: ['hu', '##gging', 'face', 'makes', 'nl', '##p', 'easy', '!']
...

Reimu
The way Hugging gets split is totally different…

Marisa
That's vocabulary and training-method differences, ZE. Even within the same model family, cased / uncased changes the split. Always use the Tokenizer paired with its model.

Hands-on 3-B — Chunk Long Text and Run Batch Inference

Exercise B Split long text with max_length and run batch inference on multiple chunks (close to what Chapter 2's pipeline does under the hood).

python scripts/ch03_tokenizer_batch.py \
  --model distilbert-base-uncased-finetuned-sst-2-english \
  --text-file /path/to/yukkuri-hugging-face/docs_en/00.md \
  --max-length 128 \
  --stride 64

If you omit --text-file, a built-in long sample is used.

Example expected output:

Loaded model + tokenizer: distilbert-base-uncased-finetuned-sst-2-english
Chunks: 8 (max_length=128, stride=64)

Chunk 0 -> POSITIVE (0.9821)
Chunk 1 -> POSITIVE (0.9544)
...
Majority vote: POSITIVE

Modification Set --stride equal to max_length for non-overlapping chunks and see how the chunk count changes.

Hands-on 3-C — Extend the Tokenizer by Adding Custom Vocabulary

Exercise C Add the yukkuri-specific word yukkuri to the vocabulary and check how splitting changes.

python scripts/ch03_tokenizer_custom_vocab.py \
  --model distilbert-base-uncased-finetuned-sst-2-english \
  --new-token "yukkuri" \
  --text "yukkuri let's take it easy"

Example expected output:

Before add: tokens=['yu', '##kk', '##uri', ...]  (many subwords)
After add:  tokens=['yukkuri', ...]
New token id: 30522
Saved to: ./models/tokenizer-with-yukkuri

Reimu
If you add vocabulary, don't you have to change the Model weights too?

Marisa
Sharp. In serious fine-tuning, embedding size changes, so the Model side needs adjustment (Chapters 6 and 7). Hands-on 3-C is the stage for learning the tokenizer extension workflow, ZE. For inference only, understand that added tokens may stay randomly initialized embeddings with no real meaning yet.

Full Chapter Scripts

Marisa
So you can practice even without the repo, this chapter includes the complete scripts as-is. Save them locally under scripts/ with the same filenames and run them, ZE. (Where you see python scripts/xxx.py, adjust the path to match where you saved the file.)

`scripts/ch03_tokenizer_compare.py`

#!/usr/bin/env python3
"""Compare tokenization across multiple Hub models."""

from __future__ import annotations

import argparse

from transformers import AutoTokenizer


def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(
        description="Show how different tokenizers split the same text.",
    )
    parser.add_argument(
        "--text",
        required=True,
        help="Input text to tokenize.",
    )
    parser.add_argument(
        "--models",
        nargs="+",
        required=True,
        help="One or more Hub model IDs (tokenizer loaded from each).",
    )
    return parser.parse_args()


def main() -> None:
    args = parse_args()
    print(f"Text: {args.text}\n")

    for model_id in args.models:
        print(f"--- {model_id} ---")
        print("(First run may download tokenizer files from the Hub.)")
        tokenizer = AutoTokenizer.from_pretrained(model_id)
        encoded = tokenizer.encode(args.text, add_special_tokens=True)
        tokens = tokenizer.convert_ids_to_tokens(encoded)

        print(f"Tokenizer: {type(tokenizer).__name__}")
        print(f"Vocab size: {tokenizer.vocab_size}")
        print(f"Tokens: {tokens}")
        print(f"IDs (first 16): {encoded[:16]}")
        print(f"Decoded: {tokenizer.decode(encoded)}\n")


if __name__ == "__main__":
    main()

`scripts/ch03_tokenizer_batch.py`

#!/usr/bin/env python3
"""Chunk long text with truncation/stride and run batch sentiment inference."""

from __future__ import annotations

import argparse
from collections import Counter
from pathlib import Path

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer


def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(
        description="Tokenize long text into chunks and classify each chunk.",
    )
    parser.add_argument(
        "--model",
        default="distilbert-base-uncased-finetuned-sst-2-english",
        help="Hub model ID for sequence classification.",
    )
    parser.add_argument(
        "--text-file",
        default=None,
        help="Optional path to a text file. If omitted, uses a built-in long sample.",
    )
    parser.add_argument(
        "--max-length",
        type=int,
        default=128,
        help="Max tokens per chunk (including special tokens).",
    )
    parser.add_argument(
        "--stride",
        type=int,
        default=64,
        help="Stride for overlapping chunks (tokenizer overflow stride).",
    )
    parser.add_argument(
        "--device",
        default="cpu",
        help='Torch device string, e.g. "cpu" or "cuda:0".',
    )
    return parser.parse_args()


def load_text(path: str | None) -> str:
    if path is None:
        return ("Hugging Face is great. " * 80).strip()
    return Path(path).read_text(encoding="utf-8")


def main() -> None:
    args = parse_args()
    text = load_text(args.text_file)

    print(f"Loading model + tokenizer: {args.model}")
    print("(First run downloads weights from the Hub; may take a few minutes.)")

    tokenizer = AutoTokenizer.from_pretrained(args.model)
    model = AutoModelForSequenceClassification.from_pretrained(args.model)
    model.eval()
    device = torch.device(args.device)
    model.to(device)

    encoded = tokenizer(
        text,
        truncation=True,
        max_length=args.max_length,
        stride=args.stride,
        return_overflowing_tokens=True,
        return_tensors="pt",
    )

    num_chunks = encoded["input_ids"].shape[0]
    print(f"Chunks: {num_chunks} (max_length={args.max_length}, stride={args.stride})\n")

    id2label = model.config.id2label
    labels: list[str] = []

    with torch.no_grad():
        for i in range(num_chunks):
            input_ids = encoded["input_ids"][i].unsqueeze(0).to(device)
            attention_mask = encoded["attention_mask"][i].unsqueeze(0).to(device)
            logits = model(input_ids=input_ids, attention_mask=attention_mask).logits
            pred_id = int(logits.argmax(dim=-1).item())
            label = id2label[pred_id]
            score = torch.softmax(logits, dim=-1)[0, pred_id].item()
            labels.append(label)
            print(f"Chunk {i} -> {label} ({score:.4f})")

    majority = Counter(labels).most_common(1)[0][0]
    print(f"\nMajority vote: {majority}")


if __name__ == "__main__":
    main()

`scripts/ch03_tokenizer_custom_vocab.py`

#!/usr/bin/env python3
"""Add a custom token to a tokenizer and save locally."""

from __future__ import annotations

import argparse
from pathlib import Path

from transformers import AutoTokenizer


def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(
        description="Add new tokens to a tokenizer and compare tokenization.",
    )
    parser.add_argument(
        "--model",
        default="distilbert-base-uncased-finetuned-sst-2-english",
        help="Hub model ID to load tokenizer from.",
    )
    parser.add_argument(
        "--new-token",
        action="append",
        required=True,
        help='Token to add (repeatable). Example: --new-token "yukkuri"',
    )
    parser.add_argument(
        "--text",
        default="yukkuri let's take it easy",
        help="Sample text to tokenize before/after adding tokens.",
    )
    parser.add_argument(
        "--output-dir",
        default="./models/tokenizer-custom",
        help="Directory to save the extended tokenizer.",
    )
    return parser.parse_args()


def show_tokens(tokenizer, text: str, prefix: str) -> None:
    ids = tokenizer.encode(text, add_special_tokens=False)
    tokens = tokenizer.convert_ids_to_tokens(ids)
    print(f"{prefix}: tokens={tokens}")
    print(f"{prefix}: ids={ids}")


def main() -> None:
    args = parse_args()
    print(f"Loading tokenizer: {args.model}")
    print("(First run may download tokenizer files from the Hub.)")

    tokenizer = AutoTokenizer.from_pretrained(args.model)
    print(f"\nSample text: {args.text}")
    show_tokens(tokenizer, args.text, "Before add")

    num_added = tokenizer.add_tokens(args.new_token)
    print(f"\nAdded {num_added} token(s): {args.new_token}")

    for token in args.new_token:
        token_id = tokenizer.convert_tokens_to_ids(token)
        print(f"New token id for {token!r}: {token_id}")

    show_tokens(tokenizer, args.text, "After add")

    out = Path(args.output_dir)
    out.mkdir(parents=True, exist_ok=True)
    tokenizer.save_pretrained(out)
    print(f"\nSaved to: {out.resolve()}")
    print(
        "Note: extending vocab changes embedding size; "
        "fine-tune the model (Ch.6–7) before expecting better semantics."
    )


if __name__ == "__main__":
    main()

Reimu's Notebook

Models only consume numeric ID sequences. Match pairs with AutoTokenizer.from_pretrained(model_id).
Be able to read encode / decode and [CLS] [SEP] attention_mask.
For long text use truncation + max_length; for batches use padding or DataCollatorWithPadding.

Marisa's One More Thing

Token count also matters for billing APIs and context length. Get in the habit of counting tokens before inference, ZE.

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained(
    "distilbert-base-uncased-finetuned-sst-2-english"
)
text = "Hugging Face Hub is awesome."
ids = tokenizer.encode(text, add_special_tokens=True)
print("Token count:", len(ids))
print("Tokens:", tokenizer.convert_ids_to_tokens(ids))

Token count: 9
Tokens: ['[CLS]', 'hugging', 'face', 'hub', 'is', 'awesome', '.', '[SEP]']

On to the Next Chapter

Reimu
So with the Tokenizer I know how to get IDs. But how does the Model compute from those IDs?

Marisa
That's Chapter 4　Model — A Peek Inside, ZE. The difference between AutoModel and AutoModelForSequenceClassification, config.json and weight files, and a manual forward without pipeline.

Reimu
Time to open the black box, huh.

Marisa
Don't fear it. Through Chapter 3 you've got the shape of the input sorted. Next we just start by checking the shape of the output tensors.

Chapter 4　Model — A Peek Inside

4.1 Differences Among `AutoModel`, `AutoModelForCausalLM`, and Related Classes

Reimu
In Chapter 3 I got how the tokenizer builds input_ids. But who does the computation after that?

Marisa
The Model, ZE. Chapter 2's pipeline bundled "tokenizer → model → post-processing" internally. This chapter we touch the model itself.

Reimu
AutoModel showed up in Chapter 0's glossary — what's AutoModelForCausalLM?

Marisa
The For〜 in the name marks a task-specific output head. The Auto* family picks the right class by inferring architecture from the Hub's config.json.

Class example	Typical task	Output shape
`AutoModel`	Feature extraction, embeddings	Hidden states per token
`AutoModelForSequenceClassification`	Text classification	Per-class logits (raw scores)
`AutoModelForCausalLM`	Text generation (GPT family)	Next-token prediction
`AutoModelForMaskedLM`	Fill-in-the-blank (BERT family)	Candidate words at masked positions

Reimu
So classification and generation use different classes. What if you pick wrong?

Marisa
Instant errors: no generate(), wrong output shape, training head mismatch, and so on. For fine-tuning in Chapter 6, ForSequenceClassification is the usual pick.

from transformers import (
    AutoModel,
    AutoModelForSequenceClassification,
    AutoModelForCausalLM,
)

# Just want features
emb_model = AutoModel.from_pretrained("distilbert-base-uncased")

# Want sentiment labels
cls_model = AutoModelForSequenceClassification.from_pretrained(
    "distilbert-base-uncased-finetuned-sst-2-english"
)

# Want continued text
gen_model = AutoModelForCausalLM.from_pretrained("gpt2")

4.2 Configuration (`config.json`) and Weights (`.safetensors` / `.bin`)

Reimu
When you download from the Hub there are tons of files… which ones are the actual model?

Marisa
Roughly blueprint + weights, two sets, ZE.

File	Role
`config.json`	Structure: layer count, hidden size, vocab size, label count, etc.
`model.safetensors` (recommended) or `pytorch_model.bin`	Trained weights
`tokenizer.json`, etc.	Tokenizer from Chapter 3 (use as a set with the model)

Reimu
Can you read just the config?

Marisa
Yes. Looking inside tells you things like how many Transformer layers there are.

from transformers import AutoConfig

cfg = AutoConfig.from_pretrained("distilbert-base-uncased")
print(cfg.model_type)      # distilbert
print(cfg.hidden_size)     # 768
print(cfg.num_hidden_layers)

Weights load separately. The checkpoint from Chapter 1 is this weight-file save format.

from transformers import AutoModelForSequenceClassification

model = AutoModelForSequenceClassification.from_pretrained(
    "distilbert-base-uncased-finetuned-sst-2-english"
)
print(model.config.id2label)
# {0: 'NEGATIVE', 1: 'POSITIVE'}

Reimu
Even label names are in the config.

Marisa
Fine-tuned models often carry id2label / label2id. You use them to turn prediction IDs back into human-readable strings, ZE.

4.3 `device_map` and Choosing GPU vs CPU

Reimu
Can I do this chapter on a laptop with no GPU?

Marisa
Yes. Small DistilBERT or GPT-2 runs in seconds to tens of seconds on CPU. With a GPU, move with .to("cuda") or device_map="auto".

import torch
from transformers import AutoModelForSequenceClassification

device = "cuda" if torch.cuda.is_available() else "cpu"
print("using device:", device)

model = AutoModelForSequenceClassification.from_pretrained(
    "distilbert-base-uncased-finetuned-sst-2-english"
)
model = model.to(device)

device_map="auto" uses accelerate to place layers across GPU / CPU automatically. Common for large models and 8-bit quantization (Hands-on 4-C).

# For larger models (GPU recommended)
model = AutoModelForCausalLM.from_pretrained(
    "gpt2",
    device_map="auto",  # small gpt2 fits on one device
)

Reimu
The one where input tensors have to be on the same device or PyTorch yells at you?

Marisa
Exactly. Chapter 3's input_ids must match model.device. pipeline did this automatically; with manual forward you align it yourself.

4.4 Inference Mode (`model.eval()`) and Why You Disable Gradients

Reimu
For inference you call eval() — that's not training evaluate, right?

Marisa
In PyTorch, model.eval() is the inference-mode switch, ZE. It turns off Dropout, fixes BatchNorm statistics, and so on. The opposite of model.train() during training.

For inference you don't need gradients either. Wrap with torch.no_grad() to save memory and speed up.

import torch

model.eval()

inputs = tokenizer("Hello HF", return_tensors="pt")
with torch.no_grad():
    outputs = model(**inputs)

logits = outputs.logits

Situation	What to use
Inference, demos	`model.eval()` + `torch.no_grad()`
Fine-tuning (Chapter 6)	`model.train()` + gradients ON

Reimu
So in Chapter 6 we drop no_grad.

Marisa
Trainer handles the training loop, so you write less yourself. Just don't forget this two-part combo in inference scripts.

4.5 Basic Parameters for Generation (`generate`)

Reimu
For GPT-family models it's generate, not forward?

Marisa
Causal language models predict the next token repeatedly to extend text. The high-level API is model.generate().

Parameters you'll touch often:

Parameter	Meaning
`max_new_tokens`	Upper limit on newly added tokens (prompt length not included)
`do_sample`	`False` = greedy (max each step), `True` = stochastic sampling
`temperature`	Randomness when sampling (lower = more conservative)
`top_p`	Cumulative probability top-p (nucleus sampling)

output_ids = model.generate(
    input_ids,
    max_new_tokens=40,
    do_sample=True,
    temperature=0.8,
    top_p=0.95,
    pad_token_id=tokenizer.eos_token_id,
)
text = tokenizer.decode(output_ids[0], skip_special_tokens=True)

Reimu
How is that different from max_length?

Marisa
max_length caps input + output combined. Lately max_new_tokens is clearer, so this book prefers it, ZE.

4.6 Hands-on 4 — Working with the Model Outside the Pipeline

Reimu
Time for hand-written forward passes?

Marisa
Hands-on 4 for this chapter. Work through A to C in order and you connect straight to data prep in Chapter 5 → Trainer in Chapter 6.

Hands-on 4 — Manual Forward, Generation, and Quantization (Optional)

Exercise A Skip the pipeline; run a classification model yourself from logits → label

cd /path/to/yukkuri-hugging-face
source .venv/bin/activate
python scripts/ch04_forward.py

Example expected output:

input_ids shape: (1, 12)
attention_mask shape: (1, 12)

text: 'This course is surprisingly easy to follow!'
predicted label: POSITIVE (confidence=0.9998)
...

Reimu
outputs.logits showed up directly. So this is what the pipeline was doing behind the scenes.

Marisa
The flow is encode → model(inputs) → softmax → argmax**. The key is passing Chapter 3's tokenizer output to the model with **inputs.

Exercise B With GPT-2, compare generation while changing temperature, top_p, and max_new_tokens

python scripts/ch04_generate.py

Example expected output:

[greedy (default)]
Hugging Face lets you create your own models...
----------------------------------------
[temperature=0.9]
Hugging Face lets you build a custom model...

Same prompt, different sampling settings, different style. Tune this for demos and creative bots, ZE.

Exercise C (optional) Load with 8-bit quantization to save memory

pip install bitsandbytes accelerate
python scripts/ch04_quantize.py

With GPU and bitsandbytes in place, you can cut VRAM use. CPU-only often fails — read the message and skip if needed.

Reimu
I think 4-bit was in the table of contents too…

Marisa
Same idea — just switch to load_in_4bit=True. It comes back in QLoRA in Chapter 7; here 8-bit is enough to know quantized loading exists, ZE.

Full Chapter Scripts

`scripts/ch04_forward.py`

"""Chapter 4-A: Manual forward pass without pipeline."""

from __future__ import annotations

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer


def predict(text: str, model_id: str = "distilbert-base-uncased-finetuned-sst-2-english") -> None:
    tokenizer = AutoTokenizer.from_pretrained(model_id)
    model = AutoModelForSequenceClassification.from_pretrained(model_id)
    model.eval()

    inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)
    print("input_ids shape:", tuple(inputs["input_ids"].shape))
    print("attention_mask shape:", tuple(inputs["attention_mask"].shape))

    with torch.no_grad():
        outputs = model(**inputs)

    logits = outputs.logits
    probs = torch.softmax(logits, dim=-1)
    pred_id = int(probs.argmax(dim=-1).item())
    label = model.config.id2label[pred_id]
    confidence = float(probs[0, pred_id])

    print(f"\ntext: {text!r}")
    print(f"predicted label: {label} (confidence={confidence:.4f})")
    print("all logits:", [round(x, 4) for x in logits[0].tolist()])


def main() -> None:
    samples = [
        "This course is surprisingly easy to follow!",
        "I could not understand a single word.",
    ]
    for text in samples:
        predict(text)
        print("-" * 40)


if __name__ == "__main__":
    main()

`scripts/ch04_generate.py`

"""Chapter 4-B: Experiment with model.generate() parameters."""

from __future__ import annotations

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer


def generate_once(
    model,
    tokenizer,
    prompt: str,
    *,
    max_new_tokens: int = 40,
    temperature: float = 1.0,
    top_p: float = 1.0,
    do_sample: bool = False,
) -> str:
    inputs = tokenizer(prompt, return_tensors="pt")
    input_ids = inputs["input_ids"].to(model.device)
    attention_mask = inputs["attention_mask"].to(model.device)

    gen_kwargs: dict = {
        "max_new_tokens": max_new_tokens,
        "pad_token_id": tokenizer.eos_token_id,
    }
    if do_sample:
        gen_kwargs.update(
            {
                "do_sample": True,
                "temperature": temperature,
                "top_p": top_p,
            }
        )

    with torch.no_grad():
        output_ids = model.generate(input_ids, attention_mask=attention_mask, **gen_kwargs)

    new_tokens = output_ids[0, input_ids.shape[1] :]
    continuation = tokenizer.decode(new_tokens, skip_special_tokens=True)
    return prompt + continuation


def main() -> None:
    model_id = "gpt2"
    prompt = "Hugging Face lets you"

    tokenizer = AutoTokenizer.from_pretrained(model_id)
    model = AutoModelForCausalLM.from_pretrained(model_id)
    model.eval()

    settings = [
        {"label": "greedy (default)", "do_sample": False, "max_new_tokens": 30},
        {"label": "temperature=0.9", "do_sample": True, "temperature": 0.9, "top_p": 1.0, "max_new_tokens": 30},
        {"label": "top_p=0.9", "do_sample": True, "temperature": 1.0, "top_p": 0.9, "max_new_tokens": 30},
        {"label": "longer (max_new_tokens=80)", "do_sample": True, "temperature": 0.8, "top_p": 0.95, "max_new_tokens": 80},
    ]

    for cfg in settings:
        label = cfg.pop("label")
        text = generate_once(model, tokenizer, prompt, **cfg)
        print(f"[{label}]")
        print(text)
        print("-" * 40)


if __name__ == "__main__":
    main()

`scripts/ch04_quantize.py`

"""Chapter 4-C (optional): Load a model in 8-bit to save memory."""

from __future__ import annotations

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig


def load_model_8bit(model_id: str = "gpt2"):
    bnb_config = BitsAndBytesConfig(load_in_8bit=True)
    tokenizer = AutoTokenizer.from_pretrained(model_id)
    model = AutoModelForCausalLM.from_pretrained(
        model_id,
        quantization_config=bnb_config,
        device_map="auto",
    )
    return tokenizer, model


def main() -> None:
    model_id = "gpt2"
    prompt = "Quantized models use less VRAM when"

    try:
        tokenizer, model = load_model_8bit(model_id)
    except ImportError as exc:
        print("bitsandbytes is not installed. Install with:")
        print("  pip install bitsandbytes accelerate")
        print(f"detail: {exc}")
        return
    except Exception as exc:  # noqa: BLE001 - demo script
        print("8-bit loading failed on this machine (CPU-only env is common).")
        print(f"detail: {exc}")
        return

    inputs = tokenizer(prompt, return_tensors="pt")
    input_ids = inputs["input_ids"].to(model.device)

    with torch.no_grad():
        out = model.generate(input_ids, max_new_tokens=25, pad_token_id=tokenizer.eos_token_id)

    text = tokenizer.decode(out[0], skip_special_tokens=True)
    print("model dtype / device:", next(model.parameters()).dtype, next(model.parameters()).device)
    print("generated:", text)


if __name__ == "__main__":
    main()

Reimu's Notebook

Pick the task head with AutoModelFor〜. Classification: ForSequenceClassification; generation: ForCausalLM.
A model is config.json (structure) + weight files. For inference use eval() + no_grad().
For generation, tune generate's max_new_tokens / temperature / top_p.

Marisa's One More Thing

With output_hidden_states=True you get intermediate representations from each layer. That's an entry point for embedding visualization and LLM interpretation, ZE.

outputs = model(**inputs, output_hidden_states=True)
last_hidden = outputs.hidden_states[-1]  # (batch, seq, hidden)
print(last_hidden.shape)

On to the Next Chapter

Marisa
Model and tokenizer are in place. Next up: data for training and evaluation. In Chapter 5 we'll build Datasets from the Hub or CSV with datasets.

Reimu
Prep the data before fine-tuning. Like the Chapter 3 map tokenization thing, but the real-deal version?

Marisa
That understanding is spot on, ZE. Next time: Chapter 5　Datasets — Preparing and Preprocessing Data.

Chapter 5　Datasets — Preparing and Preprocessing Data

5.1 The Role of the `datasets` Library

Reimu
We touched the model itself in Chapter 4. But to fine-tune, you need your own data, right?

Marisa
Exactly, ZE. Hugging Face datasets is a library for handling training data in a unified format. You can load from the Hub or from a local CSV and put everything into the same Dataset type.

┌──────────────┐     load_dataset      ┌─────────────┐
│  Hub / CSV   │ ───────────────────► │  Dataset    │
└──────────────┘                       │  (row=ex.)  │
                                       └──────┬──────┘
                                              │ map / filter
                                       ┌──────▼──────┐
                                       │ preprocessed │
                                       │ input_ids…  │
                                       └─────────────┘

Reimu
Can't we just use a pandas DataFrame?

Marisa
A DataFrame works too, but Dataset is often easier for memory efficiency, caching, and Trainer integration. The Trainer in Chapter 6 accepts datasets objects as-is.

Required packages (on the Chapter 0 venv):

pip install datasets pandas
python -c "import datasets; print(datasets.__version__)"

5.2 Loading a Dataset from the Hub

Reimu
The Hub has datasets too, not just models, right?

Marisa
It does, ZE. Paper datasets and benchmarks are published there. The basics are a single line with load_dataset.

from datasets import load_dataset

ds = load_dataset("ag_news")
print(ds)
# DatasetDict({'train': ..., 'test': ...})

Key	Meaning
`train`	Training split
`test` / `validation`	Evaluation split (depends on the dataset)
Row	An Example like a dict `{ "text": "...", "label": 1 }`

Reimu
How do we peek inside?

Marisa
Use indexing, select, and features.

print(ds["train"].features)
print(ds["train"][0])
print(ds["train"].select(range(3)))

The first run downloads from the Hub, so you need a network connection. After that, reads come from the cache (see 5.5).

5.3 `map` / `filter` / `train_test_split`

Reimu
When you want the same processing on every row, do you use a for loop?

Marisa
Use map, ZE. When we touched the tokenizer in Chapter 3, the real workflow applies map to the whole dataset.

def add_prefix(example):
    example["text"] = "[NEWS] " + example["text"]
    return example

ds = ds.map(add_prefix)

Drop unwanted rows with filter:

ds = ds.filter(lambda ex: len(ex["text"]) > 20)

Split into training and validation with train_test_split:

small = ds["train"].shuffle(seed=42).select(range(1000))
split = small.train_test_split(test_size=0.1, seed=42)
train_ds = split["train"]
eval_ds = split["test"]

Reimu
batched=True showed up in Chapter 3 too.

Marisa
Tokenization is faster in batches. We'll cover that together in the next section.

5.4 Building a Tokenized Dataset

Reimu
Chapter 3's tokenizer(...) on every row…?

Marisa
Define a function and run map(batched=True). The model from Chapter 4 wants input_ids and attention_mask.

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased")

def tokenize_batch(examples):
    return tokenizer(
        examples["text"],
        truncation=True,
        max_length=128,
        padding=False,  # leave padding to the DataCollator (Chapter 6)
    )

tokenized = ds["train"].map(tokenize_batch, batched=True, remove_columns=["text"])
print(tokenized.column_names)
# ['label', 'input_ids', 'attention_mask']

Design choice	Reason
`truncation=True`	Truncate long text (Chapter 3)
`padding=False` in map	Keep variable lengths; pad at training time in batches
`remove_columns=["text"]`	Drop raw text columns to save memory

Reimu
We keep label, huh.

Marisa
The Trainer uses it as the ground-truth label. Chapter 6's compute_metrics hooks in here too, ZE.

5.5 Tips for Large Data (Streaming, Caching)

Reimu
AG News seems fine, but what about huge datasets?

Marisa
Two weapons.

1. Streaming — read as an iterator without downloading everything:

stream = load_dataset("large_corpus", split="train", streaming=True)
for i, row in enumerate(stream):
    if i >= 3:
        break
    print(row)

2. Caching — save map results to disk. The second run is faster:

tokenized = ds.map(tokenize_batch, batched=True, load_from_cache_file=True)

You can change the cache location with HF_HOME / HF_DATASETS_CACHE (see Chapter 0's one more thing).

Reimu
I've run out of disk on Colab before…

Marisa
Using a subset with select(range(N)) is the book's standard move. Chapter 6's fine-tuning also starts with a few hundred examples, ZE.

5.6 Hands-on 5 — Load, Preprocess, and Convert CSV

Reimu
A lot of the time our own data is only in CSV, right?

Marisa
In Hands-on 5 we go Hub → preprocessing → CSV. Exercise C also connects to Chapter 6's --dataset local_csv.

Hands-on 5 — Create and Explore a Dataset

Exercise A Load the public dataset ag_news and show splits, the first row, and label names

cd /path/to/yukkuri-hugging-face
source .venv/bin/activate
python scripts/ch05_load_dataset.py

Example expected output:

dataset: ag_news
splits: ['train', 'test']
num_train: 120000
label=2 text='Wall St. Bears Claw Back ...'
label names: ['World', 'Sports', 'Business', 'Sci/Tech']

Exercise B Add label names with map → tokenize → split a small subset

python scripts/ch05_preprocess.py

columns after preprocess: ['label', 'label_name', 'input_ids', 'attention_mask']
small subset split sizes: {'train': 160, 'test': 40}

The preprocessing functions are collected in ch05_preprocess.py at the end of this chapter ("Full Chapter Scripts"). Chapter 6 passes the same flow to the Trainer.

Exercise C Convert the bundled CSV into a DatasetDict

python scripts/ch05_csv_to_dataset.py

Sample CSV location:

scripts/data/ch05_sample_reviews.csv

Example contents:

text,label
"This tutorial is clear and fun.",positive
"I got lost in the first section.",negative
...

Reimu
Only six rows, but the shape matches production.

Marisa
Dataset.from_pandas → train_test_split → align column names for the Trainer. Your own business CSV works with the same code as long as you have text / label columns, ZE.

Customization example (add more label names):

# edit label2id in ch05_csv_to_dataset.py to match your labels
label2id = {"negative": 0, "positive": 1, "neutral": 2}

Full Chapter Scripts

Marisa
So you can practice even without the repo, this chapter includes the complete scripts as-is. Save them under scripts/ with the same filenames and run them, ZE. (Where you see python scripts/xxx.py, adjust the path to match where you saved the file.)

`scripts/ch05_load_dataset.py`

"""Chapter 5-A: Load a public dataset and inspect it."""

from __future__ import annotations

from datasets import load_dataset


def main() -> None:
    dataset_name = "ag_news"
    ds = load_dataset(dataset_name)

    print("dataset:", dataset_name)
    print("splits:", list(ds.keys()))
    print("features:", ds["train"].features)
    print("num_train:", len(ds["train"]))
    print("num_test:", len(ds["test"]))

    print("\n--- first 3 rows (train) ---")
    for row in ds["train"].select(range(3)):
        print(f"label={row['label']} text={row['text'][:80]!r}...")

    label_names = ds["train"].features["label"].names
    print("\nlabel names:", label_names)


if __name__ == "__main__":
    main()

`scripts/ch05_preprocess.py`

"""Chapter 5-B: Build a reusable preprocessing pipeline with map()."""

from __future__ import annotations

from datasets import load_dataset
from transformers import AutoTokenizer


MODEL_ID = "distilbert-base-uncased"


def tokenize_batch(examples, tokenizer, max_length: int = 128):
    return tokenizer(
        examples["text"],
        truncation=True,
        max_length=max_length,
        padding=False,
    )


def add_label_names(example, label_names):
    example["label_name"] = label_names[example["label"]]
    return example


def main() -> None:
    ds = load_dataset("ag_news")
    label_names = ds["train"].features["label"].names

    tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)

    ds = ds.map(add_label_names, fn_kwargs={"label_names": label_names})
    tokenized = ds.map(
        lambda batch: tokenize_batch(batch, tokenizer),
        batched=True,
        remove_columns=["text"],
        desc="tokenizing",
    )

    print("columns after preprocess:", tokenized["train"].column_names)
    print("sample row:")
    row = tokenized["train"][0]
    print("  label:", row["label"], "->", row["label_name"])
    print("  input_ids length:", len(row["input_ids"]))
    print("  input_ids head:", row["input_ids"][:12])

    small = tokenized["train"].shuffle(seed=42).select(range(200))
    split = small.train_test_split(test_size=0.2, seed=42)
    print("\nsmall subset split sizes:", {k: len(v) for k, v in split.items()})


if __name__ == "__main__":
    main()

`scripts/ch05_csv_to_dataset.py`

"""Chapter 5-C: Convert a local CSV file into a Hugging Face Dataset."""

from __future__ import annotations

from pathlib import Path

import pandas as pd
from datasets import Dataset, DatasetDict


def csv_to_dataset(csv_path: Path) -> DatasetDict:
    df = pd.read_csv(csv_path)
    required = {"text", "label"}
    missing = required - set(df.columns)
    if missing:
        raise ValueError(f"CSV must contain columns: {sorted(required)}. Missing: {sorted(missing)}")

    label2id = {"negative": 0, "positive": 1}
    df["label_id"] = df["label"].map(label2id)
    if df["label_id"].isna().any():
        bad = df[df["label_id"].isna()]["label"].unique().tolist()
        raise ValueError(f"Unknown labels in CSV: {bad}")

    dataset = Dataset.from_pandas(df[["text", "label_id"]], preserve_index=False)
    dataset = dataset.rename_column("label_id", "label")
    split = dataset.train_test_split(test_size=0.34, seed=42, stratify_by_column="label")
    return DatasetDict({"train": split["train"], "test": split["test"]})


def main() -> None:
    repo_root = Path(__file__).resolve().parents[1]
    csv_path = repo_root / "scripts" / "data" / "ch05_sample_reviews.csv"

    ds = csv_to_dataset(csv_path)
    print("loaded from:", csv_path)
    print("splits:", {name: len(split) for name, split in ds.items()})
    print("features:", ds["train"].features)
    print("\ntrain rows:")
    for row in ds["train"]:
        label_name = "positive" if row["label"] == 1 else "negative"
        print(f"  [{label_name}] {row['text']}")


if __name__ == "__main__":
    main()

`scripts/data/ch05_sample_reviews.csv` (data file)

text,label
"This tutorial is clear and fun.",positive
"I got lost in the first section.",negative
"The examples run on my laptop without GPU.",positive
"Too many errors on Windows.",negative
"Reimu and Marisa make HF less scary.",positive
"I still do not know what logits are.",negative

Reimu's Notebook

Use load_dataset to get a split Dataset from the Hub. Each row is an Example as a dict.
Preprocess with map / filter / train_test_split. For tokenization, batched=True is the default.
For huge data, use streaming and caching. This book's fine-tuning starts with a subset first.

Marisa's One More Thing

When you need to combine multiple CSV or JSONL files, concatenate_datasets is handy.

from datasets import concatenate_datasets, load_dataset

a = load_dataset("csv", data_files="part_a.csv")["train"]
b = load_dataset("csv", data_files="part_b.csv")["train"]
merged = concatenate_datasets([a, b])
print(len(merged))

On to the Next Chapter

Marisa
The data and tokenized shapes are aligned. In Chapter 6 we pass them to Trainer for fine-tuning.

Reimu
Finally the training loop… Even without a GPU, we can start small like Chapter 0 said?

Marisa
With 800 examples and 1 epoch, a CPU demo in under an hour is realistic, ZE. Next up: Chapter 6　Fine-Tuning with Trainer.

Chapter 6　Fine-Tuning with Trainer

6.1 What Is Fine-Tuning For?

Reimu
Hub models work out of the box—why bother training again?

Marisa
General models are broad, but they can be weak on your domain, phrasing, and label definitions. Fine-tuning (FT) updates the weights on a small amount of data to fit the task, ZE.

Approach	What gets updated	Chapter in this book
Full-parameter FT	Entire model	Chapter 6 (this chapter)
LoRA / QLoRA	A tiny adapter	Chapter 7

Reimu
So it's like raising Chapter 4's ForSequenceClassification on Chapter 5's Dataset?

Marisa
Exactly. This chapter fine-tunes DistilBERT on English news 4-way classification (ag_news). CSV fans can try --dataset local_csv too (few rows, so it's demo-sized).

6.2 Key `TrainingArguments` Options

Reimu
We don't have to write the training loop ourselves?

Marisa
The Transformers way is Trainer + TrainingArguments. Here's a table of the options you'll touch most, ZE.

Option	Meaning
`output_dir`	Where checkpoints and logs are saved
`learning_rate`	Learning rate (e.g. `5e-5`)
`per_device_train_batch_size`	Batch size per GPU/CPU
`num_train_epochs`	How many passes over the full data
`evaluation_strategy`	When to evaluate, e.g. `"epoch"`
`save_strategy`	When to save checkpoints
`logging_steps`	Log loss every N steps
`report_to`	`"tensorboard"` / `"wandb"`, etc.
`push_to_hub`	Upload to the Hub after training

from transformers import TrainingArguments

args = TrainingArguments(
    output_dir="log/ch06_finetune",
    learning_rate=5e-5,
    per_device_train_batch_size=8,
    num_train_epochs=1,
    evaluation_strategy="epoch",
    save_strategy="epoch",
    logging_steps=20,
    report_to=["tensorboard"],
)

Reimu
Bigger batch size means faster?

Marisa
Generally you process more per step. But if you exceed GPU memory, you get OOM. That comes back in 6.6.

6.3 Basic `Trainer` Flow (Train → Evaluate → Save)

Reimu
What do we pass to the Trainer?

Marisa
At minimum, this.

from transformers import Trainer, DataCollatorWithPadding

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=tokenized_train,
    eval_dataset=tokenized_eval,
    tokenizer=tokenizer,
    data_collator=DataCollatorWithPadding(tokenizer),
    compute_metrics=compute_metrics,
)

trainer.train()
metrics = trainer.evaluate()
trainer.save_model("log/ch06_finetune/final")

Flow:

train_dataset ──► train() ──► checkpoint
                     │
eval_dataset  ──► evaluate() ──► accuracy, etc.
                     │
                     └──► save_model / push_to_hub

Reimu
DataCollatorWithPadding is the Chapter 3 topic.

Marisa
It pads variable-length input_ids within a batch. The standard pattern is no padding in map; let the Collator handle it, ZE.

6.4 Evaluation Metrics (accuracy, F1, perplexity, etc.)

Reimu
Isn't loss enough?

Marisa
Loss is for optimization. Humans read accuracy and F1 more easily. Pass a function to compute_metrics.

import numpy as np

def compute_metrics(eval_pred):
    logits, labels = eval_pred
    preds = np.argmax(logits, axis=-1)
    accuracy = (preds == labels).mean().item()
    return {"accuracy": accuracy}

Task	Common metrics
Classification	accuracy, F1, precision, recall
Generation (LM)	perplexity, BLEU, etc.
Regression	MSE, MAE

This chapter's demo uses accuracy only. In production, add F1 with sklearn.metrics.

6.5 Checkpoint Saving and Pushing to the Hub

Reimu
Folders keep piling up in output_dir…

Marisa
You get mid-run saves like checkpoint-500. With load_best_model_at_end=True, the best eval weights stay at the end.

Publishing to the Hub requires being logged in (Chapter 1). Manage tokens via .env or huggingface-cli login—don't put them in code.

huggingface-cli login
# or put HF_TOKEN=... in .env and let huggingface_hub read it

Push while training:

python scripts/ch06_finetune.py \
  --push-to-hub \
  --hub-model-id YOUR_USERNAME/yukkuri-distilbert-ag-news-demo

Upload only an already saved final/:

python scripts/ch06_push_hub.py \
  --hub-model-id YOUR_USERNAME/yukkuri-distilbert-ag-news-demo

Reimu
Replace YOUR_USERNAME with yours.

Marisa
Obviously, ZE. For a private repo, you can add --private.

6.6 Overfitting, Learning Rate, and Batch Size

Reimu
100% accuracy! Perfect?

Marisa
Celebrating on training data alone is risky. If eval stalls while train keeps improving, suspect overfitting.

Symptom	Things to try
train loss ↓, eval worsens	fewer epochs, more data, weight decay
loss oscillates	lower learning rate (`5e-5` → `2e-5`)
CUDA OOM	halve batch size, shorten `max_length`
training is slow	GPU, smaller model (DistilBERT), subset of data

Reimu
Does Chapter 7's LoRA help with memory too?

Marisa
For large LLMs, LoRA is the main act. This chapter is small full FT to get a feel for the Trainer, ZE.

6.7 Hands-on 6 — Fine-Tuning, Visualization, and Hub Publishing

Reimu
The day we finally press trainer.train()…

Marisa
Hands-on 6. A trains, B checks logs, C goes to the Hub. The first run takes a while because of model download.

Hands-on 6 — Trainer in Practice

Exercise A Fine-tune DistilBERT for 1 epoch on a subset of ag_news

cd /path/to/yukkuri-hugging-face
source .venv/bin/activate
pip install tensorboard scikit-learn

python scripts/ch06_finetune.py \
  --max-train 800 \
  --max-eval 200 \
  --epochs 1 \
  --batch-size 8 \
  --output-dir log/ch06_finetune

Example expected output:

Starting fine-tuning...
{'loss': 0.85, 'epoch': 0.5}
...
train loss: 0.42
eval metrics: {'eval_accuracy': 0.89, ...}
saved to: log/ch06_finetune/final

After training, run inference with the saved model:

from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch

model_dir = "log/ch06_finetune/final"
tokenizer = AutoTokenizer.from_pretrained(model_dir)
model = AutoModelForSequenceClassification.from_pretrained(model_dir)
model.eval()

text = "Stock markets rally after tech earnings beat expectations."
inputs = tokenizer(text, return_tensors="pt")
with torch.no_grad():
    pred = model(**inputs).logits.argmax(-1).item()
print(model.config.id2label[pred])  # Business, etc.

Exercise B View learning curves with TensorBoard (or W&B)

TensorBoard:

tensorboard --logdir log/ch06_finetune
# open http://localhost:6006 in a browser

If using W&B (optional):

pip install wandb
export WANDB_API_KEY="your_wandb_key"   # can also go in .env

python scripts/ch06_finetune.py \
  --report-to wandb \
  --output-dir log/ch06_wandb

Without WANDB_API_KEY, the script falls back to tensorboard.

Reimu
If the loss line goes down, that's success for now?

Marisa
In production you watch eval accuracy too. For the demo, reading the trend after 1 epoch is enough, ZE.

Exercise C Publish the trained model to the Hub (login required)

huggingface-cli login

python scripts/ch06_finetune.py \
  --max-train 800 \
  --max-eval 200 \
  --epochs 1 \
  --push-to-hub \
  --hub-model-id YOUR_USERNAME/yukkuri-distilbert-ag-news-demo

Or from a saved final/:

python scripts/ch06_push_hub.py \
  --hub-model-id YOUR_USERNAME/yukkuri-distilbert-ag-news-demo

After publishing, a short Model Card on the Hub is kind (Chapter 1's reading skills pay off).

Try with a local CSV (few rows, easy to overfit—demo only):

python scripts/ch06_finetune.py --dataset local_csv --epochs 3 --batch-size 2

Full Chapter Scripts

`scripts/ch06_finetune.py`

"""Chapter 6-A/B: Fine-tune a text classifier with Trainer."""

from __future__ import annotations

import argparse
import os
from pathlib import Path

import numpy as np
from datasets import DatasetDict, load_dataset
from transformers import (
    AutoModelForSequenceClassification,
    AutoTokenizer,
    DataCollatorWithPadding,
    Trainer,
    TrainingArguments,
    set_seed,
)


MODEL_ID = "distilbert-base-uncased"
DEFAULT_DATASET = "ag_news"


def build_datasets(dataset_name: str, max_train: int, max_eval: int) -> tuple[DatasetDict, list[str]]:
    if dataset_name == "local_csv":
        import sys

        scripts_dir = Path(__file__).resolve().parent
        if str(scripts_dir) not in sys.path:
            sys.path.insert(0, str(scripts_dir))
        from ch05_csv_to_dataset import csv_to_dataset

        repo_root = Path(__file__).resolve().parents[1]
        csv_path = repo_root / "scripts" / "data" / "ch05_sample_reviews.csv"
        raw = csv_to_dataset(csv_path)
        label_names = ["negative", "positive"]
    else:
        raw = load_dataset(dataset_name)
        label_names = raw["train"].features["label"].names

    train = raw["train"].shuffle(seed=42).select(range(min(max_train, len(raw["train"]))))
    eval_split = raw["test"] if "test" in raw else raw["validation"]
    eval_ds = eval_split.shuffle(seed=42).select(range(min(max_eval, len(eval_split))))
    return DatasetDict(train=train, eval=eval_ds), label_names


def tokenize_dataset(ds: DatasetDict, tokenizer) -> DatasetDict:
    text_column = "text"

    def preprocess(batch):
        return tokenizer(batch[text_column], truncation=True)

    tokenized = {}
    for split_name, split_ds in ds.items():
        tokenized[split_name] = split_ds.map(
            preprocess,
            batched=True,
            remove_columns=[text_column],
        )
    return DatasetDict(tokenized)


def compute_metrics(eval_pred):
    logits, labels = eval_pred
    preds = np.argmax(logits, axis=-1)
    accuracy = (preds == labels).mean().item()
    return {"accuracy": accuracy}


def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(description="Chapter 6 fine-tuning demo")
    parser.add_argument("--dataset", default=DEFAULT_DATASET, help="HF dataset name or local_csv")
    parser.add_argument("--max-train", type=int, default=800, help="subset size for quick runs")
    parser.add_argument("--max-eval", type=int, default=200)
    parser.add_argument("--epochs", type=int, default=1)
    parser.add_argument("--batch-size", type=int, default=8)
    parser.add_argument("--lr", type=float, default=5e-5)
    parser.add_argument("--output-dir", default="log/ch06_finetune")
    parser.add_argument("--report-to", default="tensorboard", choices=["tensorboard", "wandb", "none"])
    parser.add_argument("--push-to-hub", action="store_true", help="requires HF login and --hub-model-id")
    parser.add_argument("--hub-model-id", default="", help="e.g. yourname/yukkuri-distilbert-ag-news-demo")
    return parser.parse_args()


def main() -> None:
    args = parse_args()
    set_seed(42)

    raw, label_names = build_datasets(args.dataset, args.max_train, args.max_eval)
    num_labels = len(label_names)

    tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
    tokenized = tokenize_dataset(raw, tokenizer)

    model = AutoModelForSequenceClassification.from_pretrained(
        MODEL_ID,
        num_labels=num_labels,
        id2label={i: name for i, name in enumerate(label_names)},
        label2id={name: i for i, name in enumerate(label_names)},
    )

    report_to = [] if args.report_to == "none" else [args.report_to]
    if args.report_to == "wandb" and not os.getenv("WANDB_API_KEY"):
        print("WANDB_API_KEY is not set. Falling back to tensorboard.")
        report_to = ["tensorboard"]

    output_dir = Path(args.output_dir)
    output_dir.mkdir(parents=True, exist_ok=True)

    training_args = TrainingArguments(
        output_dir=str(output_dir),
        learning_rate=args.lr,
        per_device_train_batch_size=args.batch_size,
        per_device_eval_batch_size=args.batch_size,
        num_train_epochs=args.epochs,
        evaluation_strategy="epoch",
        save_strategy="epoch",
        logging_steps=20,
        report_to=report_to,
        load_best_model_at_end=True,
        metric_for_best_model="accuracy",
        push_to_hub=args.push_to_hub,
        hub_model_id=args.hub_model_id or None,
    )

    trainer = Trainer(
        model=model,
        args=training_args,
        train_dataset=tokenized["train"],
        eval_dataset=tokenized["eval"],
        tokenizer=tokenizer,
        data_collator=DataCollatorWithPadding(tokenizer),
        compute_metrics=compute_metrics,
    )

    print("Starting fine-tuning...")
    train_result = trainer.train()
    metrics = trainer.evaluate()
    print("train loss:", round(train_result.training_loss, 4))
    print("eval metrics:", {k: round(v, 4) if isinstance(v, float) else v for k, v in metrics.items()})

    save_dir = output_dir / "final"
    trainer.save_model(save_dir)
    tokenizer.save_pretrained(save_dir)
    print("saved to:", save_dir)

    if args.push_to_hub:
        if not args.hub_model_id:
            raise ValueError("--push-to-hub requires --hub-model-id")
        trainer.push_to_hub()
        print("pushed to Hub:", args.hub_model_id)


if __name__ == "__main__":
    main()

`scripts/ch06_push_hub.py`

"""Chapter 6-C helper: Push an already fine-tuned local checkpoint to the Hub."""

from __future__ import annotations

import argparse
from pathlib import Path

from transformers import AutoModelForSequenceClassification, AutoTokenizer


def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(description="Upload a local fine-tuned checkpoint to HF Hub")
    parser.add_argument(
        "--checkpoint-dir",
        default="log/ch06_finetune/final",
        help="directory saved by ch06_finetune.py",
    )
    parser.add_argument(
        "--hub-model-id",
        required=True,
        help="target repo id, e.g. yourname/yukkuri-distilbert-demo",
    )
    parser.add_argument("--private", action="store_true", help="create/use a private repo")
    return parser.parse_args()


def main() -> None:
    args = parse_args()
    checkpoint = Path(args.checkpoint_dir)
    if not checkpoint.exists():
        raise FileNotFoundError(
            f"Checkpoint not found: {checkpoint}. Run ch06_finetune.py first."
        )

    model = AutoModelForSequenceClassification.from_pretrained(checkpoint)
    tokenizer = AutoTokenizer.from_pretrained(checkpoint)

    model.push_to_hub(args.hub_model_id, private=args.private)
    tokenizer.push_to_hub(args.hub_model_id, private=args.private)
    print("upload complete:", f"https://huggingface.co/{args.hub_model_id}")


if __name__ == "__main__":
    main()

Reimu's Notebook

FT is extra training to adapt a general model to your task. This chapter uses Trainer + full parameters.
Set LR, batch, epoch, and log destination in TrainingArguments; return accuracy etc. from compute_metrics.
Share with save_model / push_to_hub. Judge overfitting by watching both train and eval.

Marisa's One More Thing

Add EarlyStoppingCallback to the Trainer to stop when eval stops improving.

from transformers import EarlyStoppingCallback

trainer = Trainer(
    ...,
    callbacks=[EarlyStoppingCallback(early_stopping_patience=2)],
)

Pair it with load_best_model_at_end=True to cut wasted epochs, ZE.

On to the Next Chapter

Marisa
This chapter covered data → Trainer → save end to end. Chapter 7 goes down the LoRA path to fine-tune big models lightly.

Reimu
Updating every parameter really squeezed the GPU…

Marisa
With peft you train only adapters and slash memory and time. Next: Chapter 7　PEFT / LoRA — Fine-Tuning with Less GPU.

Chapter 7　PEFT / LoRA — Fine-Tuning with Less GPU

7.1 Full-Parameter Updates vs Partial Updates

Reimu
We ran Trainer in Chapter 6, but GPU memory was tight… Is updating every weight really that heavy?

Marisa
Chapter 6's full fine-tuning (Full FT) sends gradients through every parameter. For 7B-class LLMs, weights + gradients + optimizer state can hit tens of GB, ZE.

Reimu
Even for small models like our distil family?

Marisa
Smaller helps, but "update everything" is the same idea. Remember the flow like this.

Method	What gets updated	Memory	Typical use
Full FT (Chapter 6)	All layers	Large	Enough data; want max performance
Partial update / PEFT (this chapter)	Adapters only	Small	Limited GPU; switch between tasks
Inference only (Chapter 8)	None	Minimal	Production serving

Chapter 6 Trainer (Full FT)
        │
        ▼  "Too heavy… where's LoRA?"
Chapter 7 PEFT / LoRA  ← you are here
        │
        ▼
Chapter 8 inference optimization

Reimu
Partial update is like tweaking just the edges of the model?

Marisa
Right. PEFT (Parameter-Efficient Fine-Tuning) is the umbrella for methods that keep trainable parameters to 1–several %, ZE. This chapter's star is LoRA (Low-Rank Adaptation).

7.2 How LoRA / QLoRA Work

Reimu
LoRA… low rank? Matrix stuff?

Marisa
Essence only. Add a small update ΔW to frozen weights W. Instead of storing ΔW as a huge matrix, approximate it with a low-rank factorization A × B.

Original weight W (frozen — not updated)
        +
LoRA update ΔW ≈ B @ A   (A, B are small rank=r matrices)
        =
Effective weight at inference W + ΔW

Term	Meaning
rank (r)	LoRA capacity. Larger → more capacity ↑, more memory ↑
lora_alpha	Update scale. Often `alpha / r` is the effective multiplier
target_modules	Layers that carry LoRA (e.g. Attention `q_proj`)
QLoRA	4-bit quantized base + LoRA. Cuts VRAM further

Reimu
Is QLoRA for Colab?

Marisa
It's the go-to when VRAM is tight, ZE. Load the base in 4-bit with bitsandbytes and train LoRA. This chapter's scripts use plain LoRA (FP16/FP32), but the idea is the same.

Minimal QLoRA sketch (reference, optional):

from transformers import AutoModelForCausalLM, BitsAndBytesConfig
from peft import LoraConfig, get_peft_model

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype="float16",
)

model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.2-1B",
    quantization_config=bnb_config,
    device_map="auto",
)
# then LoraConfig → get_peft_model (same flow as 7.3 in this chapter)

Reimu
Can we keep Chapter 6's TrainingArguments?

Marisa
Yes. Trainer + PEFT model is the standard combo, ZE. The only change is passing a model wrapped with get_peft_model.

7.3 Using the `peft` Library

Reimu
Just pip install peft?

Marisa
Add peft and accelerate to your Chapter 6 environment. The first run also downloads distilgpt2.

cd /path/to/yukkuri-hugging-face
source .venv/bin/activate

pip install "peft>=0.11" "accelerate>=0.30"

Minimal LoRA config pattern:

from peft import LoraConfig, TaskType, get_peft_model

lora_config = LoraConfig(
    task_type=TaskType.CAUSAL_LM,
    r=8,
    lora_alpha=16,
    lora_dropout=0.05,
    target_modules=["c_attn", "c_proj"],  # for distilgpt2
    bias="none",
)

model = get_peft_model(base_model, lora_config)
model.print_trainable_parameters()

Example expected output (numbers vary by model):

trainable params: 294,912 || all params: 82,316,544 || trainable%: 0.36

Reimu
Only 0.36%!

Marisa
That's why fine-tuning on less GPU is easier, ZE. target_modules changes per architecture. If unsure, check the Hub model card or PEFT examples.

7.4 Loading an Adapter on the Base Model for Inference

Reimu
After training, how do we load for inference?

Marisa
Two steps. Base → adapter, in that order.

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "distilgpt2"
adapter_dir = "log/ch07_lora_adapter/adapter"

tokenizer = AutoTokenizer.from_pretrained(adapter_dir)
base = AutoModelForCausalLM.from_pretrained(base_id)
model = PeftModel.from_pretrained(base, adapter_dir)

prompt = "Reimu: What is LoRA?\nMarisa:"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Artifact	Contents	Rough size
Base model	Full weights	~350 MB for distilgpt2
LoRA adapter only	A, B matrices + config	On the order of a few MB
Full FT checkpoint	Copy of all weights	About the same as the base

Reimu
For the Hub, the adapter alone is enough.

Marisa
Right, ZE. Put base_model_name_or_path pointing at the public base on the Hub in adapter_config.json, and readers can reproduce with base download + adapter download.

7.5 Hands-on 7 — Light Fine-Tuning with LoRA

Reimu
Finally a hands-on chapter?

Marisa
Hands-on 7. Put LoRA on causal LM distilgpt2 and train with the same Trainer style as Chapter 6.

Hands-on 7 — PEFT / LoRA in Three Parts

Exercise A Lightly fine-tune a causal language model with LoRA

python scripts/ch07_lora_train.py

Option examples (experiment with rank):

python scripts/ch07_lora_train.py --rank 16 --epochs 5 --output log/ch07_lora_r16

Example expected output:

Loading base model: distilgpt2
trainable params: ... || trainable%: 0.3x
Starting LoRA training...
Saved LoRA adapter to: log/ch07_lora_adapter/adapter
OK: Chapter 7-A LoRA training complete

Exercise B Save and share the adapter only

python scripts/ch07_lora_save_adapter.py

Check adapter metadata and sample generation; write shareable files to log/ch07_lora_adapter/export/.

Example expected output:

PeftConfig:
  peft_type: LORA
  base_model_name_or_path: distilgpt2
  r (rank): 8
  adapter size: 2.xx MB
Sample generation with adapter:
Reimu: What did we learn about LoRA?
Marisa: ...
OK: Chapter 7-B adapter save / inspect complete

To push to the Hub (token via .env or huggingface-cli login—never put real tokens in the chapter):

# example: replace with your username
huggingface-cli upload your-username/yukkuri-ch07-lora-demo log/ch07_lora_adapter/export

Exercise C Switch between multiple LoRA adapters and compare

# auto-create a second demo adapter and compare
python scripts/ch07_lora_switch.py --create-demo-b

Example expected output:

=== Adapter A ===
Reimu: Explain LoRA in one sentence.
Marisa: ...

=== Adapter B ===
Reimu: Explain LoRA in one sentence.
Marisa: ...
OK: Chapter 7-C adapter switch complete

Reimu
A and B give different text… so this is what changing rank does?

Marisa
Different data and rank change the output. In Chapter 10's capstone you'll switch and merge yukkuri-style LoRA adapters like this. First, get fixed base + swap adapters into muscle memory.

7.6 Common Errors

Reimu
Tell me the stumbling blocks upfront.

Marisa
Table time.

Error / symptom	Cause	Fix
`No module named 'peft'`	Not installed	`pip install peft`
KeyError on `target_modules`	Layer names don't match the model	Check the model's Linear layer names
CUDA OOM	rank / batch too large	`--rank 4`, lower batch size
Adapter not found	7-A not run	Run `ch07_lora_train.py` first
Generation same as base	Under-trained or not loaded	More epochs; verify PeftModel load

Full Chapter Scripts

`scripts/ch07_lora_train.py`

"""Chapter 7-A: LoRA fine-tuning for a small causal language model."""

from __future__ import annotations

import argparse
from pathlib import Path

from datasets import Dataset
from peft import LoraConfig, TaskType, get_peft_model
from transformers import (
    AutoModelForCausalLM,
    AutoTokenizer,
    DataCollatorForLanguageModeling,
    Trainer,
    TrainingArguments,
)

DEFAULT_MODEL = "distilgpt2"
DEFAULT_OUTPUT = "log/ch07_lora_adapter"


def build_tiny_dataset() -> Dataset:
    """Minimal instruction-style snippets for demo fine-tuning."""
    texts = [
        "Reimu: Hugging Face Hub is like GitHub for AI models.\n"
        "Marisa: Push your LoRA adapter and share it with the world!",
        "Reimu: LoRA trains only a small adapter, not the whole model.\n"
        "Marisa: That saves GPU memory and disk space, ze!",
        "Reimu: Can I fine-tune on my laptop?\n"
        "Marisa: With PEFT and a tiny model, yes you can!",
        "Reimu: What is a tokenizer again?\n"
        "Marisa: It turns text into token IDs the model understands.",
        "Reimu: Trainer saved a checkpoint last chapter.\n"
        "Marisa: This chapter we attach LoRA and train even lighter!",
    ]
    return Dataset.from_dict({"text": texts})


def tokenize_dataset(tokenizer: AutoTokenizer, dataset: Dataset) -> Dataset:
    def tokenize(batch: dict) -> dict:
        return tokenizer(
            batch["text"],
            truncation=True,
            max_length=128,
            padding="max_length",
        )

    return dataset.map(tokenize, batched=True, remove_columns=["text"])


def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(description="LoRA fine-tune distilgpt2")
    parser.add_argument("--model", default=DEFAULT_MODEL, help="Base model id")
    parser.add_argument(
        "--output",
        default=DEFAULT_OUTPUT,
        help="Directory to save LoRA adapter",
    )
    parser.add_argument("--epochs", type=int, default=3, help="Training epochs")
    parser.add_argument("--lr", type=float, default=2e-4, help="Learning rate")
    parser.add_argument(
        "--rank",
        type=int,
        default=8,
        help="LoRA rank (r). Higher = more capacity, more memory",
    )
    return parser.parse_args()


def main() -> None:
    args = parse_args()
    output_dir = Path(args.output)
    output_dir.mkdir(parents=True, exist_ok=True)

    print(f"Loading base model: {args.model}")
    tokenizer = AutoTokenizer.from_pretrained(args.model)
    if tokenizer.pad_token is None:
        tokenizer.pad_token = tokenizer.eos_token

    model = AutoModelForCausalLM.from_pretrained(args.model)

    lora_config = LoraConfig(
        task_type=TaskType.CAUSAL_LM,
        r=args.rank,
        lora_alpha=16,
        lora_dropout=0.05,
        target_modules=["c_attn", "c_proj"],
        bias="none",
    )
    model = get_peft_model(model, lora_config)
    model.print_trainable_parameters()

    dataset = tokenize_dataset(tokenizer, build_tiny_dataset())

    training_args = TrainingArguments(
        output_dir=str(output_dir / "checkpoints"),
        num_train_epochs=args.epochs,
        per_device_train_batch_size=2,
        learning_rate=args.lr,
        logging_steps=1,
        save_strategy="no",
        report_to="none",
        use_cpu=not __import__("torch").cuda.is_available(),
    )

    trainer = Trainer(
        model=model,
        args=training_args,
        train_dataset=dataset,
        data_collator=DataCollatorForLanguageModeling(
            tokenizer=tokenizer,
            mlm=False,
        ),
    )

    print("Starting LoRA training...")
    trainer.train()

    adapter_path = output_dir / "adapter"
    model.save_pretrained(adapter_path)
    tokenizer.save_pretrained(adapter_path)
    print(f"Saved LoRA adapter to: {adapter_path}")
    print("OK: Chapter 7-A LoRA training complete")


if __name__ == "__main__":
    main()

`scripts/ch07_lora_save_adapter.py`

"""Chapter 7-B: Save and inspect a LoRA adapter without the full base weights."""

from __future__ import annotations

import argparse
import json
from pathlib import Path

from peft import PeftConfig, PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

DEFAULT_BASE = "distilgpt2"
DEFAULT_ADAPTER = "log/ch07_lora_adapter/adapter"


def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(description="Save / inspect LoRA adapter")
    parser.add_argument("--base", default=DEFAULT_BASE, help="Base model id")
    parser.add_argument(
        "--adapter",
        default=DEFAULT_ADAPTER,
        help="Path to LoRA adapter directory",
    )
    parser.add_argument(
        "--export",
        default="log/ch07_lora_adapter/export",
        help="Directory to copy adapter metadata for sharing",
    )
    return parser.parse_args()


def adapter_size_mb(adapter_dir: Path) -> float:
    total = sum(f.stat().st_size for f in adapter_dir.rglob("*") if f.is_file())
    return total / (1024 * 1024)


def main() -> None:
    args = parse_args()
    adapter_dir = Path(args.adapter)
    export_dir = Path(args.export)

    if not adapter_dir.exists():
        raise SystemExit(
            f"Adapter not found: {adapter_dir}\n"
            "Run scripts/ch07_lora_train.py first (Chapter 7-A)."
        )

    config = PeftConfig.from_pretrained(adapter_dir)
    print("PeftConfig:")
    print(f"  peft_type: {config.peft_type}")
    print(f"  base_model_name_or_path: {config.base_model_name_or_path}")
    print(f"  r (rank): {config.r}")
    print(f"  target_modules: {config.target_modules}")
    print(f"  adapter size: {adapter_size_mb(adapter_dir):.2f} MB")

    tokenizer = AutoTokenizer.from_pretrained(adapter_dir)
    base_model = AutoModelForCausalLM.from_pretrained(args.base)
    model = PeftModel.from_pretrained(base_model, adapter_dir)

    prompt = "Reimu: What did we learn about LoRA?\nMarisa:"
    inputs = tokenizer(prompt, return_tensors="pt")
    outputs = model.generate(**inputs, max_new_tokens=40, do_sample=False)
    text = tokenizer.decode(outputs[0], skip_special_tokens=True)
    print("\nSample generation with adapter:")
    print(text)

    export_dir.mkdir(parents=True, exist_ok=True)
    for name in ("adapter_config.json", "adapter_model.safetensors"):
        src = adapter_dir / name
        if src.exists():
            (export_dir / name).write_bytes(src.read_bytes())

    readme = {
        "title": "yukkuri-hf-ch07-lora-demo",
        "base_model": args.base,
        "task": "causal_lm",
        "notes": "Adapter-only export from Chapter 7-B. Upload this folder to Hub.",
    }
    (export_dir / "adapter_card.json").write_text(
        json.dumps(readme, indent=2), encoding="utf-8"
    )
    print(f"\nExported shareable adapter files to: {export_dir}")
    print("OK: Chapter 7-B adapter save / inspect complete")


if __name__ == "__main__":
    main()

`scripts/ch07_lora_switch.py`

"""Chapter 7-C: Switch between multiple LoRA adapters on one base model."""

from __future__ import annotations

import argparse
from pathlib import Path

import torch
from peft import LoraConfig, PeftModel, TaskType, get_peft_model
from transformers import AutoModelForCausalLM, AutoTokenizer

DEFAULT_BASE = "distilgpt2"
DEFAULT_ADAPTER_A = "log/ch07_lora_adapter/adapter"
DEFAULT_ADAPTER_B = "log/ch07_lora_adapter_b/adapter"


def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(description="Compare LoRA adapters")
    parser.add_argument("--base", default=DEFAULT_BASE)
    parser.add_argument("--adapter-a", default=DEFAULT_ADAPTER_A)
    parser.add_argument("--adapter-b", default=DEFAULT_ADAPTER_B)
    parser.add_argument(
        "--create-demo-b",
        action="store_true",
        help="Create a second demo adapter if missing (different rank)",
    )
    parser.add_argument(
        "--prompt",
        default="Reimu: Explain LoRA in one sentence.\nMarisa:",
    )
    return parser.parse_args()


def generate(model, tokenizer, prompt: str) -> str:
    inputs = tokenizer(prompt, return_tensors="pt")
    device = next(model.parameters()).device
    inputs = {k: v.to(device) for k, v in inputs.items()}
    with torch.inference_mode():
        outputs = model.generate(**inputs, max_new_tokens=50, do_sample=False)
    return tokenizer.decode(outputs[0], skip_special_tokens=True)


def create_demo_adapter_b(base_model_id: str, output: Path) -> None:
    """Build a tiny second adapter so comparison works out of the box."""
    from datasets import Dataset
    from transformers import DataCollatorForLanguageModeling, Trainer, TrainingArguments

    output.mkdir(parents=True, exist_ok=True)
    tokenizer = AutoTokenizer.from_pretrained(base_model_id)
    if tokenizer.pad_token is None:
        tokenizer.pad_token = tokenizer.eos_token

    model = AutoModelForCausalLM.from_pretrained(base_model_id)
    lora_config = LoraConfig(
        task_type=TaskType.CAUSAL_LM,
        r=4,
        lora_alpha=8,
        lora_dropout=0.05,
        target_modules=["c_attn"],
        bias="none",
    )
    model = get_peft_model(model, lora_config)

    texts = [
        "Reimu: Spaces can host Gradio demos.\n"
        "Marisa: Share your model with a URL, easy!",
        "Reimu: Batch inference improves throughput.\n"
        "Marisa: Measure before you optimize, ze!",
    ]
    ds = Dataset.from_dict({"text": texts})

    def tokenize(batch):
        return tokenizer(
            batch["text"],
            truncation=True,
            max_length=64,
            padding="max_length",
        )

    ds = ds.map(tokenize, batched=True, remove_columns=["text"])
    trainer = Trainer(
        model=model,
        args=TrainingArguments(
            output_dir=str(output / "tmp"),
            num_train_epochs=2,
            per_device_train_batch_size=2,
            learning_rate=3e-4,
            logging_steps=1,
            save_strategy="no",
            report_to="none",
            use_cpu=not torch.cuda.is_available(),
        ),
        train_dataset=ds,
        data_collator=DataCollatorForLanguageModeling(tokenizer, mlm=False),
    )
    trainer.train()
    model.save_pretrained(output)
    tokenizer.save_pretrained(output)
    print(f"Created demo adapter B at {output}")


def main() -> None:
    args = parse_args()
    adapter_a = Path(args.adapter_a)
    adapter_b = Path(args.adapter_b)

    if args.create_demo_b and not adapter_b.exists():
        create_demo_adapter_b(args.base, adapter_b)

    if not adapter_a.exists():
        raise SystemExit(
            f"Adapter A not found: {adapter_a}\n"
            "Run scripts/ch07_lora_train.py first."
        )
    if not adapter_b.exists():
        raise SystemExit(
            f"Adapter B not found: {adapter_b}\n"
            "Run with --create-demo-b or train a second adapter."
        )

    tokenizer = AutoTokenizer.from_pretrained(adapter_a)
    if tokenizer.pad_token is None:
        tokenizer.pad_token = tokenizer.eos_token

    print(f"Base model: {args.base}")
    print(f"Prompt:\n{args.prompt}\n")

    base = AutoModelForCausalLM.from_pretrained(args.base)
    model_a = PeftModel.from_pretrained(base, adapter_a)
    print("=== Adapter A ===")
    print(generate(model_a, tokenizer, args.prompt))

    base = AutoModelForCausalLM.from_pretrained(args.base)
    model_b = PeftModel.from_pretrained(base, adapter_b)
    print("\n=== Adapter B ===")
    print(generate(model_b, tokenizer, args.prompt))

    print("\nTip: use model.load_adapter() / set_adapter() to hot-swap at runtime.")
    print("OK: Chapter 7-C adapter switch complete")


if __name__ == "__main__":
    main()

Reimu's Notebook

Compared to Chapter 6's Full FT, LoRA trains only a tiny fraction of parameters to save VRAM and disk.
Flow is LoraConfig → get_peft_model → Trainer, same as Chapter 6. For inference, use PeftModel.from_pretrained.
Hub sharing works with adapter only. Keep the base model ID in config.

Marisa's One More Thing

Load multiple LoRA adapters on one base and switch with set_adapter() for fast A/B tests, ZE.

from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("distilgpt2")
model = PeftModel.from_pretrained(base, "log/ch07_lora_adapter/adapter")
model.load_adapter("log/ch07_lora_adapter_b/adapter", adapter_name="style_b")
model.set_adapter("style_b")

On to the Next Chapter

Marisa
We've got "train lighter with LoRA" down. Next is Chapter 8　Making Inference Faster, Cheaper, and Production-Ready. Batch sizing and the Docker inference entry point, ZE.

Reimu
Faster in production than in training?

Marisa
Exactly. Even fine-tuned models face throughput and cost when you serve them. Only people who measure get faster.

Chapter 8　Making Inference Faster, Cheaper, and Production-Ready

8.1 Batch Inference and Throughput

Reimu
We learned LoRA in Chapter 7, but when you ship a model to production, won't people say "it's slow!"?

Marisa
That worry is Chapter 8's theme, ZE. Inference is a different optimization game from training. First up: batch inference and throughput.

Term	Meaning
Latency	Time until one request returns
Throughput	Items processed per unit time (e.g. samples/sec)
Batch size	How many items you forward at once

Request 1 ─┐
Request 2 ─┼─► [batch forward] ─► results 1..N
Request 3 ─┘
     ↑
  Use the GPU together → throughput↑ (but per-item wait time may↑)

Reimu
Is batching faster than sending one at a time?

Marisa
On a GPU, batching often raises throughput. But if the batch gets too big, you hit out-of-memory and per-item latency gets worse. Measure and decide — that's the right call.

Skeleton for a measurement script:

import time
import torch

model.eval()
with torch.inference_mode():
    start = time.perf_counter()
    outputs = model(**batch_inputs)
    if torch.cuda.is_available():
        torch.cuda.synchronize()
    elapsed = time.perf_counter() - start

Reimu
What's synchronize?

Marisa
CUDA runs asynchronously. If you stop the clock without waiting, you get a lie that's too fast. Wait for the GPU to finish before you stop, ZE.

8.2 Overview of `transformers` Optimization Options

Reimu
Does the library have a "make it fast" switch too?

Marisa
It does. Here's what we touch in this book.

Technique	Summary	Role in this book
`torch.inference_mode()`	Inference-only mode without gradients	Essential baseline
`model.eval()`	Dropout etc. for inference	Essential baseline
`device_map="auto"`	Multi-GPU / CPU offload	Covered in Chapter 4
Half precision FP16 / BF16	Less memory↓, faster↑	For GPUs
`torch.compile` (PyTorch 2+)	Graph optimization	Try if your env supports it
BetterTransformer / SDPA	Faster attention implementations	transformers may auto-select internally

Example inference wrapper:

import torch
from transformers import AutoModelForSequenceClassification

model = AutoModelForSequenceClassification.from_pretrained(
    "distilbert-base-uncased-finetuned-sst-2-english"
)
model.eval()

if torch.cuda.is_available():
    model = model.to("cuda", dtype=torch.float16)

with torch.inference_mode():
    outputs = model(**inputs)

Reimu
Training mode while training, eval mode when serving — got it.

Marisa
Chapter 6's Trainer switches internally, but in hand-written inference code you need to set it explicitly, ZE.

8.3 A Look at ONNX / Optimum

Reimu
ONNX… I've only heard the name.

Marisa
ONNX is the standard for exporting models to a cross-framework format. On Hugging Face, optimum is the gateway for ONNX conversion.

pip install "optimum[onnxruntime]"

CLI export example (classification model):

optimum-cli export onnx \
  --model distilbert-base-uncased-finetuned-sst-2-english \
  log/ch08_onnx_distilbert

Inference with ONNX Runtime from Python:

from optimum.onnxruntime import ORTModelForSequenceClassification
from transformers import AutoTokenizer

model_id = "distilbert-base-uncased-finetuned-sst-2-english"
onnx_dir = "log/ch08_onnx_distilbert"

# First run: specify export target. Later runs can load directly from onnx_dir
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = ORTModelForSequenceClassification.from_pretrained(
    model_id, export=True
)
# model.save_pretrained(onnx_dir)  # when you want to cache

Reimu
When do you go ONNX?

Marisa
For CPU production, edge devices, or when you want fixed-graph speedups. For LLM generation, stacks like vLLM / TGI are often the mainstream choice. Pick by task and deploy target.

8.4 Server Inference Options — vLLM / TGI, etc. (Overview)

Reimu
Besides Gradio, is there a "server" option?

Marisa
For large-scale text generation as an API, dedicated servers are the norm, ZE.

Product / project	Characteristics
Text Generation Inference (TGI)	Official HF. Docker images for LLMs
vLLM	High-throughput LLM inference with PagedAttention
llama.cpp / Ollama	Local CPU / small GPU (advanced)

Architecture sketch:

Client (Gradio / app)
        │ HTTP / gRPC
        ▼
┌───────────────────────┐
│  TGI / vLLM server    │  ← batching · KV cache
└───────────┬───────────┘
            ▼
        GPU cluster

Reimu
Like wiring Gradio from Chapter 9 to TGI?

Marisa
A common split: Gradio for demos, TGI for production traffic. This chapter covers concepts and the Docker entry point. Deep dives go to appendix links, ZE.

8.5 Cost vs Latency Trade-offs

Reimu
Faster isn't always better, huh?

Marisa
Fast GPUs are expensive. Always-on vs on-demand, batch vs real-time — the sweet spot changes.

Scenario	Priority	Typical setup
In-house batch analysis (overnight)	Throughput	Large batch + cheaper GPU
Chat UI	Latency	Small batch + fast GPU / cache
Public demo (Spaces)	Cost	CPU Basic + small model
Production API	Stability	TGI + autoscaling

        low latency
            ▲
            │    ● chat UI
            │
            │         ● batch processing
            └──────────────────► low cost

Reimu
Spaces' free CPU tier matters here.

Marisa
Exactly. Chapter 9 lets you feel GPU Space vs CPU Space. For Chapter 7 LoRA models at serving time, small base + adapter is often easier to operate, ZE.

8.6 Hands-on 8 — Batch Benchmarking and Docker (Optional)

Reimu
Show me the numbers?

Marisa
Hands-on 8. Same model, vary batch size, measure throughput.

Hands-on 8 — Inference Benchmark

Exercise A Measure with different batch sizes on the same model

python scripts/ch08_benchmark.py

With a GPU:

python scripts/ch08_benchmark.py --device cuda --batch-sizes 1,4,8,16,32

CPU only:

python scripts/ch08_benchmark.py --device cpu --batch-sizes 1,2,4

Expected output example:

Model: distilbert-base-uncased-finetuned-sst-2-english
Device: cuda
 batch |  latency(ms) |  samples/sec
------------------------------------
     1 |        12.34 |         81.0
     4 |        18.56 |        215.5
     8 |        28.90 |        276.8

Best throughput at batch_size=8 (276.8 samples/sec)
OK: Chapter 8-A batch benchmark complete

Exercise B Try running an inference Docker image (optional)

TGI is for LLMs, but as an entry point you can try an HF inference server in Docker. GPU environments are often required (optional exercise).

# Docker installed. NVIDIA Container Toolkit needed for GPU use
docker pull ghcr.io/huggingface/text-generation-inference:latest

# Example: small public LLM (first download takes time)
docker run --gpus all -p 8080:80 \
  -v $HOME/.cache/huggingface:/data \
  ghcr.io/huggingface/text-generation-inference:latest \
  --model-id HuggingFaceH4/zephyr-7b-beta

Health check from another terminal:

curl http://localhost:8080/health

On CPU-only environments, prefer ONNX export (8.3) or Chapter 9's Gradio CPU demo instead.

pip install "optimum[onnxruntime]"
optimum-cli export onnx \
  --model distilbert-base-uncased-finetuned-sst-2-english \
  log/ch08_onnx_distilbert

Reimu
If Docker isn't an option, ONNX is OK?

Marisa
OK, ZE. Hands-on 8-B is about "containers are an option in production." No need to force GPU Docker.

Tweak ideas:

# Vary sequence length to get a feel for memory
python scripts/ch08_benchmark.py --seq-len 256 --batch-sizes 1,2,4,8

8.7 Connecting from Chapters 6–7

Reimu
How does everything so far tie into inference?

Marisa
Let's line it up.

Chapter	What we did	Impact on inference
Ch. 6 Trainer	Full FT	Accuracy↑, model size unchanged
Ch. 7 LoRA	Adapter FT	Ship adapters in MB; share the base
Ch. 8	Measurement · optimization	Same model — how you run it changes speed
Ch. 9 Gradio	Public UI	User-perceived latency surfaces

Example: batched LoRA inference:

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

tokenizer = AutoTokenizer.from_pretrained("distilgpt2")
base = AutoModelForCausalLM.from_pretrained("distilgpt2")
model = PeftModel.from_pretrained(base, "log/ch07_lora_adapter/adapter")
model.eval()

prompts = ["Hello!", "LoRA is efficient.", "Batch me."]
inputs = tokenizer(prompts, padding=True, return_tensors="pt")

with torch.inference_mode():
    outputs = model.generate(**inputs, max_new_tokens=20)

Full Chapter Scripts

Marisa
So you can work hands-on even without the repo, this chapter's finished scripts are included as-is. Save them under scripts/ with the same filenames and run, ZE. (Where you see python scripts/xxx.py, adjust the path to match where you saved the file.)

`scripts/ch08_benchmark.py`

"""Chapter 8-A: Benchmark inference throughput at different batch sizes."""

from __future__ import annotations

import argparse
import statistics
import time

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

DEFAULT_MODEL = "distilbert-base-uncased-finetuned-sst-2-english"


def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(description="Batch inference benchmark")
    parser.add_argument("--model", default=DEFAULT_MODEL)
    parser.add_argument(
        "--batch-sizes",
        default="1,2,4,8",
        help="Comma-separated batch sizes to test",
    )
    parser.add_argument("--seq-len", type=int, default=128, help="Input length")
    parser.add_argument("--warmup", type=int, default=3, help="Warmup iterations")
    parser.add_argument("--iters", type=int, default=10, help="Timed iterations")
    parser.add_argument(
        "--device",
        default="auto",
        choices=["auto", "cpu", "cuda"],
        help="Device for inference",
    )
    return parser.parse_args()


def resolve_device(choice: str) -> torch.device:
    if choice == "cpu":
        return torch.device("cpu")
    if choice == "cuda":
        if not torch.cuda.is_available():
            raise SystemExit("CUDA requested but not available.")
        return torch.device("cuda")
    return torch.device("cuda" if torch.cuda.is_available() else "cpu")


def make_batch(tokenizer, texts: list[str], seq_len: int) -> dict:
    return tokenizer(
        texts,
        padding="max_length",
        truncation=True,
        max_length=seq_len,
        return_tensors="pt",
    )


def benchmark(
    model,
    tokenizer,
    batch_size: int,
    seq_len: int,
    warmup: int,
    iters: int,
    device: torch.device,
) -> dict:
    sample = "This chapter measures batch inference throughput for Hugging Face models."
    texts = [sample] * batch_size

    model.eval()
    inputs = make_batch(tokenizer, texts, seq_len)
    inputs = {k: v.to(device) for k, v in inputs.items()}

    with torch.inference_mode():
        for _ in range(warmup):
            model(**inputs)

        latencies = []
        for _ in range(iters):
            if device.type == "cuda":
                torch.cuda.synchronize()
            start = time.perf_counter()
            model(**inputs)
            if device.type == "cuda":
                torch.cuda.synchronize()
            latencies.append(time.perf_counter() - start)

    total_samples = batch_size * iters
    total_time = sum(latencies)
    throughput = total_samples / total_time
    return {
        "batch_size": batch_size,
        "latency_ms": statistics.mean(latencies) * 1000,
        "throughput_samples_per_sec": throughput,
    }


def main() -> None:
    args = parse_args()
    device = resolve_device(args.device)
    batch_sizes = [int(x.strip()) for x in args.batch_sizes.split(",") if x.strip()]

    print(f"Model: {args.model}")
    print(f"Device: {device}")
    print(f"Batch sizes: {batch_sizes}")
    print(f"Warmup: {args.warmup}, timed iters: {args.iters}\n")

    tokenizer = AutoTokenizer.from_pretrained(args.model)
    model = AutoModelForSequenceClassification.from_pretrained(args.model)
    model.to(device)

    print(f"{'batch':>6} | {'latency(ms)':>12} | {'samples/sec':>12}")
    print("-" * 36)

    results = []
    for bs in batch_sizes:
        row = benchmark(
            model,
            tokenizer,
            batch_size=bs,
            seq_len=args.seq_len,
            warmup=args.warmup,
            iters=args.iters,
            device=device,
        )
        results.append(row)
        print(
            f"{row['batch_size']:>6} | "
            f"{row['latency_ms']:>12.2f} | "
            f"{row['throughput_samples_per_sec']:>12.1f}"
        )

    best = max(results, key=lambda r: r["throughput_samples_per_sec"])
    print(
        f"\nBest throughput at batch_size={best['batch_size']} "
        f"({best['throughput_samples_per_sec']:.1f} samples/sec)"
    )
    print("OK: Chapter 8-A batch benchmark complete")


if __name__ == "__main__":
    main()

Reimu's Notebook

Throughput depends on batch size and GPU utilization. Don't guess — ch08_benchmark.py to measure.
For inference, eval() + inference_mode() are baseline. Pick ONNX / TGI / vLLM by deploy target.
Speed · cost · latency trade off. Spaces demos can lean on CPU + small models.

Marisa's One More Thing

Pass batch_size to pipeline and it batches inference internally. Easier entry to throughput gains than a hand-written loop, ZE.

from transformers import pipeline

clf = pipeline("sentiment-analysis", model="distilbert-base-uncased-finetuned-sst-2-english")
texts = ["Great!", "Bad.", "OK"] * 10
results = clf(texts, batch_size=8)
print(len(results))

On to the Next Chapter

Marisa
We've got the entry to measurement and optimization. Next is Chapter 9 — Building a Demo App with Gradio. Time to share the model by URL and get feedback, ZE.

Reimu
I want to show off the model we sped up — with a UI!

Marisa
That's the spirit. Gradio + Hugging Face Spaces — setting up the capstone project in Chapter 10.

Chapter 9　Building a Demo App with Gradio

Reimu
We sped up inference through Chapter 8, but… how do I show friends "our AI is amazing!"?

Marisa
Handing over a notebook alone means some people won't run it because of environment differences. That's why you need a demo app, ZE.

Role of a demo	Explanation
Reproducibility	"Open this URL, same UI"
Sharing	One link for SNS, articles, video descriptions
Feedback	Real inputs expose weak spots
Hiring / review	Submission for competitions or internal PoCs

Ch. 7 LoRA (training)
        │
Ch. 8 inference optimization (run faster)
        │
Ch. 9 Gradio + Spaces (show it)  ← you are here
        │
Ch. 10 capstone project (connect everything)

Reimu
Can we show Chapter 6 Trainer outputs here too?

Marisa
Yes. For classification, Gradio + pipeline is fastest. For generation, wrap generate in a function. This chapter starts with a text classification demo, ZE.

9.2 Basic Gradio UI Components

Reimu
With Gradio, I don't have to write HTML?

Marisa
Python alone gives you a web UI. Line up components and wire inputs/outputs to a function.

pip install "gradio>=4.0"

Minimal example:

import gradio as gr

def greet(name: str) -> str:
    return f"Hello, {name}!"

demo = gr.Interface(fn=greet, inputs="text", outputs="text")
demo.launch()

Common components:

Component	Use
`gr.Textbox`	Text input
`gr.Label`	Classification scores
`gr.Slider`	Temperature, max_tokens, etc.
`gr.Image`	Image I/O (vision)
`gr.Audio`	Audio (Whisper, etc.)
`gr.Examples`	Sample inputs list

Example layout with Blocks:

import gradio as gr

with gr.Blocks() as demo:
    gr.Markdown("# My Demo")
    with gr.Row():
        inp = gr.Textbox(label="Input")
        out = gr.Label(label="Output")
    inp.change(fn=my_predict, inputs=inp, outputs=out)

demo.launch()

Reimu
Interface or Blocks — which one?

Marisa
Interface for one function, one screen. Blocks when you need layout freedom. This chapter's scripts use Blocks, ZE.

9.3 Putting a Model on a Web UI

Reimu
Can we put Chapter 2's pipeline straight on the UI?

Marisa
Yes. Just call pipeline inside the inference function.

from transformers import pipeline
import gradio as gr

classifier = pipeline(
    "sentiment-analysis",
    model="distilbert-base-uncased-finetuned-sst-2-english",
)

def predict(text: str):
    if not text.strip():
        return {"POSITIVE": 0.0, "NEGATIVE": 0.0}
    results = classifier(text)
    return {r["label"]: r["score"] for r in results}

with gr.Blocks() as demo:
    text = gr.Textbox(label="Text", lines=3)
    label = gr.Label(label="Sentiment")
    text.change(predict, inputs=text, outputs=label)

demo.launch()

For Chapter 7 LoRA, use PeftModel instead of pipeline (generation tasks):

# Classification LoRA example (concept)
from peft import PeftModel
from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch

base = AutoModelForSequenceClassification.from_pretrained("base-model-id")
model = PeftModel.from_pretrained(base, "path/to/lora/adapter")
tokenizer = AutoTokenizer.from_pretrained("path/to/lora/adapter")
model.eval()

def predict_lora(text: str):
    inputs = tokenizer(text, return_tensors="pt")
    with torch.inference_mode():
        logits = model(**inputs).logits
    # ... softmax → label dict

Reimu
Try locally first, then publish — safe order.

Marisa
That order is safe, ZE. In Hands-on A next, save ch09_gradio_app.py from the end of this chapter (Full Chapter Scripts) and run it.

9.4 Deploying to Hugging Face Spaces

Reimu
Spaces is hosting on the Hub?

Marisa
Hugging Face Spaces hosts Gradio / Streamlit / Docker apps on free to paid tiers, ZE. Push like a Git repo.

Deploy flow:

1. Create a Space on the Hub (SDK: Gradio)
2. Push app.py + requirements.txt
3. Space builds → public URL issued

Example requirements.txt for a Space:

transformers>=4.40
torch
gradio>=4.0

Space README.md header (YAML metadata):

---
title: Yukkuri HF Sentiment Demo
emoji: 🤗
colorFrom: blue
colorTo: green
sdk: gradio
sdk_version: 4.44.0
app_file: app.py
pinned: false
---

Finished Space code is in app.py under Full Chapter Scripts at the end of this chapter. You can copy to another repo and push.

# Example: after cloning a new Space repo
cp /path/to/yukkuri-hugging-face/scripts/app.py ./app.py
cp requirements.txt ./   # with the content above

git add app.py requirements.txt README.md
git commit -m "Add Gradio sentiment demo"
git push

Reimu
Build failures… I see them a lot.

Marisa
requirements version conflicts and app.py path typos are classics. Check the Space Logs tab. Reproduce locally in the same venv requirements, ZE.

9.5 Secrets and API Keys

Reimu
We don't paste Hub tokens into a Space, right…?

Marisa
Absolutely not. On Spaces, put them in Settings → Secrets. Locally, .env (not in git).

# .env (do not commit to the repo)
HF_TOKEN=hf_xxxxxxxxxxxxxxxx

Example reading from Gradio / Python:

import os
from huggingface_hub import login

token = os.environ.get("HF_TOKEN")
if token:
    login(token=token)

On Spaces, registering HF_TOKEN in Secrets injects it as an environment variable.

Where	OK?	Use
`.env` (local)	OK (gitignore)	Development
Space Secrets	OK	Production deploy
Hard-coded in `app.py`	No	Leak risk
This chapter's Markdown	No	Placeholders only in samples

Reimu
Chapter 1 token hygiene pays off here.

Marisa
Read and Write permissions minimal too. Don't put a Write token in a public demo Space — iron rule, ZE.

9.6 Hands-on 9 — Gradio Demo and Spaces

Reimu
I want a URL!

Marisa
Hands-on 9. Local Gradio → publish to Space → feel CPU vs GPU.

Hands-on 9 — Build and Publish a Demo

Exercise A Build a text classification demo with Gradio

pip install "gradio>=4.0"
python scripts/ch09_gradio_app.py

Open http://127.0.0.1:7860 in a browser. For a temporary public URL:

python scripts/ch09_gradio_app.py --share

Expected behavior:

Enter English text → POSITIVE / NEGATIVE scores appear
Examples buttons provide sample inputs

Exercise B Push to Spaces and share the URL

Create a Space at huggingface.co/new-space (SDK: Gradio)
Clone the generated Git remote
Place these files:

your-space/
├── app.py              ← copy from scripts/app.py
├── requirements.txt
└── README.md           ← with YAML metadata

git add app.py requirements.txt README.md
git commit -m "Deploy Chapter 9 sentiment demo"
git push

After the build finishes, https://huggingface.co/spaces/<user>/<name> is your share URL.

Exercise C Feel the difference between GPU Space and CPU Space

Setting	Pros	Cons
CPU Basic (free)	Zero cost; good for small models	Large models / generation are slow
GPU (paid credits)	Practical LLM generation	Watch cost and sleep

Same app.py — change Space Hardware and the feel changes.

CPU: distilbert classification demo is plenty snappy (this chapter's default)
GPU: for Chapter 7 LoRA generation demos (full use in Chapter 10)

app.py tweak sketch for a generation demo on GPU Space (Chapter 10 preview):

# Concept: GPU Space + small causal LM
from transformers import pipeline
import gradio as gr

gen = pipeline("text-generation", model="distilgpt2")

def complete(prompt: str):
    out = gen(prompt, max_new_tokens=40, do_sample=True, top_p=0.9)
    return out[0]["generated_text"]

gr.Interface(fn=complete, inputs="text", outputs="text").launch()

Reimu
Free CPU for classification, GPU for the capstone project.

Marisa
That understanding is OK, ZE. Keep cost down and share by URL first.

9.7 Connection from Chapter 8 — Feeling Latency in the UI

Reimu
In Gradio, will users notice slowness from Chapter 8's benchmark?

Marisa
Exactly. ch08_benchmark.py numbers surface as wait time in the demo.

Improvement order (recommended):

1. Small model / LoRA adapter (Ch. 7)
2. batch / FP16 (Ch. 8)
3. CPU vs GPU Space choice (this chapter)
4. API layer like TGI if needed (advanced)

Small trick — show inference time in Gradio:

import time

def predict_with_timing(text: str):
    start = time.perf_counter()
    result = predict(text)
    elapsed_ms = (time.perf_counter() - start) * 1000
    return result, f"{elapsed_ms:.1f} ms"

Full Chapter Scripts

`scripts/ch09_gradio_app.py`

"""Chapter 9-A: Local Gradio demo for text classification."""

from __future__ import annotations

import argparse

import gradio as gr
from transformers import pipeline

DEFAULT_MODEL = "distilbert-base-uncased-finetuned-sst-2-english"


def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(description="Gradio text classification demo")
    parser.add_argument("--model", default=DEFAULT_MODEL)
    parser.add_argument("--host", default="127.0.0.1")
    parser.add_argument("--port", type=int, default=7860)
    parser.add_argument(
        "--share",
        action="store_true",
        help="Create a temporary public Gradio link",
    )
    return parser.parse_args()


def build_demo(classifier) -> gr.Blocks:
    def predict(text: str):
        if not text.strip():
            return {"POSITIVE": 0.0, "NEGATIVE": 0.0}
        results = classifier(text)
        if isinstance(results[0], list):
            results = results[0]
        return {item["label"]: item["score"] for item in results}

    with gr.Blocks(title="Yukkuri HF Ch09 Demo") as demo:
        gr.Markdown(
            "# Ch. 9 demo: text sentiment analysis\n"
            "Chapter 2 `pipeline` mounted on a Gradio UI, ZE."
        )
        with gr.Row():
            text_in = gr.Textbox(
                label="Input text",
                placeholder="I love Hugging Face!",
                lines=3,
            )
            label_out = gr.Label(label="Prediction", num_top_classes=2)
        examples = gr.Examples(
            examples=[
                ["This book makes Hugging Face easy to learn!"],
                ["I am worried about CUDA out of memory."],
                ["Gradio demos are fun to share."],
            ],
            inputs=text_in,
        )
        text_in.change(fn=predict, inputs=text_in, outputs=label_out)
        gr.Markdown(
            "Deploy this app to **Hugging Face Spaces** with `app.py` (Hands-on 9-B)."
        )
    return demo


def main() -> None:
    args = parse_args()
    print(f"Loading pipeline: {args.model}")
    classifier = pipeline("sentiment-analysis", model=args.model)

    demo = build_demo(classifier)
    demo.launch(
        server_name=args.host,
        server_port=args.port,
        share=args.share,
    )


if __name__ == "__main__":
    main()

`scripts/app.py`

"""Hugging Face Spaces entry point (Chapter 9-B).

Deploy by creating a Space with Gradio SDK and pushing this file as app.py.
"""

import gradio as gr
from transformers import pipeline

MODEL_ID = "distilbert-base-uncased-finetuned-sst-2-english"

print(f"Loading model: {MODEL_ID}")
classifier = pipeline("sentiment-analysis", model=MODEL_ID)


def predict(text: str):
    if not text or not text.strip():
        return {"POSITIVE": 0.0, "NEGATIVE": 0.0}
    results = classifier(text)
    if isinstance(results[0], list):
        results = results[0]
    return {item["label"]: item["score"] for item in results}


with gr.Blocks(title="Yukkuri HF Space Demo") as demo:
    gr.Markdown(
        "# Yukkuri Hugging Face — Sentiment Demo\n"
        "Chapter 9 Space template. CPU Basic works for this tiny model."
    )
    text_in = gr.Textbox(label="Text", lines=3)
    label_out = gr.Label(label="Sentiment")
    text_in.submit(predict, inputs=text_in, outputs=label_out)
    gr.Examples(
        examples=[
            ["Learning LoRA on Colab is awesome!"],
            ["My GPU ran out of memory again."],
        ],
        inputs=text_in,
    )

if __name__ == "__main__":
    demo.launch()

Reimu's Notebook

Gradio builds web demos in Python alone; Spaces publishes them by URL.
Tokens only in .env / Space Secrets. Never in code or Markdown.
CPU Space for small classification; GPU Space for generation / LoRA production demos (Ch. 10).

Marisa's One More Thing

Duplicate a Space to fork someone else's demo and tweak it. Finding Gradio Spaces on HF and duplicating for practice works too, ZE.

# Local dev: hot reload
python scripts/ch09_gradio_app.py
# Gradio can try reload mode on code changes (depends on environment)

Example cloning a Space repo with Hub CLI:

git clone https://huggingface.co/spaces/your-username/your-space-name

On to the Next Chapter

Marisa
From Chapter 1's Hub through Chapter 6 Trainer, Chapter 7 LoRA, Chapter 8 measurement, and this chapter's Gradio + Spaces — we're set. Next is Chapter 10 Capstone Project — Yukkuri Live-Comment Generator Bot, ZE.

Reimu
We're publishing an AI that talks like us to the whole world!?

Marisa
Data collection → LoRA → inference optimization → Space publish — end to end. Everything connects in the capstone. Look forward to it!

Chapter 10　Capstone Project — Yukkuri Live-Comment Generator Bot

10.1 Project Planning

Reimu
Through Chapter 9 we've done Hub, Pipeline, LoRA, and Gradio. So what do we build at the end?

Marisa
A capstone project, ZE. Theme: Yukkuri live-comment generator bot — feed game situations, get Reimu-and-Marisa-style one-liners back.

Reimu
…we're training an AI on our speech patterns?

Marisa
It's a small demo for this book. Not production character AI — the goal is wiring Chapters 1–9 into one pipeline.

【Input】Game situation (e.g. right after beating a boss)
        │
        ▼
【Preprocess】Script JSONL → training text format
        │
        ▼
【Train】Japanese GPT-2 + LoRA (Ch. 7)
        │
        ▼
【Infer】Prompt + generate (Ch. 4)
        │
        ▼
【Publish】Gradio + Spaces (Ch. 9)
        │
        ▼
【Improve】Eval → add data → retrain (10.8)

Chapter	Role in this chapter
1	Pick base model and dataset from the Hub
3	Design tokenization format
5	JSONL → `Dataset`
7	Add speech style with LoRA
8	Inference params · batch (optional)
9	Share with Gradio / Space

Reimu
Copyright — are we OK?

Marisa
Safest: scripts you wrote yourself. No unauthorized reuse of existing videos. This book's scripts/data/ch10_dialogues.jsonl is only short original examples, ZE.

10.2 Data Collection Policy

Reimu
How much data do we need?

Marisa
Production character quality wants hundreds to thousands of examples. This book's demo is around 20 — enough to see the flow. Decide the format first.

10.2.1 JSONL — One Line per Scene

Marisa
One line per scene in JSONL. Minimal fields, ZE.

{"situation": "After boss defeat", "reimu": "We did it!", "marisa": "Don't let your guard down yet, ZE."}

Reimu
From video subtitles?

Marisa
Example workflow:

Source	How to build	Notes
Original scripts	Spreadsheet → JSONL export	Safest
Subtitles (your videos)	Your videos only	Not others' videos
Existing corpora	Hub Datasets — check license	Read commercial / derivative use
Synthetic data	LLM draft → human edit	Always review; don't use raw

10.2.2 Where Sample Data Lives

Bundled example:

scripts/data/ch10_dialogues.jsonl

Line count:

wc -l scripts/data/ch10_dialogues.jsonl

Reimu
About 20 lines?

Marisa
Demo size, ZE. Add rows in the same format when you grow it.

10.3 Preprocessing and Tokenizer Tuning

Reimu
You can't train on raw JSONL, right?

Marisa
Causal LMs (GPT family) predict continuation text. Pack each example into one string block.

[Situation] After boss defeat
Reimu: We did it! That took forever!
Marisa: Don't let your guard down. There might be a secret room, ZE.

Reimu
Like Chapter 5's map?

Marisa
Exactly. This chapter wraps it in a script.

cd /path/to/yukkuri-hugging-face
source .venv/bin/activate

python scripts/ch10_prepare_data.py

Expected output example:

Loaded 20 examples from .../ch10_dialogues.jsonl
  train: 17
  validation: 3
Saved to: log/ch10_dataset
OK: Chapter 10 data preparation complete

Peek inside (Chapter 5 review):

from datasets import load_from_disk

ds = load_from_disk("log/ch10_dataset")
print(ds)
print(ds["train"][0]["text"])

Reimu
Split into train and validation.

Marisa
Used in 10.8's improvement loop to check generation quality on validation, ZE. Change the ratio with --val-ratio.

python scripts/ch10_prepare_data.py --val-ratio 0.2

10.3.1 About the Tokenizer

Marisa
Use the base model's tokenizer as-is by default. If you need frequent custom words like "yukkuri", consider vocabulary extension from Chapter 3 — skipped in this chapter.

10.4 Choosing a Base Model (Japanese LLM)

Reimu
English distilgpt2 would sound wrong in Japanese.

Marisa
This chapter's production pick is rinna/japanese-gpt2-medium, ZE. Japanese GPT-2 family, manageable on a personal PC.

Model	Language	Size	Use in this chapter
`rinna/japanese-gpt2-medium`	Japanese	Medium	Recommended (production demo)
`distilgpt2`	English	Small	Smoke test (flow check on CPU)
7B-class LLM	Multilingual	Large	Colab Pro, etc. QLoRA (Ch. 7)

Reimu
Try English first, then Japanese?

Marisa
Recommended order, ZE.

# 1) Flow check only (minutes · first download)
python scripts/ch10_lora_train.py --smoke --epochs 1

# 2) Production Japanese model (first run downloads from Hub)
python scripts/ch10_lora_train.py --epochs 5

Read the model card on the Hub (Chapter 1):

https://huggingface.co/rinna/japanese-gpt2-medium

Always check license and terms of use.

10.5 LoRA Fine-Tuning

Reimu
Chapter 7 LoRA goes live here.

Marisa
Same flow. Prepare data → LoRA → save adapter. This chapter's script is ch10_lora_train.py, ZE.

python scripts/ch10_prepare_data.py
python scripts/ch10_lora_train.py \
  --model rinna/japanese-gpt2-medium \
  --epochs 5 \
  --lr 3e-4 \
  --rank 8

Main options:

Option	Meaning
`--model`	Base model ID
`--epochs`	Epoch count (small data overfits easily)
`--lr`	Learning rate
`--rank`	LoRA rank
`--smoke`	`distilgpt2` CPU smoke test

Expected output example (excerpt):

trainable params: 294,912 || all params: ...
Starting LoRA training...
Saved adapter to: log/ch10_lora_adapter/adapter
OK: Chapter 10 LoRA training complete

Reimu
Too many epochs — memorize the scripts?

Marisa
Right, ZE. With ~20 examples, start at 3–5 epochs and watch. If eval_loss drops too far, add data or cut epochs.

Files after training:

log/ch10_lora_adapter/
├── adapter/              # LoRA + tokenizer (inference · Gradio)
│   ├── adapter_config.json
│   ├── adapter_model.safetensors
│   └── ...
└── checkpoints/          # Trainer mid-run (empty if save_strategy=no)

Push adapter only to the Hub (Ch. 6 · Ch. 7 review):

huggingface-cli login

# Optional: upload to Hub (replace username)
from huggingface_hub import HfApi

api = HfApi()
api.upload_folder(
    folder_path="log/ch10_lora_adapter/adapter",
    repo_id="your-username/yukkuri-comment-lora",
    repo_type="model",
)

10.6 Inference Script and Prompt Design

Reimu
Training's done. How do we output comments?

Marisa
Prompt format — make it continue text. Match the same shape as training data, ZE.

def build_prompt(situation: str) -> str:
    return f"[Situation]{situation}\nReimu:"

You want the model to generate after「霊夢:」→ newline →「魔理沙:」…

Try from CLI:

python scripts/ch10_generate.py --situation "First time on this stage"

Generation parameters (Chapter 4 review):

python scripts/ch10_generate.py \
  --situation "Game over" \
  --temperature 0.8 \
  --top-p 0.9 \
  --max-new-tokens 100

Parameter	Effect
`temperature`	Higher → more random · creative
`top_p`	Narrow tokens by cumulative probability
`max_new_tokens`	Max generation length (too long → drift)

Reimu
Different text every time.

Marisa
Because do_sample=True, ZE. For reproducibility try temperature=0 or do_sample=False (model-dependent).

Example from Python:

from pathlib import Path
import json
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

adapter = Path("log/ch10_lora_adapter/adapter")
cfg = json.loads((adapter / "adapter_config.json").read_text(encoding="utf-8"))
base_name = cfg["base_model_name_or_path"]

tokenizer = AutoTokenizer.from_pretrained(adapter)
base = AutoModelForCausalLM.from_pretrained(base_name)
model = PeftModel.from_pretrained(base, adapter)
model.eval()

situation = "Rare drop"
prompt = f"[Situation]{situation}\nReimu:"
inputs = tokenizer(prompt, return_tensors="pt")
with torch.no_grad():
    out = model.generate(**inputs, max_new_tokens=80, do_sample=True, temperature=0.85)
print(tokenizer.decode(out[0], skip_special_tokens=True))

10.7 Publishing with Gradio + Spaces

Reimu
Playing alone is boring. I want everyone to try it.

Marisa
Put LoRA inference on Chapter 9's Gradio. Local script: ch10_gradio_app.py, ZE.

python scripts/ch10_gradio_app.py

Open http://127.0.0.1:7861 in a browser, enter a【状況】, press「生成するのだ」.

10.7.1 Space Layout

Notes for Spaces:

Item	Recommendation
SDK	Gradio
Hardware	Japanese GPT-2 + LoRA often runs on CPU Basic
Model	Push adapter to Hub → `from_pretrained` from Space
Secrets	`HF_TOKEN` (when writing). Chapter 9 — don't leak

Minimal Space app.py (adapter on Hub):

import gradio as gr
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

ADAPTER_ID = "your-username/yukkuri-comment-lora"

tokenizer = AutoTokenizer.from_pretrained(ADAPTER_ID)
base_name = "rinna/japanese-gpt2-medium"
base = AutoModelForCausalLM.from_pretrained(base_name)
model = PeftModel.from_pretrained(base, ADAPTER_ID)
model.eval()

def predict(situation: str) -> str:
    prompt = f"[Situation]{situation.strip()}\nReimu:"
    inputs = tokenizer(prompt, return_tensors="pt")
    out = model.generate(**inputs, max_new_tokens=100, do_sample=True, temperature=0.85)
    return tokenizer.decode(out[0], skip_special_tokens=True)

with gr.Blocks() as demo:
    gr.Markdown("# Yukkuri Live-Comment Bot")
    s = gr.Textbox(label="[Situation]")
    o = gr.Textbox(label="Generated output", lines=8)
    gr.Button("Generate").click(predict, s, o)

demo.launch()

Reimu
What about requirements.txt?

Marisa
Space example, ZE.

transformers>=4.40
peft>=0.11
accelerate>=0.30
torch
gradio>=4.0
sentencepiece

# Flow from Ch. 9: clone Space repo and push
git clone https://huggingface.co/spaces/your-username/yukkuri-comment-bot
# Place app.py and requirements.txt
cd yukkuri-comment-bot
git add app.py requirements.txt
git commit -m "Add ch10 comment bot"
git push

10.8 Improvement Loop (Eval → Add Data → Retrain)

Reimu
When output is meh, how do we fix it?

Marisa
Run a loop.

① Generate on fixed scenes (eval)
② Human labels: good / meh / bad
③ Fix bad lines → add to JSONL
④ prepare → LoRA retrain
⑤ Update Space

Batch eval script:

python scripts/ch10_eval_samples.py

Output:

log/ch10_eval.jsonl

Example line:

{"situation": "After boss defeat", "generated": "[Situation] After boss defeat\nReimu: ..."}

Reimu
No automatic good/bad judgment?

Marisa
This book recommends human review, ZE. BLEU and perplexity alone won't measure "sounds like us." Pair with Chapter 8 latency measurement for quality and speed.

Improvement checklist:

Symptom	Fix
Speech style breaks	More scripts · unify prompt format with training
Same phrase repeats	Raise `temperature` / diversify data
English mixed in	Switch to Japanese base model
Generation too long	Lower `max_new_tokens`
Training slow	Debug with `--smoke`, then GPU / Colab for production

Comprehensive Hands-on — Connect Chapters 1–9 and Finish

Reimu
Last step — run the whole flow from the start!

Marisa
Comprehensive hands-on checklist, ZE. Check all boxes and you clear the main book.

Step 0: Environment (Chapter 0)

source .venv/bin/activate
python scripts/check_env.py
pip install "peft>=0.11" "gradio>=4.0" sentencepiece

Step 1: Confirm model on Hub (Chapter 1)

[ ] Read Model Card for rinna/japanese-gpt2-medium
[ ] Confirmed license

Step 2: Data prep (Ch. 5 · 10.2–10.3)

# After adding one custom row
python scripts/ch10_prepare_data.py

Exercise A Add one original scene as one line to ch10_dialogues.jsonl.

Step 3: LoRA training (Ch. 7 · 10.5)

python scripts/ch10_lora_train.py --epochs 5

Exercise B Note the trainable params line for --smoke vs production model in your notebook.

Step 4: Inference (Ch. 4 · 10.6)

python scripts/ch10_generate.py --situation "End of stream"

Exercise C Compare temperature at 0.5 and 1.0; note which feels more "live-comment-like."

Step 5: Gradio publish (Ch. 9 · 10.7)

python scripts/ch10_gradio_app.py

[ ] Tried 3 scenes in local UI
[ ] (Optional) Pushed adapter to Hub and created a Space

Step 6: Improvement loop (10.8)

python scripts/ch10_eval_samples.py

[ ] Opened log/ch10_eval.jsonl, fixed one "bad" scene in scripts, retrained

Reimu
Long… but if I got here, I can build a Space alone.

Marisa
That's the capstone goal, ZE.

Full Chapter Scripts

`scripts/ch10_prepare_data.py`

"""Chapter 10: Prepare JSONL dialogues for LoRA training."""

from __future__ import annotations

import argparse
import json
from pathlib import Path

from datasets import Dataset, DatasetDict

DEFAULT_INPUT = Path(__file__).resolve().parent / "data" / "ch10_dialogues.jsonl"
DEFAULT_OUTPUT = Path("log/ch10_dataset")


def format_example(row: dict) -> str:
    """Single training text block (causal LM)."""
    return (
        f"[Situation]{row['situation']}\n"
        f"Reimu: {row['reimu']}\n"
        f"Marisa: {row['marisa']}\n"
    )


def load_jsonl(path: Path) -> list[dict]:
    rows: list[dict] = []
    with path.open(encoding="utf-8") as f:
        for line in f:
            line = line.strip()
            if line:
                rows.append(json.loads(line))
    return rows


def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(description="Prepare ch10 training dataset")
    parser.add_argument("--input", type=Path, default=DEFAULT_INPUT)
    parser.add_argument("--output", type=Path, default=DEFAULT_OUTPUT)
    parser.add_argument(
        "--val-ratio",
        type=float,
        default=0.15,
        help="Validation split ratio",
    )
    return parser.parse_args()


def main() -> None:
    args = parse_args()
    rows = load_jsonl(args.input)
    texts = [format_example(r) for r in rows]

    dataset = Dataset.from_dict({"text": texts})
    split = dataset.train_test_split(test_size=args.val_ratio, seed=42)
    dataset_dict = DatasetDict(train=split["train"], validation=split["test"])

    args.output.mkdir(parents=True, exist_ok=True)
    dataset_dict.save_to_disk(str(args.output))

    print(f"Loaded {len(texts)} examples from {args.input}")
    print(f"  train: {len(dataset_dict['train'])}")
    print(f"  validation: {len(dataset_dict['validation'])}")
    print(f"Saved to: {args.output}")
    print("\nSample:")
    print(dataset_dict["train"][0]["text"])
    print("OK: Chapter 10 data preparation complete")


if __name__ == "__main__":
    main()

`scripts/ch10_lora_train.py`

"""Chapter 10: LoRA fine-tune yukkuri comment generator."""

from __future__ import annotations

import argparse
from pathlib import Path

import torch
from datasets import load_from_disk
from peft import LoraConfig, TaskType, get_peft_model
from transformers import (
    AutoModelForCausalLM,
    AutoTokenizer,
    DataCollatorForLanguageModeling,
    Trainer,
    TrainingArguments,
)

# Japanese: rinna/japanese-gpt2-medium (first run downloads ~500MB)
# CPU demo: distilgpt2 (English-ish output; use for smoke test only)
DEFAULT_MODEL = "rinna/japanese-gpt2-medium"
DEFAULT_DATASET = Path("log/ch10_dataset")
DEFAULT_OUTPUT = Path("log/ch10_lora_adapter")


def detect_target_modules(model_name: str) -> list[str]:
    if "gpt2" in model_name.lower() or "rinna" in model_name.lower():
        return ["c_attn", "c_proj"]
    return ["q_proj", "v_proj"]


def tokenize_dataset(tokenizer, dataset, max_length: int):
    def tokenize(batch: dict) -> dict:
        return tokenizer(
            batch["text"],
            truncation=True,
            max_length=max_length,
            padding="max_length",
        )

    return dataset.map(tokenize, batched=True, remove_columns=["text"])


def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(description="Ch10 LoRA training")
    parser.add_argument("--model", default=DEFAULT_MODEL)
    parser.add_argument("--dataset", type=Path, default=DEFAULT_DATASET)
    parser.add_argument("--output", type=Path, default=DEFAULT_OUTPUT)
    parser.add_argument("--epochs", type=int, default=5)
    parser.add_argument("--lr", type=float, default=3e-4)
    parser.add_argument("--rank", type=int, default=8)
    parser.add_argument("--max-length", type=int, default=256)
    parser.add_argument(
        "--smoke",
        action="store_true",
        help="Use distilgpt2 for quick CPU smoke test",
    )
    return parser.parse_args()


def main() -> None:
    args = parse_args()
    model_id = "distilgpt2" if args.smoke else args.model

    if not args.dataset.exists():
        raise SystemExit(
            f"Dataset not found: {args.dataset}\n"
            "Run: python scripts/ch10_prepare_data.py"
        )

    args.output.mkdir(parents=True, exist_ok=True)

    print(f"Loading dataset: {args.dataset}")
    dataset_dict = load_from_disk(str(args.dataset))

    print(f"Loading base model: {model_id}")
    tokenizer = AutoTokenizer.from_pretrained(model_id)
    if tokenizer.pad_token is None:
        tokenizer.pad_token = tokenizer.eos_token

    model = AutoModelForCausalLM.from_pretrained(model_id)

    lora_config = LoraConfig(
        task_type=TaskType.CAUSAL_LM,
        r=args.rank,
        lora_alpha=16,
        lora_dropout=0.05,
        target_modules=detect_target_modules(model_id),
        bias="none",
    )
    model = get_peft_model(model, lora_config)
    model.print_trainable_parameters()

    train_ds = tokenize_dataset(
        tokenizer, dataset_dict["train"], args.max_length
    )
    eval_ds = tokenize_dataset(
        tokenizer, dataset_dict["validation"], args.max_length
    )

    use_cpu = not torch.cuda.is_available()
    training_args = TrainingArguments(
        output_dir=str(args.output / "checkpoints"),
        num_train_epochs=args.epochs,
        per_device_train_batch_size=2,
        per_device_eval_batch_size=2,
        learning_rate=args.lr,
        eval_strategy="epoch",
        logging_steps=5,
        save_strategy="no",
        report_to="none",
        use_cpu=use_cpu,
    )

    trainer = Trainer(
        model=model,
        args=training_args,
        train_dataset=train_ds,
        eval_dataset=eval_ds,
        data_collator=DataCollatorForLanguageModeling(
            tokenizer=tokenizer,
            mlm=False,
        ),
    )

    print("Starting LoRA training...")
    trainer.train()

    adapter_path = args.output / "adapter"
    model.save_pretrained(adapter_path)
    tokenizer.save_pretrained(adapter_path)
    print(f"Saved adapter to: {adapter_path}")
    print("OK: Chapter 10 LoRA training complete")


if __name__ == "__main__":
    main()

`scripts/ch10_generate.py`

"""Chapter 10: Generate yukkuri-style comments with LoRA adapter."""

from __future__ import annotations

import argparse
from pathlib import Path

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

DEFAULT_ADAPTER = Path("log/ch10_lora_adapter/adapter")


def build_prompt(situation: str) -> str:
    return f"[Situation]{situation}\nReimu:"


def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(description="Ch10 comment generation")
    parser.add_argument(
        "--adapter",
        type=Path,
        default=DEFAULT_ADAPTER,
        help="Path to saved LoRA adapter",
    )
    parser.add_argument(
        "--situation",
        default="After boss defeat",
        help="Game situation for the comment",
    )
    parser.add_argument("--max-new-tokens", type=int, default=80)
    parser.add_argument("--temperature", type=float, default=0.8)
    parser.add_argument("--top-p", type=float, default=0.9)
    return parser.parse_args()


def main() -> None:
    args = parse_args()
    if not args.adapter.exists():
        raise SystemExit(
            f"Adapter not found: {args.adapter}\n"
            "Run: python scripts/ch10_prepare_data.py\n"
            "     python scripts/ch10_lora_train.py"
        )

    import json

    print(f"Loading adapter from: {args.adapter}")
    cfg_path = args.adapter / "adapter_config.json"
    if not cfg_path.exists():
        raise SystemExit(f"Missing adapter_config.json in {args.adapter}")
    base_name = json.loads(cfg_path.read_text(encoding="utf-8"))[
        "base_model_name_or_path"
    ]

    tokenizer = AutoTokenizer.from_pretrained(args.adapter)
    base_model = AutoModelForCausalLM.from_pretrained(base_name)
    model = PeftModel.from_pretrained(base_model, args.adapter)
    model.eval()

    device = "cuda" if torch.cuda.is_available() else "cpu"
    model = model.to(device)

    prompt = build_prompt(args.situation)
    inputs = tokenizer(prompt, return_tensors="pt").to(device)

    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            max_new_tokens=args.max_new_tokens,
            do_sample=True,
            temperature=args.temperature,
            top_p=args.top_p,
            pad_token_id=tokenizer.pad_token_id,
        )

    text = tokenizer.decode(outputs[0], skip_special_tokens=True)
    print("\n--- Generated ---")
    print(text)
    print("-----------------")


if __name__ == "__main__":
    main()

`scripts/ch10_gradio_app.py`

"""Chapter 10: Gradio demo for yukkuri comment generator."""

from __future__ import annotations

import argparse
import json
from pathlib import Path

import gradio as gr
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

DEFAULT_ADAPTER = Path("log/ch10_lora_adapter/adapter")


def load_generator(adapter_path: Path):
    tokenizer = AutoTokenizer.from_pretrained(adapter_path)
    base_name = tokenizer.name_or_path
    cfg_path = adapter_path / "adapter_config.json"
    if cfg_path.exists():
        base_name = json.loads(cfg_path.read_text())["base_model_name_or_path"]

    base_model = AutoModelForCausalLM.from_pretrained(base_name)
    model = PeftModel.from_pretrained(base_model, adapter_path)
    model.eval()
    device = "cuda" if torch.cuda.is_available() else "cpu"
    model = model.to(device)
    return model, tokenizer, device


def make_predict(model, tokenizer, device, max_new_tokens: int, temperature: float, top_p: float):
    def predict(situation: str) -> str:
        if not situation.strip():
            return "Enter a situation."
        prompt = f"[Situation]{situation.strip()}\nReimu:"
        inputs = tokenizer(prompt, return_tensors="pt").to(device)
        with torch.no_grad():
            outputs = model.generate(
                **inputs,
                max_new_tokens=max_new_tokens,
                do_sample=True,
                temperature=temperature,
                top_p=top_p,
                pad_token_id=tokenizer.pad_token_id,
            )
        return tokenizer.decode(outputs[0], skip_special_tokens=True)

    return predict


def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(description="Ch10 Gradio Space demo")
    parser.add_argument("--adapter", type=Path, default=DEFAULT_ADAPTER)
    parser.add_argument("--host", default="127.0.0.1")
    parser.add_argument("--port", type=int, default=7861)
    parser.add_argument("--share", action="store_true")
    return parser.parse_args()


def main() -> None:
    args = parse_args()
    if not args.adapter.exists():
        raise SystemExit(
            f"Adapter not found: {args.adapter}\n"
            "Train first: python scripts/ch10_lora_train.py"
        )

    print(f"Loading: {args.adapter}")
    model, tokenizer, device = load_generator(args.adapter)
    predict = make_predict(model, tokenizer, device, max_new_tokens=100, temperature=0.85, top_p=0.9)

    with gr.Blocks(title="Yukkuri Comment Bot") as demo:
        gr.Markdown(
            "# Ch. 10: yukkuri live-comment generator bot\n"
            "Enter a [Situation] to generate Reimu/Marisa-style comments, ZE."
        )
        situation = gr.Textbox(
            label="[Situation]",
            placeholder="e.g. After boss defeat",
            lines=2,
        )
        out = gr.Textbox(label="Generated output", lines=8)
        btn = gr.Button("Generate")
        gr.Examples(
            examples=[
                ["First time on this stage"],
                ["Game over"],
                ["Right after Space publish"],
            ],
            inputs=situation,
        )
        btn.click(fn=predict, inputs=situation, outputs=out)

    demo.launch(server_name=args.host, server_port=args.port, share=args.share)


if __name__ == "__main__":
    main()

`scripts/ch10_eval_samples.py`

"""Chapter 10.8: Evaluate generations on fixed situations (improvement loop)."""

from __future__ import annotations

import argparse
import json
from pathlib import Path

DEFAULT_ADAPTER = Path("log/ch10_lora_adapter/adapter")


def build_prompt(situation: str) -> str:
    return f"[Situation]{situation}\nReimu:"

EVAL_SITUATIONS = [
    "After boss defeat",
    "First time on this stage",
    "Game over",
    "Viewer comment",
]


def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(description="Ch10 batch eval for improvement loop")
    parser.add_argument("--adapter", type=Path, default=DEFAULT_ADAPTER)
    parser.add_argument("--output", type=Path, default=Path("log/ch10_eval.jsonl"))
    return parser.parse_args()


def main() -> None:
    import torch
    from peft import PeftModel
    from transformers import AutoModelForCausalLM, AutoTokenizer

    args = parse_args()
    if not args.adapter.exists():
        raise SystemExit(f"Adapter not found: {args.adapter}")

    tokenizer = AutoTokenizer.from_pretrained(args.adapter)
    cfg_path = args.adapter / "adapter_config.json"
    base_name = json.loads(cfg_path.read_text())["base_model_name_or_path"]
    base_model = AutoModelForCausalLM.from_pretrained(base_name)
    model = PeftModel.from_pretrained(base_model, args.adapter)
    model.eval()
    device = "cuda" if torch.cuda.is_available() else "cpu"
    model = model.to(device)

    args.output.parent.mkdir(parents=True, exist_ok=True)
    with args.output.open("w", encoding="utf-8") as f:
        for situation in EVAL_SITUATIONS:
            prompt = build_prompt(situation)
            inputs = tokenizer(prompt, return_tensors="pt").to(device)
            with torch.no_grad():
                outputs = model.generate(
                    **inputs,
                    max_new_tokens=100,
                    do_sample=True,
                    temperature=0.8,
                    top_p=0.9,
                    pad_token_id=tokenizer.pad_token_id,
                )
            text = tokenizer.decode(outputs[0], skip_special_tokens=True)
            record = {"situation": situation, "generated": text}
            f.write(json.dumps(record, ensure_ascii=False) + "\n")
            print(f"\n[{situation}]\n{text}\n")

    print(f"Wrote eval log: {args.output}")
    print("OK: Review outputs, add bad cases to ch10_dialogues.jsonl, re-train.")


if __name__ == "__main__":
    main()

`scripts/space_ch10_app.py`

"""Hugging Face Spaces entry point for Chapter 10 (optional).

Upload your trained adapter to Hub, set ADAPTER_ID below, and push as app.py.
"""

from __future__ import annotations

import gradio as gr
import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

# After pushing adapter: your-username/yukkuri-comment-lora
ADAPTER_ID = "your-username/yukkuri-comment-lora"
BASE_MODEL_ID = "rinna/japanese-gpt2-medium"

print(f"Loading base: {BASE_MODEL_ID}")
tokenizer = AutoTokenizer.from_pretrained(ADAPTER_ID)
base = AutoModelForCausalLM.from_pretrained(BASE_MODEL_ID)
model = PeftModel.from_pretrained(base, ADAPTER_ID)
model.eval()
device = "cuda" if torch.cuda.is_available() else "cpu"
model = model.to(device)


def predict(situation: str) -> str:
    if not situation or not situation.strip():
        return "Enter a [Situation]."
    prompt = f"[Situation]{situation.strip()}\nReimu:"
    inputs = tokenizer(prompt, return_tensors="pt").to(device)
    with torch.no_grad():
        out = model.generate(
            **inputs,
            max_new_tokens=100,
            do_sample=True,
            temperature=0.85,
            top_p=0.9,
            pad_token_id=tokenizer.pad_token_id,
        )
    return tokenizer.decode(out[0], skip_special_tokens=True)


with gr.Blocks(title="Yukkuri Comment Bot") as demo:
    gr.Markdown("# Yukkuri Live-Comment Bot (Ch. 10 Space)")
    situation = gr.Textbox(label="[Situation]", lines=2)
    output = gr.Textbox(label="Generated output", lines=8)
    gr.Button("Generate").click(predict, situation, output)
    gr.Examples(
        examples=[["After boss defeat"], ["Game over"], ["End of stream"]],
        inputs=situation,
    )

if __name__ == "__main__":
    demo.launch()

`scripts/data/ch10_dialogues.jsonl` (data file)

{"situation": "After boss defeat", "reimu": "We did it! That took forever!", "marisa": "Don't let your guard down. There might be a secret room, ZE."}
{"situation": "First time on this stage", "reimu": "This looks tough…", "marisa": "Watch the enemy patterns first. There's always a pattern, ZE."}
{"situation": "Game over", "reimu": "One more try…", "marisa": "Your save is still there. Just add more training data, ZE."}
{"situation": "Rare drop", "reimu": "Whoa, amazing gear!", "marisa": "Lucky roll, ZE. But match stats to your build."}
{"situation": "Tutorial", "reimu": "Got the controls down?", "marisa": "Timing beats button mashing every time, ZE."}
{"situation": "Before secret boss", "reimu": "My legs are shaking…", "marisa": "You made it this far—you can win. Like LoRA, small steps add up, ZE."}
{"situation": "Stream start", "reimu": "Everyone watching?", "marisa": "Today's the Hugging Face finale. Even comments go to AI now, ZE."}
{"situation": "Waiting on model download", "reimu": "Still not done…", "marisa": "First run can be several GB. Cache it and the next run is fast, ZE."}
{"situation": "No GPU", "reimu": "CPU only…", "marisa": "Small models and LoRA still work. Borrow a GPU on Colab too, ZE."}
{"situation": "Right after Space publish", "reimu": "Got the URL!", "marisa": "Write usage in the README and collect feedback, ZE."}
{"situation": "Weird comments", "reimu": "The tone is off…", "marisa": "Add data and retrain. Evaluate → add → FT cycle, ZE."}
{"situation": "Good comment generated", "reimu": "That sounds like us!", "marisa": "Prompt and LoRA are lining up—that's proof, ZE."}
{"situation": "Long play session", "reimu": "So sleepy…", "marisa": "Save a checkpoint and rest. Humans overfit too, ZE."}
{"situation": "Co-op play", "reimu": "Marisa, cover me!", "marisa": "On it. I run inference; you gather data, ZE."}
{"situation": "Ranking update", "reimu": "We climbed the ranks!", "marisa": "Same base model, different adapter—different personality, ZE."}
{"situation": "Hit a bug", "reimu": "It won't run!", "marisa": "Read the full error. Typos and CUDA OOM are classics, ZE."}
{"situation": "Ending", "reimu": "Congrats on the clear!", "marisa": "All ten chapters done. Review commands in the appendices, ZE."}
{"situation": "Bonus stage", "reimu": "There's more?", "marisa": "One more thing lives outside the book too—read the official docs, ZE."}
{"situation": "Viewer comment", "reimu": "They said we need more data", "marisa": "Fair. This book is a demo. Add scripts for production, ZE."}
{"situation": "End of stream", "reimu": "Good work today!", "marisa": "Next, duplicate your Space and hack on it, ZE!"}

Reimu's Notebook

JSONL scripts → ch10_prepare_data.py → LoRA → ch10_generate.py → Gradio is the straight path.
Japanese: rinna/japanese-gpt2-medium; flow check: --smoke (distilgpt2).
Quality rises via eval → human fix → add to JSONL → retrain.

Marisa's One More Thing

In production, pin the base model on the Hub and version only the adapter — rollback gets easier, ZE.

# Image: push adapter only with a tag
huggingface-cli upload your-username/yukkuri-comment-lora ./log/ch10_lora_adapter/adapter .

Loading multiple speech styles (live / commentary / banter) via LoRA switching (Chapter 7 ch07_lora_switch.py) is an advanced Chapter 10 exercise.

End of the Book

Marisa
From Chapter 0's environment setup through this chapter's Yukkuri live-comment bot — the main 10 chapters end here, ZE.

Reimu
Good work! Are there appendices?

Marisa
Appendices in docs_en/00-toc.md — CLI list, troubleshooting, hands-on index. When stuck, go back to the relevant chapter.

Reimu
…next we're streaming our book on a Space!

Marisa
That's the spirit. The Hugging Face world keeps growing as you touch it, ZE. Take it easy!

Appendix A: Environment Setup Cheat Sheet

Chapter 0 summary + copy-paste commands. If stuck, return to the main text at docs_en/00.md.

Reimu
Appendix means cheat sheet?

Marisa
Exactly, ZE. venv, CUDA, Colab, and Kaggle in shortest steps. Less dialogue here, more commands.

A.1 Minimum setup (local; CPU is fine)

Item	Recommendation
OS	macOS / Linux / WSL2 (Windows: WSL recommended)
Python	3.10+ (3.11 recommended)
Disk	10 GB+ free (for model cache)
GPU	Optional (NVIDIA + CUDA makes training easier)

cd /path/to/yukkuri-hugging-face
python3 --version
python3 -m venv .venv
source .venv/bin/activate   # Windows: .venv\Scripts\Activate.ps1
pip install --upgrade pip

Bulk install of basic packages for this book:

pip install \
  "transformers>=4.40" \
  "datasets>=2.18" \
  "accelerate>=0.28" \
  "huggingface_hub>=0.22" \
  "torch" \
  "sentencepiece" \
  "protobuf"

Added from Chapter 7 onward:

pip install "peft>=0.11" "gradio>=4.0"

Verify:

python scripts/check_env.py

A.2 Common venv operations

# Activate
source .venv/bin/activate

# Which python
which python

# List packages
pip list | grep -E "transformers|torch|datasets"

# Delete and recreate venv (when broken)
deactivate
rm -rf .venv
python3 -m venv .venv && source .venv/bin/activate
pip install --upgrade pip
# Re-run the pip install from A.1

What to put in .gitignore (Chapter 0):

.venv/
__pycache__/
*.pyc
.cache/
log/
.env

A.3 CUDA / PyTorch (NVIDIA GPU)

Marisa
GPU owners only, ZE. Pick a wheel that matches your environment at PyTorch Get Started.

CUDA 12.x example:

pip install torch --index-url https://download.pytorch.org/whl/cu124

Verify:

import torch
print(torch.__version__)
print("CUDA:", torch.cuda.is_available())
if torch.cuda.is_available():
    print(torch.cuda.get_device_name(0))

Symptom	What to check
`CUDA: False`	Driver / CUDA version / torch build mismatch
`CUDA out of memory`	Lower batch size, smaller model, LoRA, 8bit (Ch. 4)

A.4 Cache and environment variables

# Default cache location
ls ~/.cache/huggingface/

# Move cache to another drive (bash)
export HF_HOME="/path/to/large-disk/huggingface"
export TRANSFORMERS_CACHE="$HF_HOME/hub"

# Offline only (already downloaded)
import os
os.environ["HF_HUB_OFFLINE"] = "1"

A.5 Google Colab

Example first Colab cell:

!pip install -q "transformers>=4.40" "datasets>=2.18" accelerate peft gradio

import torch
print(torch.__version__, "CUDA:", torch.cuda.is_available())

Item	Notes
GPU	Menu "Runtime → T4 GPU" etc.
Drive	`from google.colab import drive; drive.mount('/content/drive')` for persistence
Book scripts	Clone from GitHub or upload files

!git clone https://github.com/your-org/yukkuri-hugging-face.git
%cd yukkuri-hugging-face
!python scripts/check_env.py

On free tier days without GPU, use Chapter 2 Pipeline or --smoke for CPU checks.

A.6 Kaggle Notebooks

Kaggle also offers GPU. Flow is similar to Colab, ZE.

Create a Notebook at kaggle.com
Settings → turn GPU ON
Add Data → mount datasets if needed
pip install in the first cell → run scripts/

import sys
!pip install -q transformers datasets accelerate peft
sys.path.append("/kaggle/working/yukkuri-hugging-face")

Colab vs Kaggle	Rough comparison
Colab	Easy; Google account
Kaggle	Weekly GPU quota; great with competitions and datasets

A.7 HF token (Chapter 1)

pip install -U "huggingface_hub[cli]"
huggingface-cli login
# or
export HF_TOKEN="hf_xxxxxxxx"   # put in .env; do not commit

Pushing requires a token with write permission.

Reimu's Notebook (Appendix A)

Activate venv → pip → check_env.py is the first gate every time.
GPU is not required. Colab / Kaggle are for borrowing a GPU.
Cache location can be changed with HF_HOME.

Appendix B: Frequently Used CLI Commands

Examples for huggingface-cli / git / running this book's scripts/.

Reimu
I forget the commands every time…

Marisa
Collected them in Appendix B. Copy-paste and swap in your your-username, ZE.

B.1 Hugging Face Hub CLI

Install and login:

pip install -U "huggingface_hub[cli]"
huggingface-cli whoami
huggingface-cli login
huggingface-cli logout

Download models and datasets:

# Cache entire model
huggingface-cli download distilbert-base-uncased-finetuned-sst-2-english

# Specific files only
huggingface-cli download meta-llama/Llama-2-7b-hf config.json --include "*.json"

# Expand into a local directory
huggingface-cli download distilbert-base-uncased-finetuned-sst-2-english --local-dir ./my-model

Upload (Chapters 6 and 10):

# Upload folder to model repo
huggingface-cli upload your-username/my-model ./log/ch06_output .

# Single file
huggingface-cli upload your-username/my-model ./log/ch10_lora_adapter/adapter/adapter_config.json

Cache:

huggingface-cli scan-cache
huggingface-cli delete-cache

B.2 Git (for Spaces; Chapter 9)

git clone https://huggingface.co/spaces/your-username/your-space-name
cd your-space-name

# First time only
git config user.email "you@example.com"
git config user.name "Your Name"

git add app.py requirements.txt
git commit -m "Update Gradio demo"
git push

Clone a model repo:

git clone https://huggingface.co/bert-base-uncased
# Large models use Git LFS; follow the README

B.3 Python scripts (by chapter; representative)

Ch.	Command
0	`python scripts/check_env.py`
1	`python scripts/download_model.py --model-id MODEL`
1	`python scripts/inspect_model.py --model-id MODEL`
2	`python scripts/ch02_pipeline_sentiment.py`
3	`python scripts/ch03_tokenizer_compare.py`
4	`python scripts/ch04_forward.py`
5	`python scripts/ch05_load_dataset.py`
6	`python scripts/ch06_finetune.py --epochs 1`
7	`python scripts/ch07_lora_train.py`
8	`python scripts/ch08_benchmark.py`
9	`python scripts/ch09_gradio_app.py`
10	`python scripts/ch10_prepare_data.py && python scripts/ch10_lora_train.py`

How to read help (common):

python scripts/ch06_finetune.py --help

B.4 pip / packages

pip install --upgrade pip
pip install transformers datasets accelerate peft gradio
pip freeze > requirements.txt
pip install -r requirements.txt

Example requirements.txt for a Space:

transformers>=4.40
torch
datasets>=2.18
accelerate>=0.28
peft>=0.11
gradio>=4.0
sentencepiece
protobuf

B.5 Logs and debugging

# Save run log (Ch. 0)
python scripts/check_env.py 2>&1 | tee log/env_check.txt

# Verbose logging (transformers)
export TRANSFORMERS_VERBOSITY=debug
python scripts/ch02_pipeline_sentiment.py

B.6 Environment variable quick reference

Variable	Purpose
`HF_TOKEN`	CLI / Hub authentication
`HF_HOME`	Cache root
`TRANSFORMERS_CACHE`	Model cache
`HF_HUB_OFFLINE`	`1` for offline only
`CUDA_VISIBLE_DEVICES`	GPU index to use (e.g. `0`)
`WANDB_API_KEY`	W&B integration (Ch. 6; optional)

export CUDA_VISIBLE_DEVICES=0
export HF_HOME="$HOME/hf-cache"

Reimu's Notebook (Appendix B)

Read with huggingface-cli download; write with login + upload.
Spaces use git push as the default.
When lost, python scripts/xxx.py --help.

Appendix C: Model Selection Flowchart

Decision guide for "which model to pick" from Chapters 1, 4, and 10.

Reimu
There are too many models on the Hub — I can't choose…

Marisa
Prune by task, language, and GPU, ZE. Follow the flow below and you'll usually land on one.

C.1 Overall flow (text version)

Start: What do you want to do?
│
├─ Text classification / sentiment → Encoder family (BERT / DistilBERT)
│                              pipeline("sentiment-analysis")
│
├─ Translation / summarization / QA → Task-specific or multilingual Seq2Seq
│                              pipeline("translation"), etc.
│
├─ Image classification / detection → ViT / DETR, etc. (Ch. 2 Vision)
│
├─ Speech-to-text → Whisper family
│
└─ Text generation / chat → Causal LM (GPT-2 / LLaMA family)
                               For Japanese, rinna, etc.

        ↓ Multiple model candidates

    Language? — Japanese required → prefer Japanese-trained models
        │
    GPU memory? — tight → Distil family / LoRA / 8bit
        │
    License OK? — No → pick another model (Ch. 1, 1.4)
        │
    Check limits and training data in the Model Card
        │
    Try small → pipeline or ch02 scripts
        │
    If not enough → FT (Ch. 6) or LoRA (Ch. 7)

C.2 Mermaid flow (view in a Mermaid viewer)

flowchart TD
    A[What task?] --> B{Need generation?}
    B -->|No| C[Encoder: BERT / DistilBERT]
    B -->|Yes| D[Causal LM: GPT / LLaMA family]
    C --> E{Japanese?}
    D --> E
    E -->|Yes| F[Search Japanese models on the Hub]
    E -->|No| G[Try English-base models]
    F --> H{Enough VRAM?}
    G --> H
    H -->|No| I[LoRA / small model / Colab]
    H -->|Yes| J[Full FT is also OK]
    I --> K[Check Model Card and license]
    J --> K
    K --> L[Smoke test with pipeline]

C.3 Starting points by task (IDs used in this book)

Task	Example in this book	Size
Sentiment (EN)	`distilbert-base-uncased-finetuned-sst-2-english`	Small
Sentiment (JA)	`daigo/bert-base-japanese-sentiment` etc. (Ch. 2)	Medium
Translation	`staka/fugumoji-enja` etc. (Ch. 2)	Medium
Classification FT	`distilbert-base-uncased` + custom data (Ch. 6)	Small
LoRA demo	`distilgpt2` / `rinna/japanese-gpt2-medium`	Small–medium
Vision	`google/vit-base-patch16-224` (Ch. 2)	Medium
Speech	`openai/whisper-tiny` (Ch. 2)	Small

C.4 Selection checklist

On the Hub model page, check these top to bottom.

[ ] Task (Tags: text-classification, text-generation, etc.)
[ ] Language (includes Japanese if needed)
[ ] Downloads and last updated (not too stale)
[ ] Model Card (training data, limits, bias)
[ ] License (commercial use OK? attribution required?)
[ ] Inference example (README code runs)
[ ] Fits your GPU (parameter count, quantization available)

C.5 Failure patterns and what to try next

When it fails	Next move
Japanese breaks	Switch to Japanese pretrained
OOM	Smaller model / lower batch / LoRA / 8bit
Too slow	Distil family / Ch. 8 batch benchmarks
Tone doesn't match	Add data + LoRA (Ch. 10)
License NG	Pick another; prefer Apache / MIT for commercial use

Reimu's Notebook (Appendix C)

Narrow in order: task → language → VRAM → license.
Try small with pipeline first.
Skipping the Hub Model Card bites you later.

Appendix D: Troubleshooting Guide

Cross-chapter index of errors from the end of Chapter 2 and each chapter.

Reimu
Error messages in English are scary…

Marisa
Look up this table by keywords in the message, ZE. Copy the full text and search — that's fastest.

D.1 Environment & Installation

Error / Symptom	Cause	Fix
`ModuleNotFoundError: transformers`	venv not used / pip not run	`source .venv/bin/activate` → pip install
Won't run on `Python 3.9`	Version too old	Upgrade to Python 3.10+ (Appendix A)
`pip` slow / fails	Network	Retry, mirror, proxy settings
`torch` and CUDA mismatch	Wrong wheel	Reinstall from pytorch.org

D.2 Hub & Authentication

Error / Symptom	Cause	Fix
`401 Unauthorized`	Not logged in / invalid token	`huggingface-cli login`
`403 Forbidden`	Gated model not approved	Apply for access on the Hub page
`Repository Not Found`	Model name typo	Copy-paste the ID to verify
`HF_TOKEN` not working	Env var not set	`.env` or `export` (don't commit to git)

huggingface-cli whoami

D.3 Download & Cache

Error / Symptom	Cause	Fix
First run never finishes	Large download	Wait / try a smaller model
Disk full	Cache bloated	Change `HF_HOME`, `huggingface-cli scan-cache`
`Connection reset`	Unstable network	Re-run; offline only after download

huggingface-cli scan-cache

D.4 CUDA & Memory

Error / Symptom	Cause	Fix
`CUDA out of memory`	Batch / model too large	`per_device_train_batch_size=1`, smaller model, LoRA, 8bit
`CUDA available: False`	CPU torch / driver	CUDA steps in Appendix A. Small models work on CPU too
`device-side assert`	Label out of range, etc.	Check class count matches `num_labels` (Chapter 6)

import torch
torch.cuda.empty_cache()

D.5 Pipeline & Inference

Error / Symptom	Cause	Fix
Model name typo	Wrong `-` / `_`	Copy exactly from Hub URL
Japanese output wrong	English model	Switch to a Japanese model (Appendix C)
Output short / repetitive	`max_new_tokens` too small	Tune Chapter 4 `generate` parameters
`pipeline` slow	First DL + CPU	Cached from 2nd run onward. GPU recommended

Representative example from Chapter 2:

# Specify model explicitly
from transformers import pipeline
clf = pipeline("sentiment-analysis", model="distilbert-base-uncased-finetuned-sst-2-english")

D.6 Tokenizer & Data

Error / Symptom	Cause	Fix
`padding` error	Batch length mismatch	`padding=True` or DataCollator (Chapter 3)
Over-truncation	`max_length` too small	Increase `max_length`
`map` slow	Recomputing every time	`dataset.map(..., load_from_cache_file=True)`
CSV column mismatch	Schema mismatch	Match column names in `ch05_csv_to_dataset.py`

D.7 Trainer & Training

Error / Symptom	Cause	Fix
`loss=nan`	Learning rate too high	Lower `learning_rate` (e.g. `2e-5`)
Overfitting	Little data / many epochs	Fewer epochs, more data, watch eval
No checkpoints	`save_strategy`	Set `save_steps` in `TrainingArguments`
W&B error	API key	`report_to="none"` or set key

TrainingArguments(..., report_to="none")

D.8 LoRA & PEFT

Error / Symptom	Cause	Fix
`target_modules` error	Model structure differs	Check `c_attn` (GPT-2) vs `q_proj` (LLaMA)
Adapter not found	Wrong path	Check `log/ch07_lora_adapter/adapter`
Generation still English	English base model	Switch to `rinna/japanese-gpt2-medium`, etc. (Chapter 10)

D.9 Gradio & Spaces

Error / Symptom	Cause	Fix
Port in use	7860 occupied	`--port 7861`
Space won't start	Missing `requirements.txt`	Check logs under Space "Logs"
GPU Space billing	Hardware setting	Start with CPU Basic and a small model
Secrets leaked	Hardcoded in code	Space Settings → Secrets

D.10 Debugging Pattern

Marisa
A 3-step pattern that works in any chapter.

1. Copy the full error (through the Exception type on the first line)
2. Shrink to minimal repro code (1 batch / 1 example)
3. Check options with --help on that chapter's script

python scripts/ch06_finetune.py --max-train 32 --max-eval 16 --epochs 1

Reimu's Notebook (Appendix D)

venv and login solve half the problems.
OOM → smaller model, LoRA, lower batch.
If stuck, go back to the chapter script via Appendix F.

Appendix E: Reference Links

Official docs, courses, and community. URLs are as of writing time. If 404, search within the site.

Reimu
After finishing the book, where should I look?

Marisa
Official docs are the right route, ZE. Community is for questions. Bookmark these links and reuse them.

E.1 Hugging Face Official

Name	URL	Purpose
Hub home	https://huggingface.co	Search models & datasets
Transformers docs	https://huggingface.co/docs/transformers	API reference
Datasets docs	https://huggingface.co/docs/datasets	Data loading & `map`
Hub client	https://huggingface.co/docs/huggingface_hub	CLI, auth, upload
PEFT	https://huggingface.co/docs/peft	LoRA, etc.
Accelerate	https://huggingface.co/docs/accelerate	Distributed & mixed precision
Gradio	https://www.gradio.app/docs	UI
Spaces docs	https://huggingface.co/docs/hub/spaces	Deploy
Model Cards	https://huggingface.co/docs/hub/model-cards	How to write README
License list	https://huggingface.co/docs/hub/repositories-licenses	Terms of use

E.2 Courses & Tutorials

Name	URL	Notes
HF free courses	https://huggingface.co/learn	NLP / LLM courses
Transformers quick tour	https://huggingface.co/docs/transformers/quicktour	Official pipeline intro
Open Source AI Cookbook	https://huggingface.co/learn/cookbook	Recipe collection
PEFT quick tour	https://huggingface.co/docs/peft/quicktour	LoRA intro

Name	URL	Notes
PyTorch official	https://pytorch.org
Get Started (CUDA)	https://pytorch.org/get-started/locally/	Wheel selection
bitsandbytes	https://github.com/TimDettmers/bitsandbytes	8bit / 4bit

E.4 Inference & Operations (Advanced)

Name	URL	Notes
Optimum	https://huggingface.co/docs/optimum	ONNX, etc. (Chapter 8)
Text Generation Inference	https://github.com/huggingface/text-generation-inference	TGI
vLLM	https://github.com/vllm-project/vllm	Fast inference
Safetensors	https://huggingface.co/docs/safetensors	Weight format

E.5 Community

Name	URL	Notes
HF forum	https://discuss.huggingface.co	Questions & bug reports
Discord	https://hf.co/join/discord	Chat
GitHub transformers	https://github.com/huggingface/transformers	Issues & PRs
GitHub datasets	https://github.com/huggingface/datasets

Tips when asking questions:

- transformers / datasets version
- Minimal repro code (~10 lines)
- Full error message
- GPU yes/no (output of check_env.py)

E.6 Japanese Resources (Optional)

Name	Notes
Company tech blogs	Japanese model case studies (rinna, etc.)
Qiita / Zenn	Search for "Hugging Face", "LoRA"
This book's repo	`docs_en/` main text + appendices

Japanese model example (Chapter 10):

https://huggingface.co/rinna/japanese-gpt2-medium

E.7 Where to Go Back in This Book

Goal	Chapter
Environment	Chapter 0, `appendix-a.md`
Hub	Chapter 1
Quick start	Chapter 2
Training	Chapters 6–7
Publishing	Chapters 9–10
Commands	`appendix-b.md`
Errors	`appendix-d.md`
Script index	`appendix-f.md`

Reimu's Notebook (Appendix E)

Official docs come first.
If stuck, ask on the forum with a minimal repro.
This book is a beginner's single path — go deeper via each official source.

Appendix F: Chapter Hands-On Complete Code Index

Index of scripts/ and mapping to chapters and hands-ons. Full code for each script appears at the end of each chapter under "Full Script for This Chapter." If reading without the repo, refer there.

Reimu
There are so many scripts, I'm lost.

Marisa
Made a lookup table by chapter number and hands-on ID. From the repo root, run python scripts/..., ZE.

F.1 Common

File	Chapter	Description
`scripts/check_env.py`	0	Environment & version check

cd /path/to/yukkuri-hugging-face
source .venv/bin/activate
python scripts/check_env.py

F.2 Chapter 1 — Hub

Hands-on	File	Example run
1-A	(browser)	Read a Model Card on the Hub
1-B	CLI	`huggingface-cli login`
1-C	`scripts/download_model.py`	below
1-C	`scripts/inspect_model.py`	below

python scripts/download_model.py \
  --model-id distilbert-base-uncased-finetuned-sst-2-english

python scripts/inspect_model.py \
  --model-id distilbert-base-uncased-finetuned-sst-2-english

F.3 Chapter 2 — Pipeline

Hands-on	File	Example run
2-A	`scripts/ch02_pipeline_sentiment.py`	Japanese sentiment analysis
2-B	`scripts/ch02_pipeline_translation.py`	EN→JA translation
2-C	`scripts/ch02_pipeline_compare.py`	Model comparison
(§2.5)	`scripts/ch02_pipeline_vision.py`	Image classification
(§2.6)	`scripts/ch02_pipeline_whisper.py`	Speech recognition

python scripts/ch02_pipeline_sentiment.py
python scripts/ch02_pipeline_translation.py
python scripts/ch02_pipeline_compare.py
python scripts/ch02_pipeline_vision.py
python scripts/ch02_pipeline_whisper.py

F.4 Chapter 3 — Tokenizer

Hands-on	File	Example run
3-A	`scripts/ch03_tokenizer_compare.py`	Segmentation comparison
3-B	`scripts/ch03_tokenizer_batch.py`	Long-text batch
3-C	`scripts/ch03_tokenizer_custom_vocab.py`	Add vocabulary

python scripts/ch03_tokenizer_compare.py
python scripts/ch03_tokenizer_batch.py
python scripts/ch03_tokenizer_custom_vocab.py

F.5 Chapter 4 — Model

Hands-on	File	Example run
4-A	`scripts/ch04_forward.py`	Manual forward
4-B	`scripts/ch04_generate.py`	`generate` parameters
4-C	`scripts/ch04_quantize.py`	8bit (optional; bitsandbytes)

python scripts/ch04_forward.py
python scripts/ch04_generate.py --temperature 0.8 --top-p 0.9
python scripts/ch04_quantize.py   # optional

F.6 Chapter 5 — Datasets

Hands-on	File	Example run
5-A	`scripts/ch05_load_dataset.py`	Load `ag_news`
5-B	`scripts/ch05_preprocess.py`	map + tokenize
5-C	`scripts/ch05_csv_to_dataset.py`	CSV → Dataset
Data	`scripts/data/ch05_sample_reviews.csv`	Sample for 5-C

python scripts/ch05_load_dataset.py
python scripts/ch05_preprocess.py
python scripts/ch05_csv_to_dataset.py
python scripts/ch05_csv_to_dataset.py --csv scripts/data/ch05_sample_reviews.csv

F.7 Chapter 6 — Trainer

Hands-on	File	Example run
6-A/B	`scripts/ch06_finetune.py`	Classification FT + logs
6-C	`scripts/ch06_push_hub.py`	Push to Hub

python scripts/ch06_finetune.py --max-train 800 --max-eval 200 --epochs 1
python scripts/ch06_push_hub.py --help
# Before push: huggingface-cli login

F.8 Chapter 7 — LoRA

Hands-on	File	Example run
7-A	`scripts/ch07_lora_train.py`	LoRA training
7-B	`scripts/ch07_lora_save_adapter.py`	Verify adapter
7-C	`scripts/ch07_lora_switch.py`	Switch adapter

python scripts/ch07_lora_train.py
python scripts/ch07_lora_save_adapter.py
python scripts/ch07_lora_switch.py --create-demo-b

Expected output location: log/ch07_lora_adapter/adapter

F.9 Chapter 8 — Inference Optimization

Hands-on	File	Example run
8-A	`scripts/ch08_benchmark.py`	Batch size benchmark
8-B	(optional)	Docker / TGI — see Chapter 8 main text

python scripts/ch08_benchmark.py
python scripts/ch08_benchmark.py --batch-sizes 1,2,4,8

F.10 Chapter 9 — Gradio / Spaces

Hands-on	File	Example run
9-A	`scripts/ch09_gradio_app.py`	Local Gradio
9-B	`scripts/app.py`	Space template (sentiment analysis)
9-C	(Space settings)	CPU / GPU hardware comparison

python scripts/ch09_gradio_app.py
python scripts/ch09_gradio_app.py --port 7860 --share

F.11 Chapter 10 — Capstone Project

Step / Topic	File	Example run
Data	`scripts/data/ch10_dialogues.jsonl`	Dialogue scripts
Preprocess	`scripts/ch10_prepare_data.py`	Create Dataset
LoRA	`scripts/ch10_lora_train.py`	Training
Inference	`scripts/ch10_generate.py`	CLI generation
Gradio	`scripts/ch10_gradio_app.py`	UI
Evaluation	`scripts/ch10_eval_samples.py`	Improvement loop
Space	`scripts/space_ch10_app.py`	Chapter 10 Space

python scripts/ch10_prepare_data.py
python scripts/ch10_lora_train.py --smoke --epochs 1
python scripts/ch10_lora_train.py --epochs 5
python scripts/ch10_generate.py --situation "After boss defeat"
python scripts/ch10_gradio_app.py --port 7861
python scripts/ch10_eval_samples.py

Expected output locations:

log/ch10_dataset/
log/ch10_lora_adapter/adapter/
log/ch10_eval.jsonl

F.12 File List (Alphabetical)

scripts/
├── app.py
├── check_env.py
├── ch02_pipeline_compare.py
├── ch02_pipeline_sentiment.py
├── ch02_pipeline_translation.py
├── ch02_pipeline_vision.py
├── ch02_pipeline_whisper.py
├── ch03_tokenizer_batch.py
├── ch03_tokenizer_compare.py
├── ch03_tokenizer_custom_vocab.py
├── ch04_forward.py
├── ch04_generate.py
├── ch04_quantize.py
├── ch05_csv_to_dataset.py
├── ch05_load_dataset.py
├── ch05_preprocess.py
├── ch06_finetune.py
├── ch06_push_hub.py
├── ch07_lora_save_adapter.py
├── ch07_lora_switch.py
├── ch07_lora_train.py
├── ch08_benchmark.py
├── ch09_gradio_app.py
├── ch10_eval_samples.py
├── ch10_generate.py
├── ch10_gradio_app.py
├── ch10_lora_train.py
├── ch10_prepare_data.py
├── download_model.py
├── inspect_model.py
├── space_ch10_app.py
└── data/
    ├── ch05_sample_reviews.csv
    └── ch10_dialogues.jsonl

F.13 Mapping to Chapter Docs

Main text	Appendix
`docs_en/00.md`	Appendix A
`docs_en/01.md`	Appendix B, F.2
`docs_en/02.md`	Appendix D, F.3
`docs_en/03.md`	F.4
`docs_en/04.md`	Appendix C, F.5
`docs_en/05.md`	F.6
`docs_en/06.md`	F.7
`docs_en/07.md`	F.8
`docs_en/08.md`	F.9
`docs_en/09.md`	F.10
`docs_en/10.md`	F.11

Reimu's Notebook (Appendix F)

When lost, run the chNN_ script for that chapter number.
Chapter 10 order: ch10_prepare_data → ch10_lora_train → ch10_gradio_app.
Finish all of them and you've cleared all 10 chapters!

Marisa's One More Thing

Jump to the main text with rg "Hands-on" docs_en/, ZE.

rg "Hands-on" docs_en/
ls scripts/ch*.py scripts/*.py

Thanks for making it this far. Take it easy!

ランキング参加中

プログラミング

github.com

Chapter 0: Introduction — How to Use This Book

0.1 Characters and Roles

0.2 The Big Picture of Hugging Face in This Book

0.3 Required Environment

0.3.1 Checking Python version

0.3.2 Virtual environment (venv)

0.3.3 Bulk install of packages used in this book

0.3.4 Environment check script

0.3.5 Using Google Colab

0.4 How to Run the Hands-ons

Step 1: Read

Step 2: Run

Step 3: Tweak

Step 4: Reflect

Hands-on 0 — Environment check + one-line inference

0.5 Mini Glossary

0.6 Repository Layout (Bundled with This Book)

Full Chapter Scripts

scripts/check_env.py

Reimu's Notebook

Marisa's One More Thing

On to the Next Chapter

Chapter 1: Hugging Face Hub — Let's Take a Peek

1.1 What Is the Hub? Why Does Everyone Use It?

1.2 Creating an Account and Issuing a Token

Logging in with the CLI

1.3 How to Read Model Cards and Dataset Cards

1.4 The Habit of Checking Licenses and Terms of Use

Hands-on 1 — Browse the Hub and Download

Hands-on 1-A — Find a model on the Hub and read the README

Hands-on 1-B — Log in with huggingface-cli login

Hands-on 1-C — Download a model locally and peek inside

Full Chapter Scripts

scripts/download_model.py

scripts/inspect_model.py

Reimu's Notebook

Marisa's One More Thing

On to the Next Chapter

Chapter 2: Pipeline — Experience "3-Line Inference"

2.1 Installing the Transformers Library

2.2 What Is pipeline? — Preprocessing Through Inference in One Place

2.3 Try Text Classification and Sentiment Analysis

2.4 Try Translation, Summarization, and Question Answering

Translation (EN → FR)

Summarization (English)

Question answering (extractive QA)

2.5 Image Classification and Object Detection (Vision)

2.6 Speech Recognition (Whisper and Others)

Hands-on 2 — Pipeline Practice

Hands-on 2-A — Japanese text sentiment analysis

Hands-on 2-B — Swap English–Japanese translation models

Hands-on 2-C — Compare accuracy and speed by swapping models on the same task

End of Chapter: Common Errors and Fixes

End of Chapter: Review Quiz (Reimu vs Marisa)

Full Chapter Scripts

scripts/ch02_pipeline_sentiment.py

scripts/ch02_pipeline_translation.py

scripts/ch02_pipeline_compare.py

scripts/ch02_pipeline_vision.py

scripts/ch02_pipeline_whisper.py

Reimu's Notebook

Marisa's One More Thing

On to the Next Chapter

Chapter 3 Tokenizer — Turning Strings into a Form Models Can Understand

3.1 Why Tokenization Is Needed

3.2 Basics of AutoTokenizer

3.3 Encode, Decode, and Special Tokens

3.4 Padding and Truncation

3.5 Batching and DataCollator

Hands-on 3 — Tokenizer in Practice

Hands-on 3-A — Compare the Same Sentence Across Multiple Model Tokenizers

Hands-on 3-B — Chunk Long Text and Run Batch Inference

Hands-on 3-C — Extend the Tokenizer by Adding Custom Vocabulary

Full Chapter Scripts

scripts/ch03_tokenizer_compare.py

scripts/ch03_tokenizer_batch.py

scripts/ch03_tokenizer_custom_vocab.py

Reimu's Notebook

Marisa's One More Thing

On to the Next Chapter

`scripts/check_env.py`

Hands-on 1-B — Log in with `huggingface-cli login`

`scripts/download_model.py`

`scripts/inspect_model.py`

2.2 What Is `pipeline`? — Preprocessing Through Inference in One Place

`scripts/ch02_pipeline_sentiment.py`

`scripts/ch02_pipeline_translation.py`

`scripts/ch02_pipeline_compare.py`

`scripts/ch02_pipeline_vision.py`

`scripts/ch02_pipeline_whisper.py`

Chapter 3　Tokenizer — Turning Strings into a Form Models Can Understand

3.2 Basics of `AutoTokenizer`

3.5 Batching and `DataCollator`

`scripts/ch03_tokenizer_compare.py`

`scripts/ch03_tokenizer_batch.py`

`scripts/ch03_tokenizer_custom_vocab.py`

Chapter 4　Model — A Peek Inside

4.1 Differences Among `AutoModel`, `AutoModelForCausalLM`, and Related Classes

4.2 Configuration (`config.json`) and Weights (`.safetensors` / `.bin`)

4.3 `device_map` and Choosing GPU vs CPU

4.4 Inference Mode (`model.eval()`) and Why You Disable Gradients

4.5 Basic Parameters for Generation (`generate`)

`scripts/ch04_forward.py`

`scripts/ch04_generate.py`

`scripts/ch04_quantize.py`

Chapter 5　Datasets — Preparing and Preprocessing Data

5.1 The Role of the `datasets` Library

5.3 `map` / `filter` / `train_test_split`

`scripts/ch05_load_dataset.py`

`scripts/ch05_preprocess.py`

`scripts/ch05_csv_to_dataset.py`

`scripts/data/ch05_sample_reviews.csv` (data file)

Chapter 6　Fine-Tuning with Trainer

6.2 Key `TrainingArguments` Options

6.3 Basic `Trainer` Flow (Train → Evaluate → Save)

`scripts/ch06_finetune.py`

`scripts/ch06_push_hub.py`

Chapter 7　PEFT / LoRA — Fine-Tuning with Less GPU

7.3 Using the `peft` Library

`scripts/ch07_lora_train.py`

`scripts/ch07_lora_save_adapter.py`

`scripts/ch07_lora_switch.py`

Chapter 8　Making Inference Faster, Cheaper, and Production-Ready

8.2 Overview of `transformers` Optimization Options