Best Books on Fine-Tuning & Small Language Models
Fine-tuning and small language models go from black box to controllable when you pair systems thinking with hands-on transformer workflows, from Chip Huyen’s production loop to Hugging Face fine-tuning practice in Tunstall et al.
Designing Machine Learning Systems
Chip Huyen
After finishing Designing Machine Learning Systems, you treat fine-tuning like a continuously monitored production experiment, not a one-time training run.
Build an evaluation loop before scaling model changes.
It reframes LLM adaptation as an engineering lifecycle: data, iteration, evaluation, observability, and safe deployment. That matters for fine-tuning small language models because small improvements can vanish without the right feedback loops.
Natural Language Processing with Transformers, Revised Edition
Lewis Tunstall, Leandro von Werra, Thomas Wolf
Natural Language Processing with Transformers turns transformer fine-tuning into repeatable Hugging Face recipes across real tasks, not vague theory.
Start with a baseline, then measure improvement with held-out evaluation.
You get practical workflows that translate directly to small language model fine-tuning: dataset handling, training setup, and task-specific adaptation. It is especially useful when you want the “how” behind prompt tuning, fine-tuning, and evaluation for NLP.
Generative AI on AWS
Chris Fregly, Antje Barth, Shelbee Eigenbrode
Generative AI on AWS connects LLM fine-tuning choices to deployment constraints like cost, latency, and managed infrastructure.
Parameter-efficient adaptation reduces operational burden in deployment.
If your goal is small language models, this helps you pick an adaptation path that survives real infrastructure limits. It also keeps you grounded in how training and serving design decisions affect the end results.
Hands-On Large Language Models
Jay Alammar, Maarten Grootendorst
Hands-On Large Language Models makes adaptation feel tangible by focusing on model behavior, evaluation, and practical iteration for real use cases.
Evaluation defines whether adaptation actually helped.
The emphasis on getting from model to working system supports fine-tuning decisions for smaller models, where evaluation and targeted iteration are critical. It is a strong bridge between conceptual understanding and applied practice.
Transformers for Natural Language Processing and Computer Vision
Denis Rothman
Transformers for Natural Language Processing and Computer Vision helps you connect transformer mechanisms to adaptation behavior across domains, not just text.
Domain task design matters as much as the model size.
When you fine-tune smaller models, architecture-level intuition helps you predict failure modes and choose realistic adaptation strategies. The accessible path also supports translating methods across tasks with different structure and noise.
Build a Large Language Model (from Scratch)
Sebastian Raschka
Build a Large Language Model (from Scratch) makes the small-model tradeoffs concrete by showing what each component contributes to learning and capacity.
Understand bottlenecks before deciding what to fine-tune.
To fine-tune small language models well, you need to understand what changes when you do not have huge scale: representational limits, optimization sensitivity, and tuning pressure. This clarifies the “why” behind adaptation choices.
Start with a baseline, then measure improvement with held-out evaluation.
Generative AI with LangChain
Ben Auffarth
Generative AI with LangChain pushes you to think in end-to-end workflows where adapting smaller open models plugs into applications cleanly.
Glue logic determines whether model changes matter.
Fine-tuning rarely ends at training: you need retrieval, orchestration, and evaluation in the loop. This helps you place small model adaptation in practical application patterns, so gains show up for users.
Machine Learning Engineering with Python
Andrew P. McMahon
Machine Learning Engineering with Python turns fine-tuning into a repeatable engineering practice: data pipelines, experiments, and deployment discipline.
Treat ML changes as experiments with measurable outcomes.
If you are working with small language models, the engineering fundamentals help you avoid “trained but not working” outcomes. This supports the operational side of tuning: tooling, testing, and system integration.
Speech and Language Processing
Dan Jurafsky
Speech and Language Processing (Dan Jurafsky) builds the mental model for language modeling that fine-tuning methods rely on, especially around tokens and probabilities.
Understand tokenization and objectives before tuning.
Even if you are focused on small language models today, the conceptual base in language modeling makes it easier to interpret what fine-tuning is optimizing. It is a useful reference when you want to connect classical NLP ideas to modern adaptation.
Can we tailor this list for you?
Type your question in the bar below and the AI will tailor a fresh set of picks just for you.