Fully migrate to Pydantic v2

[achille]

After updating to the latest version llm commandline fails
with: ImportError: cannot import name 'field_validator' from 'pydantic

Log:

% pip install llm
Requirement already satisfied: llm in /opt/homebrew/lib/python3.10/site-packages (0.14)
Requirement already satisfied: click in /opt/homebrew/lib/python3.10/site-packages (from llm) (8.1.3)
Requirement already satisfied: openai>=1.0 in /opt/homebrew/lib/python3.10/site-packages (from llm) (1.30.5)
Requirement already satisfied: click-default-group>=1.2.3 in /opt/homebrew/lib/python3.10/site-packages (from llm) (1.2.4)
Requirement already satisfied: sqlite-utils>=3.35.0 in /opt/homebrew/lib/python3.10/site-packages (from llm) (3.36)
Requirement already satisfied: sqlite-migrate>=0.1a2 in /opt/homebrew/lib/python3.10/site-packages (from llm) (0.1b0)
Requirement already satisfied: pydantic>=1.10.2 in /opt/homebrew/lib/python3.10/site-packages (from llm) (1.10.6)
Requirement already satisfied: PyYAML in /opt/homebrew/lib/python3.10/site-packages (from llm) (6.0)
Requirement already satisfied: pluggy in /opt/homebrew/lib/python3.10/site-packages (from llm) (1.3.0)
Requirement already satisfied: python-ulid in /opt/homebrew/lib/python3.10/site-packages (from llm) (2.2.0)
Requirement already satisfied: setuptools in /opt/homebrew/lib/python3.10/site-packages (from llm) (69.2.0)
Requirement already satisfied: pip in /opt/homebrew/lib/python3.10/site-packages (from llm) (24.0)
Requirement already satisfied: anyio<5,>=3.5.0 in /opt/homebrew/lib/python3.10/site-packages (from openai>=1.0->llm) (3.6.2)
Requirement already satisfied: distro<2,>=1.7.0 in /opt/homebrew/lib/python3.10/site-packages (from openai>=1.0->llm) (1.9.0)
Requirement already satisfied: httpx<1,>=0.23.0 in /opt/homebrew/lib/python3.10/site-packages (from openai>=1.0->llm) (0.24.1)
Requirement already satisfied: sniffio in /opt/homebrew/lib/python3.10/site-packages (from openai>=1.0->llm) (1.3.0)
Requirement already satisfied: tqdm>4 in /opt/homebrew/lib/python3.10/site-packages (from openai>=1.0->llm) (4.65.0)
Requirement already satisfied: typing-extensions<5,>=4.7 in /opt/homebrew/lib/python3.10/site-packages (from openai>=1.0->llm) (4.12.0)
Requirement already satisfied: sqlite-fts4 in /opt/homebrew/lib/python3.10/site-packages (from sqlite-utils>=3.35.0->llm) (1.0.3)
Requirement already satisfied: tabulate in /opt/homebrew/lib/python3.10/site-packages (from sqlite-utils>=3.35.0->llm) (0.9.0)
Requirement already satisfied: python-dateutil in /opt/homebrew/lib/python3.10/site-packages (from sqlite-utils>=3.35.0->llm) (2.8.2)
Requirement already satisfied: idna>=2.8 in /opt/homebrew/lib/python3.10/site-packages (from anyio<5,>=3.5.0->openai>=1.0->llm) (3.4)
Requirement already satisfied: certifi in /opt/homebrew/lib/python3.10/site-packages (from httpx<1,>=0.23.0->openai>=1.0->llm) (2022.12.7)
Requirement already satisfied: httpcore<0.18.0,>=0.15.0 in /opt/homebrew/lib/python3.10/site-packages (from httpx<1,>=0.23.0->openai>=1.0->llm) (0.17.3)
Requirement already satisfied: six>=1.5 in /opt/homebrew/lib/python3.10/site-packages (from python-dateutil->sqlite-utils>=3.35.0->llm) (1.16.0)
Requirement already satisfied: h11<0.15,>=0.13 in /opt/homebrew/lib/python3.10/site-packages (from httpcore<0.18.0,>=0.15.0->httpx<1,>=0.23.0->openai>=1.0->llm) (0.14.0)

[notice] A new release of pip is available: 24.0 -> 24.1
[notice] To update, run: python3.10 -m pip install --upgrade pip

% llm help
Traceback (most recent call last):
File "/opt/homebrew/bin/llm", line 5, in
from llm.cli import cli
File "/opt/homebrew/lib/python3.10/site-packages/llm/init.py", line 18, in
from .plugins import pm
File "/opt/homebrew/lib/python3.10/site-packages/llm/plugins.py", line 17, in
pm.load_setuptools_entrypoints("llm")
File "/opt/homebrew/lib/python3.10/site-packages/pluggy/_manager.py", line 398, in load_setuptools_entrypoints
plugin = ep.load()
File "/opt/homebrew/Cellar/python@3.10/3.10.14/Frameworks/Python.framework/Versions/3.10/lib/python3.10/importlib/metadata/init.py", line 171, in load
module = import_module(match.group('module'))
File "/opt/homebrew/Cellar/python@3.10/3.10.14/Frameworks/Python.framework/Versions/3.10/lib/python3.10/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/opt/homebrew/lib/python3.10/site-packages/llm_claude_3.py", line 3, in
from pydantic import Field, field_validator, model_validator
ImportError: cannot import name 'field_validator' from 'pydantic' (/opt/homebrew/lib/python3.10/site-packages/pydantic/init.cpython-310-darwin.so)

[cmungall]

It looks like the llm-claude-3 plugin assumes that pydantic2 is being used.

This could be fixed by making the imports defensive here:
https://github.com/simonw/llm-claude-3/blob/main/llm_claude_3.py

See my PR

• Allow for use of Pydantic v1 as well as v2. #147 for how to do this

Then we'd also want to extend the testing strategy see #169

However, I think the most straightforward thing to do is drop support for pydantic1 and force a pydantic2 install - what do you think @simonw? It's been over a year since pydantic2 has been out

[simonw]

Yeah, it's time. I continued supporting both because I was worried about LLM needing to run in environments that were committed to Pydantic v1 - but as of Feb 2025 I'm OK dropping that requirement.

Users who really want to stick with Pydantic v1 can pin to a version of LLM <=0.22.

[simonw]

I spent $1 on the new Claude Code tool getting it to implement this upgrade for me, which seemed to work. PR incoming.

https://gist.github.com/simonw/e14f5217c85f0557427fe3f3d7750f03

[simonw]

Trying a different approach: https://docs.pydantic.dev/latest/migration/#code-transformation-tool

uvx bump-pydantic llm

That didn't cost $1 and gave me this diff:

diff --git a/llm/models.py b/llm/models.py
index c7e3c72..bb5300d 100644
--- a/llm/models.py
+++ b/llm/models.py
@@ -23,7 +23,7 @@ from typing import (
 from .utils import mimetype_from_path, mimetype_from_string, token_usage_string
 from abc import ABC, abstractmethod
 import json
-from pydantic import BaseModel
+from pydantic import ConfigDict, BaseModel
 from ulid import ULID
 
 CONVERSATION_NAME_LENGTH = 32
@@ -618,10 +618,7 @@ class AsyncResponse(_BaseResponse):
 
 
 class Options(BaseModel):
-    # Note: using pydantic v1 style Configs,
-    # these are also compatible with pydantic v2
-    class Config:
-        extra = "forbid"
+    model_config = ConfigDict(extra="forbid")
 
 
 _Options = Options
diff --git a/llm/templates.py b/llm/templates.py
index 0cf1616..23aa2b6 100644
--- a/llm/templates.py
+++ b/llm/templates.py
@@ -1,4 +1,4 @@
-from pydantic import BaseModel
+from pydantic import ConfigDict, BaseModel
 import string
 from typing import Optional, Any, Dict, List, Tuple
 
@@ -12,9 +12,7 @@ class Template(BaseModel):
     # Should a fenced code block be extracted?
     extract: Optional[bool] = None
     extract_last: Optional[bool] = None
-
-    class Config:
-        extra = "forbid"
+    model_config = ConfigDict(extra="forbid")
 
     class MissingVariables(Exception):
         pass