Pydantic model field: convert empty string to None

[natasha-aleksandrova]

It is very common for API to accept empty strings for query params as None, I'm trying to find a generic way to define fields to be coerced to None.

class MyQueryModel(BaseModel):
    my_field: Optional[int]

Right now above field would fail if empty string is passed as not a valid value for int. I'd like it to be treated as similar to Optional.

Is there anything built-in? is it possible to define a custom type something like Union[None, int, EmptyStrToNone]?

[PrettyWood]

Hi @natasha-aleksandrova
You can achieve what you want by either create a custom type

from typing import Optional, Union

from pydantic import BaseModel
from pydantic.validators import str_validator


def empty_to_none(v: str) -> Optional[str]:
    if v == '':
        return None
    return v


class EmptyStrToNone(str):
    @classmethod
    def __get_validators__(cls):
        yield str_validator
        yield empty_to_none


class M(BaseModel):
    x: Union[None, int, EmptyStrToNone]


assert M(x=None).x is None
assert M(x=0).x == 0
assert M(x='').x is None
assert M(x='q').x == 'q'

or add a validator

from typing import Union

from pydantic import BaseModel as PydanticBaseModel, validator


class BaseModel(PydanticBaseModel):
    @validator('*')
    def empty_str_to_none(cls, v):
        if v == '':
            return None
        return v


class M(BaseModel):
    x: Union[None, int, str]


assert M(x=None).x is None
assert M(x=0).x == 0
assert M(x='').x is None
assert M(x='q').x == 'q'

In the second case I created a custom BaseModel so that all empty string after validation will be considered as None.
It means if you set constr(strip_whitespace=True) and set a value ' ', it will still be considered as None. If you want to validate before validation, just set @validator('*', pre=True).
Hope it helps

[wiryonolau]

Hi how do you do this in pydantic2 ? since it no longer have pre key

[VanDuongEmpire]

@wiryonolau you would use mode="before" in your model or field validator decorator, e.g:

@model_validator(mode="before")
def empty_str_to_none(values):
  if values['x'] == '':
    values['x'] = None
  return values

More details about different validator modes in pydantic2: https://docs.pydantic.dev/latest/concepts/validators/#before-after-wrap-and-plain-validators

[wiryonolau]

I see, do I need to use model_validator instead of field_validator ?

[VanDuongEmpire]

@wiryonolau you can use either of them depending on your use case and preference, model_validator gives you access to all model properties while field_validator gives you access to the specified field only

[frederickch]

or add a validator

from typing import Union

from pydantic import BaseModel as PydanticBaseModel, validator


class BaseModel(PydanticBaseModel):
    @validator('*')
    def empty_str_to_none(cls, v):
        if v == '':
            return None
        return v


class M(BaseModel):
    x: Union[None, int, str]


assert M(x=None).x is None
assert M(x=0).x == 0
assert M(x='').x is None
assert M(x='q').x == 'q'

In the second case I created a custom BaseModel so that all empty string after validation will be considered as None. It means if you set constr(strip_whitespace=True) and set a value ' ', it will still be considered as None. If you want to validate before validation, just set @validator('*', pre=True). Hope it helps

In pydantic V2 @field_validator("*", mode="before") is incompatible with discriminated unions unless you use a callable Discriminator, so when migrating I used a model_validator:

class BaseModel(PydanticBaseModel):
    @model_validator(mode="before")
    @classmethod
    def empty_str_to_none(cls, data):
        if isinstance(data, dict):
            return {k: (None if v == '' else v for k, v in values.items()}
        return data

[Serialk89]

Hello ! I am using this way, I add this comment because today I searched for the same thing hahaha and I found another way to do it.

from pydantic import BaseModel, constr

class CustomerAddress(BaseModel):
    countrySubDivisionMajorName: constr(min_length=1)
    districtReference: constr(min_length=1)
    residentialStreetName: constr(min_length=1)
    residentialStreetNumber: constr(min_length=1)

[y2kbugger]

This is also semantically correct in many cases

[conradogarciaberrotaran]

constr is discouraged though, use Annotated with StringConstrains:

from constr docstrings:

    !!! warning "Discouraged"
        This function is **discouraged** in favor of using
        [`Annotated`](https://docs.python.org/3/library/typing.html#typing.Annotated) with
        [`StringConstraints`][pydantic.types.StringConstraints] instead.

        This function will be **deprecated** in Pydantic 3.0.

[y2kbugger]

Thanks!!

[y2kbugger]

I've found that using the BeforeValidator and Annotated to be a nice solution in my case.

See: https://docs.pydantic.dev/latest/api/functional_validators/#pydantic.functional_validators.BeforeValidator

from typing import Annotated, TypeAlias

from pydantic import BeforeValidator


def _empty_str_to_none(v: str | None) -> None:
    if v is None:
        return None
    if v == '':
        return None
    raise ValueError('Value is not empty') # Not str or None, Fall to next type. e.g. Decimal, or a non-empty str


EmptyStrToNone: TypeAlias = Annotated[None, BeforeValidator(_empty_str_to_none)]

Used like this:

class MyExampleModel(BaseModel):
    places: List[str]
    dates: List[dt.date]
    codes: List[str]
    prices: List[EmptyStrToNone | Decimal]

This solves static typechecking ala pyright/mypy as well as having Pydantic to the right thing at runtime. And can be used just as a type hint throughout codebase.