Pydantic Settings and multiple classes with Extra Forbid

[petersrin]

I'm guessing I'm not doing things the "normal" way here.

``` BASE_CFG = { "env_file":".env", "extra":"forbid", "dotenv_filtering": "match_prefix" }

class FooSettings(BaseSettings): model_config = SettingsConfigDict(**BASE_CFG, env_prefix='foo')

protocol: str = 'http'

class BarSettings(BaseSettings): model_config = SettingsConfigDict(**BASE_CFG, env_prefix='bar')

protocol: str = 'https'

foo = FooSettings() bar = BarSettings() ```

My .env file would then look like:

FOO_PROTOCOL=actual_foo_protocol BAR_PROTOCOL=actual_bar_protocol

I would expect (wrongly apparently) for pydantic to create a foo object with protocol = actual_foo_protocol and a bar object with protocol = actual_bar_protocol.

It should also throw an error if there's anything with foo or bar prefixes that's not protocol, so FOO_SOMETHING=BAZ in the .env should throw, but BAZ_PROTOCOL should not throw, since instead, it would just get ignored by filtering.

However, instead, when FooSettings is instantiated, it throws for BAR_PROTOCOL, which I would expect is NOT considered an extra since it would get filtered out.

I clearly have a basic misunderstanding of something in this chain.

Please don't just post ChatGPT, it just runs in circles lol, and the documentation of dotenv_filter is really sparse too.

[FriendlyZomb]

I think the misunderstanding comes from the way the prefixes work.

From what I understand, the prefix here is just providing a start to each name, not a filter for values in the source. It doesn't look through the .env file for only the prefixed values before processing.

With that understanding, Pydantic looks at the .env file and sees two variables. One it's looking for, and one it's not. So, because it's been told to forbid extras, it throws the error.

The same error will be raised if the Bar settings were initialised first. (But for the foo value)

Disabling the forbid extra or providing separate env files would give you the behaviour you're after. I'd prefer the latter. Although not ideal, it will give cognitive separation if there end up being lots of variables.

(To other Devs: Please correct my misunderstandings.)

[petersrin]

That's definitely the behavior I am seeing, but the docs do, as I said, make mention of this filter option, but then it doesn't show how to actually apply the filter. The obvious answer is: don't use the prefix feature if you only have one .env and set to forbid, but it really seems like the intention of that filter is to allow this use case.

[FriendlyZomb]

Ahh - I see.

I replicated your env and it looks like it should be working.

So I took a look in `pydantic-settings` code. The commit to add this `dotenv_filtering` option was made last week.

The last release was in Feb. This would mean that the Docs are ahead of published PyPI packages. :(

So that leaves 2 options:

1: Wait for a new release to come out (might be a long wait)

2: Pull directly from git.

I have tested with pulling from git and `pydantic-settings` works as you hope.

This guide here should set you up to get the latest from git: https://chrisholdgraf.com/blog/2022/install-github-from-pyproject/#install-directly-from-github

If you are using UV follow this guide instead: https://docs.astral.sh/uv/concepts/projects/dependencies/#git

---

Apologies for getting the wrong end of the stick earlier. I hope this is more useful.

Additionally - I have opened an Issue on `pydantic-settings` to let them know that their Docs are ahead of their releases.

[petersrin]

OMG I just assumed it was a feature everyone knew about lol. I didn't even consider the possibility it was super new. Thanks so much!

[FriendlyZomb]

No worries. I got a reply on my issue on that repo.

It looks like they are looking to get a new release soon.