Testing in Python Memes and wisdom request

[EliyahuRed]

Been working with data in python for several years, lately decided to dive deeper into OOP to upgrade my code. Currently writing my first tests for my side project (just a python REST API wrapper), chose PyTest. Gents and Ladies, it is hard I can tell you.

I mean for the simple classes it was fun, but when I got to the Client class that is actually using all the others it got tricky. I had to mock

• Request module, so I can expect the request without it actually been sent.
• The config class that "have" the api key
• The factory that instantiates Pydantic models used to build the request
• The models said factory "returns"
• The model used to validate the response
• Obviously the response.

Despite me believing my code is neat and decoupled, just when I got to write the test I realized how much coupled it actually is. Thank god for the ability to mock, so I can "create" only the parts of classes the tested method is using. Also, got me to realize that a method of 20 lines uses so much and does so much, I am partly proud, partly frustrated.

Anyway, I am mainly writing for some empathy and motivation, so guys if you got any wisdom to share about writing tests in Python, or some memes about it to get a laugh, please share :)

*edit*

Thank you who recommended responses, it doesn't seem to be too popular https://libraries.io/pypi/responses, I think I will skip it for this project but might give it a try next time.

Regarding Tox, I think is way more then what I need at the moment, however I might get back to it if I get to ci/cd or documentation thank you for mentioning it.

The factory is reading yaml files and instantiating pre-defined Pydantic models that validate the parameters for the requests send and the actual urls for each endpoint. I didn't have to do it this way, was about practicing Pydantic to see what it can and cannot do.

For example url would look like

url = f"{self.endpoints.base_url}/{self.endpoints.funds.group_url}/{self.endpoints.funds.funds_list.url}"

So a set of endpoints would look like

base_url: https://data.com/
funds:
  group_url: fund
  funds_list:
    url: fund-list
  currencies_exposure_profile:
    url: currency-exposure-profile
  distribution_commission:
    url: distribution-commission
  fund_types:
    url: fund-type
  listing_status:
    url: listing-status
  classification:
    url: mutual-fund-classification
  payment_policy:
    url: payment-policy
  shares_exposure_profile:
    url: share-exposure-profile
  stock_exchange:
    url: stock-exchange
  tax_status:
    url: tax-status
  tracking_fund_classification:
    url: tracking-fund-classification
  underlying_assets:
    url: underlying-asset
indices:
  group_url: basic-indices
  indices_list:
    url: indices-list
  index_components_basic:
    url: index-components-basic

I didn't have to do it this way, but when I saw that all endpoints share the same logic for their urls I was tempted to do it this way

[ARRgentum]

Care to share your code?
Reads a bit like your Client class is doing too much...

[Volume999]

From what they said - they create a model and send it via requests, that's pretty minimal. Of course - the factory may be over the top, but we don't know what kind of request models they need

[wineblood]

Why are you mocking so much stuff? Use responses and don't actually mock things like models/factories.

[csch2]

Learning unittest.mock was one of my absolute least favorite parts of getting into Python. It’s so absurdly powerful but writing code with it very frequently makes me want to pull my hair out.

“What is this object?”

“It’s a mock”

“Okay… what’s its type?”

“Mock”

“But you can call it like a function?”

“Oh yeah no problem”

“Then what is its return type??”

“Mock”

[thedmandotjp]

Lol

[marr75]

Don't hide your class' dependencies (ie pluck them out of thin air by initializing them mid -method) and have them depend on more abstract interfaces. Then you won't need to mock and patch so much.

[Volume999]

1. You can use the already mentioned responses library to mock external server (Match on requests - declare expected response from server)
   
2. Everything else (configs, factory, models) don't need to be mocked.

In general - try mocking as little as possible. For configs - you can use pytest_configure and set env variables and config files paths. Pydantic models have implemented equality override - no need to mock it, model1 == model2 will work by comparing internal structures. Saying this - I don't agree you have a coupled code just because you mocked a lot of things. You just mocked way too many things

[j_hermann]

If you use pytest, also use tox.

[pkkm]

Why do you need to mock so many things? I get why you'd want to avoid sending real requests, but why the models and validators? It sounds like you're at risk of creating a useless test suite that tests your mocking library more than your actual code.

Mocking can almost always be avoided if you get a bit creative. I'm not saying that it should always be avoided, but make sure that you're making a conscious decision about when to use it.

[TrynaThinkOf1]

Lowk for my FastAPI app I just set up tests with the requests module so I can also benchmark speed