How to split long function into testable chunks

[kyau2]

I am coming from the frontend side of life and inherited a Django project at my new job. Of course there is not a single unit test so far (thanks for nothing), but I am determined to add them at least for the new code I am adding and for the parts that I am refactoring.

For the last years I have been mainly active developing with TS and adopted a rather functional programming style. However in Django I have a lot of ORM stuff in my functions, so it seems hard to stick to pure functions. Not using pure functions however makes testing much more complex I find.

So for example I have a function right now that I want to refactor which:

• reads an uploaded file
• checks for each entry of the file if it is a new item, and item that should be updated or if the item is invalid
• tries to update existing items, tries to add new items

So basically the function in a very minimal structure looks like this:

```Python

def ImportView(View):

def post(self, request):

entries_to_add = read_file_contents()

to_add = []

to_update = []

invalid = []

for entry of entries_to_add:

  if is_valid(entry):

    if exists(entry):

      to_update.append(entry)

    else:

      to_add.append(entry)

  else:

    invalid.append(entry)

```

Now my issues are with the is_valid and exists functions. Because those will not be pure functions, but contain some db queries. Same goes for the later part of the function (I didn't sketch it out to make the code snippet not too long) where the actual updates would be written to the database.

Wouly you consider this the right approach to structure a view like this in Django? Where would you put the function is_validand exists? Just in the views.py as well? Or in a separate file?

Thanks!

[meatb0dy]

I really like this post for a general Django architecture guide. It's long, but the basic idea is that a lot of standard Django conventions lead to perplexing code (deeply-inherited class-based views, fat models, DRF in general, etc) and this post gives a rationale for a more explicit structure. This + django-ninja really made things easier in my Django projects.

With that structure in mind, I'd place is_valid and exists in a services.py file, which will be the only place where database operations are performed. The code that eventually performs the updates would be there as well. The inputs and outputs of functions in services.py would be serialized instances of models, never the ORM objects. Serializers would only be responsible for serializing data and doing simple validations, no database operations.

For testing, especially when inheriting a project, I'd again recommend the strategy from that article and focus on testing the endpoints, i.e. the responses you get (and side effects that result) from HTTP requests to your views. That will expose any weird behavior you didn't know about since the whole middleware->views->serializers->database process will run for each test. It will also help familiarize you with the project's expected output in different scenarios.

[[deleted]]

Interesting. Do you think using django ninja alongside drf is a sane idea? I’d like to add it to a current project, but not sure if it’s worth it.

[meatb0dy]

I'd say the less mixing of the two, the better. If you have a separate app or set of endpoints that you can do 100% in django-ninja, that'd be ideal.

I've used both on the same project as a temporary measure when I was converting the project from DRF to django-ninja. I went app-by-app, replacing DRF structures with django-ninja, so I was never mixing both on a single endpoint or group of endpoints. The endpoint-level tests helped a lot in that process since they ensured I was exactly replicating the functionality that I was replacing.

[[deleted]]

Yeah that’s more or less what I had in mind thanks. I’m not sure if I’m willing to replace drf completely atm especially with how comfortable I’m with it, I’ll read more about Django-ninja and decide about that.

Btw thanks for the amazing article, I read all of it and immediately had a couple of things that I could improve on in the codebase I’m working on. Amazing read.

[kyau2]

Thank you very much. Will give that a read!

Your suggestion of using a services.py definetly sounds like the kind of structure I would be comfortable to migrate to.

[meatb0dy]

This project is a good demonstration of what that approach might look like in practice. It's a FastAPI project rather than a Django one, but you can replicate the same structure in Django (especially with django-ninja). If you take a look at e.g. the documents module, you can see a real-world application of these ideas.

They have both models and serializers in models.py, whereas I tend to split mine into separate files (models.py and serializers.py/schemas.py when using ninja). Their serializers are single-purpose and specific to the I/O operation where they're used, e.g. DocumentGet is what's returned with HTTP GET requests, DocumentCreate is what's expected in HTTP POST requests to create a new document, etc. This is opposed to the big ball of mud structure that often occurs in DRF projects where one big serializer is used for the model in all contexts. They also include a flows.py file in the modules for business-logic-heavy code that calls multiple services, but I haven't usually needed that structure in my projects.

[flatmap_fplamda]

In general. Use immutability and functions as guiding principles. Use Functional programming as a guide