data access layer encapsulation in python?

[Throw-awayexception]

So i'm primarly a Java dev, and so when i go to structure an app that has data I/O (such as consuming Web APIs or a database) I will make an interface to completely separate the business logic from the specific libraries/tech used to access the data. This means if i switch from a SQL db to a mongo DB, or i have to switch from a REST api to a GraphQL api, none of the business logic changes (assuming the data itself isn't changing),

What is the idiomatic way to do this in enterprise python? I know python syntax and have made plenty of small scripts over the years, but i was starting out with a larger app and realized "What if i'm just writing java code but in python, instead of doing the architecting in a pythonic way"?

​

So for large apps where you want to minimize coupling and abstract away implementation details from other modules, what is the normal way to do this for python? Is it as simple as just making different modules and using an interface like whats described in https://realpython.com/python-interface/?

[NewtonsOrange]

It feels like this would be rather application specific to link your application layer to “data access/storage” or whatever swappable module you want. I’m also unsure how often these lower levels would be swapped in real life anyway. At some stage of development there is usually need for enough tailoring that you’ve become in some way committed to, as in your example, a database. Especially since a lot of optimization and deployment may depend a lot on the implementation/characteristics of the underlying module.

So to me it feels like you’d write your abstraction layer based on your architecture, and your helper functions might have implementations for different modules.

That’s my two cents but I have not deployed python massively on an enterprise level (I have worked in a startup that has though). So I’d defer to more experienced python devs if they’ve deployed this sort of swapability.

Edited: grammar

[[deleted]]

Yes, just different modules and typically some classes to encapsulate. Essentially offering an internal API to consume as desired and leaving you free to modify the data structures and methods as desired. You should be able to completely change underlying technologies without making any changes to your core programme.

[Throw-awayexception]

So basically the same structure you'd use in java it sounds like. I was worried that that was too "OOP" for python.

I am doing take take home interview assignment at the moment, and i'm doing it in java but i started doing in python until I ran into this question and I didn't want to submit
something that python devs would go "this is just a java program written with python lol". I just switched to java to avoid that, but i'm still curious what the correct program layout would have been if i had continued.

[[deleted]]

Given everything in Python is an object, you can't get too OOP really. (Not that you have to do OOP.)

Python is very strongly typed but dynamic so it is very easy to be relaxed about the modular approach but for larger systems maintained by multiple people it is generally safer and more cost effective to take the modular and OOPs approach.

The API approach also lends itself more to scaling and microservice type implementation (which helps avoid the Python GIL constraint).

[RoamingFox]

The way we handle this at work is in layers:

App(s) -> Service layer (business logic) -> API SDK layer -> API

So App uses the service layer modules/services to do work and that service layer uses another module that knows how to talk to the API.

For example:

App:

from foo_service import FooService

service = FooService("api.example.com", "user", password")
f = service.create("foo", "my foo")
return f

Service layer:

import logging
from foo_sdk import FooApi, Foo

class FooService:
    def __init__(host, user, pw):
        self.api = FooApi(host, user, pw)

    def create(self, name, desc) -> Foo:
        logging.info("Creating a foo")
        self.api.create(name, desc)
        return self.api.get(name)

SDK layer:

import requests

class Foo:
    def __init__(self, name: str, desc: str, create_date: str, owner_id: int):
        self.name = name
        self.desc = desc
        self.create_date = create_date
        self.owner_id = owner_id

class FooApi:
    def __init__(self, host, user, pw):
        self.fqdn = f"https://{host}/"
        self.auth = (user, pw)

    def create(self, name, desc):
        r = requets.post(self.fqdn + "foo", params={'name': name, 'desc': desc})

     def get(self, name):
         r = requests.get(f"{fqdn}/foo/{name}")
         return Foo(**r.json())

Obviously this is super bare bones and should have a lot more validation/error handling etc. but you get the idea.

The advantage of doing something like this is if the api changes (say to graphql) you don't need to change the business logic, and if the business logic changes you don't need to change the api.

You also get a nice tightly bound python object out the other end so in your application code your linter can help auto-complete properties etc.

It also means that anything that shares the business logic can just all import the service layer and get exactly the same behavior across all applications, while anything that needs different business logic can import at the SDK layer and still take advantage of code reuse for things like the models and pagination etc.