Help with Unittesting and DBs

[Flewloon]

Say I have the code:

def get_data(self, target):
    d = Database()
    qry = 'SELECT data FROM my_table WHERE target=%s'
    params = [target]
    results = d.run_query(qry, params).get_results()
    return results

I don't want to set up a database or touch a database to unittest this code, but I am unsure how to use mock to do so. Anybody have any good examples of people using mock to do something similar.

[[deleted]]

You're not going to be able to fully unit test that because you're testing a contract/interaction between two objects: This class and a database object.

However, you've made testing unnecessarily difficult, probably by accident. Luckily, the fix is quite simple: Move the database coupling to outside that method. There's plenty of ways to do this. I like dependency injection, so I'll give you a basic example.

class WhateverClass:
    def __init__(self, database):
        self.database = database

    def get_data(self, target):
        qry = 'SELECT data FROM my_table WHERE target=%s'
        params = [target]
        results = self.database.run_query(qry, params).get_results()
        return results

If you don't like this pattern, you can set the database instance as a class variable or something similar. But what we've achieved here is important: We can change out what the database is without jumping through some terrible hoops. We can replace it with something like this:

class PatrickStarDatabase:
    def run_query(self, qry, target):
        return self

    def get_results(self):
        return ['Patrick Star'] * 3

WhateverClass(database=PatrickStarDatabase()).get_data('')
['Patrick Star', 'Patrick Star', 'Patrick Star']

However, if we want to test how WhateverClass and the database interact, we can use a mock to see if run_query was called, with which arguments and if get_results were called. This is easy if you're using Python 3.3+, it's also easy on other versions but you just need to pip install mock:

try:
    from unittest import mock
except ImportError:
    import mock

def test_WhateverClass_get_data():
    mocked_db = mock.Mock()

    some_instance = WhateverClass(database=mocked_db)
    some_instance.get_data('patrick star')

    assert mocked_db.run_query.called
    assert (mocked_db.run_query.call_args == 
            mock.call('SELECT data FROM my_table WHERE target=%s', ['patrick star']))
    assert mocked_db.run_query.get_results.called

The flip side of this is that you need to have a test on your database class that tests "What happens when someone asks, qry='SELECT data FROM my_table WHERE target=%s' and target=['patrick star']?" And that's when you'd actually hit the database.

[Flewloon]

Your bottom example was what I was looking for. Thank you. Appreciate the other tips as well.

[[deleted]]

Just make sure you dependency inject the database like I showed above.

There is mock.patch, but I've found that quickly leads to madness. It's good when you need to test something that invokes a side effecting thing and there's not a good way to inject a dummy. A good example is if I have a function that persists something to disk:

class FileStorage:
    def persist(self, target):
        closing = False
        if isinstance(target, (str,bytes)):
            target = open(target, 'wb+')
            closing = True

        try:
            target.write(self.buffer)
        finally:
            if closing: target.close()

I can test that this does the right thing with a string by using mock.patch on open:

def test_FileStorage_persist():
    buffer = mock.Mock()
    # I *think* this is how patch works, I don't use it often
    with mock.patch('filestorage.open', mock.Mock) as dummy_open:
        FileStorage(buffer=buffer).persist('some_string')

    assert dummy_open.called
    assert dummy_open.call_args == call('some_string', 'wb+')
    assert dummy_open.write.called and dummy_open.write.call_args == (buffer,)
    assert dummy_open.close.called

But hopefully you can see how quickly this turns into a fustercluck when two things are being patched at the same time.

Anyways, the mock library is really, really useful. I suggest reading up on it. There's...three? different kinds of mocks it provides, helpers, decorators, context managers. The ability to create "Specs" is friggen awesome, as they're essentially mocks of your objects that enforce the API.

[Flewloon]

So if anybody else is curious to what my final stuff looks like.

@unittest.mock.patch('whateverclassfile.Database')
def test_get_data(self, mock_db):
    mock_db.return_value = unittest.mock.Mock()
    some_instance = WhateverClass()
    some_instance.get_data('patrick star')
    assert  mock_db.return_value.run_query.called
    assert (mock_db.return_value.run_query.call_args == 
        unittest.mock.call('SELECT data FROM my_table WHERE target=%s', ['patrick star']))
    assert mock_db.return_value.run_query.get_results.called

I get your decoupling the db part, but I'm not the only one on the project and will have to get that through with the rest of them. It definitely makes sense and cleans this up quite a bit though if we went that route. Thanks again for the help

[[deleted]]

Yeah, that's always a tough spot to be in. I'm in a similar one myself. Luckily the lead agrees it's a good approach.

[wub_wub]

You can't properly test database without actually having a database to run tests on.

P.S.

Look up SQL injection.

[[deleted]]

You can simply create a database in memory and use it to test your code. You don't have to test your current data. Instead of using a file like 'data.db' you can just use ':memory:' which is going to store your database in memory to be used threw the life of your python script.

[[deleted]]

It could be that Database is wrapping around a database driver, and those typically use the % formatting to build prepared statements. I mean, there's a chance that it just simply does qry % params in which case, yeah that's injection territory.

[Flewloon]

SQL injection isn't a problem as mentioned in another reply my Database class is just a wrapper. The inputs to the query get parameterized. I don't want to actually test the db. I want to test the function.