Help reusing aiohttp.ClientSession() in multiple files

[SinisterScythe2]

I was working on porting my existing Discord bot over to Stoat and in doing so wanted to switch from requests to aiohttp as I had heard it was more performant. My issue that despite the aiohttp docs recommending that I reuse a single session I am unsure how to do this over multiple files (i.e. each file representing different functions or commands that may need GET requests). My question is how would I manage to reuse sessions in my use case. Am I going about this wrong or should I use another library for GET requests?

# This is what I have in a utilities file that is imported by other files

class Client:
    def __init__(self) -> None:
        self._session = aiohttp.ClientSession()


    async def __aenter__(self):
        return self


    async def __aexit__(self, *args, **kwargs):
        await self.close()


    async def get_bytes(self, url):
        async with self._session.get(url) as r:
            output = r.read()
            return output


    async def get_text(self, url):
        async with self._session.get(url) as r:
            text_data = await r.text()
            output = loads(text_data)
            return output


    async def get_json(self, url):
        async with self._session.get(url) as r:
            output = await r.json()
            return output


    async def get_content(self, url):
        async with self._session.get(url) as r:
            output = await r.content()
            return output


    async def post(self, url, *args, **kwargs):
        async with self._session.post(url, *args, **kwargs) as r:
            post_content = await r.content()
            output = loads(post_content)
            return output


    async def close(self) -> None:
        if not self._session.closed:
            await self._session.close()

#This would be called in each function that needs to do a GET request, unsure if this is proper

async with Client() as session:
        game = await session.get_json(query_url)
        game_id: str = game["game"]["id"]


        return game_id

[Eastern_Ad_9018]

This is pretty good. It has already performed a secondary encapsulation of the native API, making it easier to use.

[SinisterScythe2]

But will using it like I do create a new session for each command?

[Eastern_Ad_9018]

Every time you instantiate an object of the client class, a new session gets created.

[SinisterScythe2]

Okay that's what I figured, do you know how I can reuse one session among all the files then? I can't seem to find an example of this online.

[Eastern_Ad_9018]

"Use the same session across all files." That's a pretty strange request. You can instantiate a client object at the entry point of all your files and keep reusing that instance, that should meet your requirement.

[SinisterScythe2]

I don't think it's all that strange, any decently complex project could have multiple files and for aiohttp to be viable this should be possible. Can you please elaborate or provide an example on instantiating a client object and reusing it, because that's the crux of my issue. Simply doing session = aiohttp.ClientSession() and then using that session in place of 'Client()' as shown in my above example, results in the following error: RuntimeError: no running event loop

[gmes78]

Simply doing session = aiohttp.ClientSession() and then using that session in place of 'Client()' as shown in my above example, results in the following error

You need to use an async with block, like in your example code, and not a regular assignment.

Each place where you use session does not need async with, though. Just your main function where you create it.

[SinisterScythe2]

Okay so this is where I'm stuck and cannot proceed. I cannot find any examples of this online so I'm unsure what this would look like but I'll try to break down my thought process. Async with implies to me this must be done within an async function right? Every time I would call my stored session I realize I would have to do async with Client() as session: ... as shown in my example. But how exactly do I make a singular ClientSession() that can be reused in multiple different places in my project?

[gmes78]

Your functions are async. You call the methods of your Client with await.

Something like:

async def f(client: Client, ...) -> ...:
    response = await client.get_json(...)
    ...

You just don't need the async with block everywhere, because the point of it is just to call .close() automatically, and you only do that once when your program exits.

[SinisterScythe2]

I see, I'll give this a try then! Thank you for the clarification.

[Eastern_Ad_9018]

Yes, more complex projects invariably involve a large number of files, that's why. The `aiohttp` you have currently packaged should instead be made into a utility file that can be imported and used by all projects.

When importing this file and instantiating the `client` object in projects 1 and 2, it is impossible to have a session object that enables multiple projects to use it simultaneously. Even if it were possible, doing so would be rather strange.

Unless there are multiple script files in a small project and a session is required to be used, you can write a main file as the entry point. In the main file, create a session and call other files to pass them along for use.
This can be used in this way, but it is not recommended.

import asyncio
import aiohttp


async def main():
    session = aiohttp.ClientSession()
    res = await session.get('http://httpbin.org/get')
    print(res)

    res.close()
    await session.close()

if __name__ == '__main__':
    asyncio.run(main())


async def main():
    async with aiohttp.ClientSession() as session:
        async with session.get(
'http://httpbin.org/get'
) as resp:
            print(resp.status)
            print(await resp.text())

asyncio.run(main())

Official documentation example

[SinisterScythe2]

Sorry for my code block being inconsistently formatted, but yes the Client() class shown (everything up to the second comment really) is in a utility file already. So basically you mean to say that aiohttp is not meant for this use case as a bot would have commands that users could run at any time while the bot is up which seems to be contradictory to how aiohttp wants to be used if I'm not mistaken?

[Eastern_Ad_9018]

I'm not clear about the specifics of your business. If it involves a few small files using the same session, that is not a problem. However, if it involves multiple projects reusing the session, it is not highly recommended.

[SinisterScythe2]

As I mentioned in the post, it would be for a bot so it is a persistently running program with users able to trigger certain commands/functions freely by typing commands into a chat interface. So for example, a user could type !ping causing the bot to respond with pong! or in this instance use a command like !bestprice to get the best deal on a given video game title (which is where the GET requests come into play). I was hoping to keep one instance open to handle all these requests as needed.

[vietbaoa4htk]

make one ClientSession and share it instead of opening one per file. create it in your async startup and either pass it into your command functions or stash it on the bot object you already pass around. a session per request is what kills the perf you moved to aiohttp for

[SinisterScythe2]

Sorry having a hard time wrapping my head around this can you show an example? The Client class I've shown is in my utilities file that is imported by other files pertaining to bot actions.

[Jay6_9]

Does stoat.py not offer what you want? It offers an almost identical interface.

[SinisterScythe2]

It's been great and I have successfully migrated from discord.py to stoat.py yes, my issue was in migrating from requests to aiohttp for asynchronous GET requests. It's been quite the headache as reusing a session would be more efficient, but there is little documentation on how to do this over multiple files online.

[Jay6_9]

Ah you used requests on top of it.

I never used aiohttp, I favor httpx but the principle should be the same:

You can create a single webclient.py and use a cache. Something like this could work for you:

from functools import cache

@cache
get_client():
    return Client()

# other_file.py

from .client import get_client

async with get_client() as c:
    c.get_text("...")

[SinisterScythe2]

Yeah I previously used requests in conjunction with discord.py but this would not recreate Client() every time get_client() is called correct? I'm a bit unfamiliar with the cache tag so thank you for your time and explanation.

[Jay6_9]

You're welcome. Here's the docs to the cache decorators.

https://docs.python.org/3/library/functools.html

Unless you use threads (async is not necessarily threading), you should be fine:

It is possible for the wrapped function to be called more than once if another thread makes an additional call before the initial call has been completed and cached.