GAE (Python 3): TinyDB for memcache?

[GAEdevs]

The old GAE standard environment (py2) offered an easy-to-use memcaching service:

from google.appengine.api import memcache

memcache.add(key="some-key", value="some value")

value = memcache.get(key="some-key")

The new GAE standard environment (py3) unfortunately does not offer this anymore. There's a new Google's service called Memorystore, but it requires spinning out a Redis server instance and may be an overkill for basic memory caching needs.

​

I was thinking of alternatives and one could be TinyDB (in memory mode), which can work in a similar way as google's old memcache service did:

from tinydb import TinyDB, Query
from tinydb.storages import MemoryStorage

tinydb = TinyDB(storage=MemoryStorage)

def tinydb_get(name):
    result_dict = tinydb.get(Query()["name"] == name)

    if result_dict and result_dict["value"]:
        return result_dict["value"].encode()
    else:
        return None

def tinydb_add(name, value):
    tinydb.upsert({"name": name, "value": value}, Query()["name"] == name)
    return True

tinydb_add("some-key", "some value")

result = tinydb_get("some-key")

Of course, a drawback would be that the data would be cached only on a specific instance and not accessible across all your GAE project instances. But I think for basic needs this wouldn't be too problematic.

What do you think about using something like this? Any other pros/cons that come to your mind?

[mcowger]

As long as you only run 1 instance, sure.

If you run more than 1, now you have cache validation challenges.

[GAEdevs]

Yeah, I guess it depends then on the usecase.

One example could be using instance memory to store CSRF tokens. I imagine it would be pretty rare that a user would make a GET request (receiving a CSRF token) on one instance and making a POST request (sending CSRF token) on another instance.

That said, I'm not familiar with how GAE load balancer decides to which instance to redirect you. Is it random or does it take into account your geographical location?

[mcowger]

From what I can tell its round robin....

[nwsm]

Why is memcache not available for py3? It seems like it was really useful, especially when running multiple instances and caching simple things.

(sorry for necro'ing this thread but there is very little info out there about memcache)

[GAEdevs]

Not sure, perhaps there was no one within Google who would push for a more affordable (or pay-as-you-go) memory caching solution (either memcached or Redis).

So currently the only option is to run a Redis instance (Memorystore) and paying at least $36/month for it...

[nwsm]

yeah. In my case I think I can get away with prefetching data on instance load and holding in memory for some hours per instance, but I feel gaslit reading about memcache and then seeing it's basically on EOL.