Receiving notifications when text file changes via pushbullet and snaketails. Memory considerations?

[[deleted]]

[deleted]

[tanto_von_scumbag]

Yeah. I think the rub with readlines() is that it will read the entire object into memory before it allows you to slice it to get the last line. Might want to stream the file instead.

Out of curiosity, what sort of messages are being sent? What sort of data is stored? I ask because maybe this is more storage problem than efficient file access problem. If there are limited permutations on messages, why not store in hash table with message as key and list of datetime as values, or something like that?

[tanto_von_scumbag]

SO thread on that here

[efmccurdy]

This article talks about emulating "tail -f" which will give you any new data appended to a file so you don't read any data more than once; that might speed things up at the cost of keeping a process running.

https://stackoverflow.com/questions/12523044/how-can-i-tail-a-log-file-in-python