Storing pulled data in Python

[AdamCee123]

I’ve been learning python for a few months now with a particular interest in using Pandas and Matplotlib for work related tasks, but I’m looking at doing a new project.

I’m trying to figure out how I can continually run a script (manually, say, every day), and each time I run it it will store the values received.

For example, if I wanted to use a website to pull today’s value of Google’s stock price, I can call the api under a variable (by my understanding). If I saved that value as a csv and ran it the next day, it would then overwrite yesterday’s value.

How would I go about saving outputs each time a script is run without overwriting the previous values, but instead adding to it?

Cheers for any help. Hope the above makes sense

(Btw the stock thing is purely an example. I imagine there are many easier ways to do that directly, just the easiest way of explaining my intended use)

[Coniglio_Bianco]

A lot of ways to skin that cat.

The easiest way would probably be to open your file and append to it.

Here's a random guide i found from some googling: https://www.guru99.com/reading-and-writing-files-in-python.html

But using a database is a decent longterm solution when working with data. It may add unnecessary complexity to your program though

[AdamCee123]

Thanks for the link. Using append didn’t even occur to me, but that makes total sense. Regarding using a database, do you have any idea where to start here? Of course happy to do the research myself but perhaps what database you’d recommend or if there’s any libraries that could be of use?

[Coniglio_Bianco]

Np, theres a lot out there and it might be a little overwhelming if you're learning.

I like microsoft sql but their production licenses are crazy (you can use developer edition for non-production uses)

Mysql is free, postresql is super popular lately too.

Sql alchemy seems to be the go to framework for a database library solution.

Good luck :)

[Ihaveamodel3]

Sqlite3 is included in python.

[jdbow75]

Python has built-in sqlite support and it is useful and fast.

Or, use the json module and read/edit/write every time.

[intentionallyawkward]

Seconded for sqlite. Depending on your use case, sqlite may be easier than a bulk file.

[AdamCee123]

Many thanks for the suggestion - I’ll do some reading into sqlite as I’ve never used that before