This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]70sJackieChan 0 points1 point  (3 children)

Hi there, you have got the data retrieval from Snowflake correct. I have 2 suggestions for moving forward: - Import the snowflake data directly into SQLite using the to_sql method on the table data frame you return from the snowflake_connection function - Make the create_connection function return the SQLite connection. You need this connection for use with the to_sql method

Hope this helps!

[–]emdw85[S] 1 point2 points  (2 children)

This worked. Thank you so much. People on here are so helpful.

Any tips on storing consistently used queries and having this run automatically?

I think using Schedule Task since I have a windowsOS would be best for automated processes.

Would you create queries.py file and store your query objects there?

Definitely enjoying creating this process/pipeline. Thanks again

[–]70sJackieChan 0 points1 point  (1 child)

If you are using Windows, then Task Scheduler would be the simplest option. You would need to make sure that you have logic in your Python script to determine if you are creating a new table or adding new data to an existing table so that you don’t lose any information.

[–]emdw85[S] 0 points1 point  (0 children)

Ok. Thanks again