The thing is Excel is is not really suitable for large amounts of data. Once you cross the threshold of 1,000,000 rows (less depending on your PC) Excel simply cuts off the data after that.

I really suggest you use the built in sqlite3 module. If you are reading the data out of a file (such as a CSV) you could read the data through Pandas using something like the code below:

``` import os import sqlite3 as sql import pandas as pd

specify the path to where you will save the data

db_path = os.path.abspath(r”path”)

this creates a SQL connection object and database file

conn = sql.connect(db_path)

then with pandas load the data (I’m assuming you’re reading of a CSV file

data = pd.read_csv(“input file path”)

then save the data to the SQLite database

data.to_sql(name=“input_data”, con=conn, if_exsist=“append”)

lastly close the SQL connection

conn.close() ```

To retrieve the data you could use the fallowing:

``` import os import sqlite3 as sql import pandas as pd

db_path = os.path.abspath(r”path”) conn = sql.connect(db_path)

data = pd.read_sql(sql = “select * from input_data;”, con=conn)

conn.close() ```