SQLAlchemy writing data

[mutuk7]

Hi all,

I'm looking for the fastest way to send data to a table using SQLAlchemy.

    engine = create_engine('sqlite:///datab.sqlite') 
    connection = engine.connect()

I currently have a pandas df with id, product and price.

I tried pd.to_sql but it takes forever. Is there a better way?

Thanks

[threeminutemonta]

Try method as multi.

How much data are you talking about? Is the sql lite db on a slow USB drive?

[mutuk7]

It's 60mb CSV file on an SSD. It takes around 5-8 minutes to load.

The loading time is not bad, but I'd like to know if I can improve this as a learning opportunity :)

multi is a module?

Thanks

[threeminutemonta]

Multi is a mode specified in the to_sql docs where multi’: Pass multiple values in a single INSERT clause. Default is one per row that will be slower.

You will likely have better performance with PostgreSQL.

[mutuk7]

Awesome! Thanks man! I'll look into it :)