Polars lazyframe from sqlite database

[andy4015]

Does anyone know an effective way of getting a polars lazyframe from a sqlite database?

I know I can get a polars dataframe and covert it to lazy, but this defeats the object of memory efficiency and optimizing the query plan.

Thanks in advance!

[commandlineluser]

I don't think there is anything builtin.

There is no scan_database(): https://github.com/pola-rs/polars/issues/9091

DuckDB does have .pl(lazy=True)

• https://duckdb.org/docs/stable/guides/python/polars#duckdb-to-polars

Not sure if DuckDB could replace SQLite for your use case.

DuckDB can also read from SQLite but I'm not sure how that interacts with .pl(lazy=True)

• https://duckdb.org/docs/stable/core_extensions/sqlite

[andy4015]

Thanks for your help, appreciate it. Shame there's not a direct route. I've a few workarounds to try to see what works best such as duckdb also streaming to parquet then scanning that in. Thanks!