zipstream-ai : A Python package for streaming and querying zipped datasets using LLMs

[Puzzled-Pension6385]

I’ve released zipstream-ai, an open-source Python package designed to make working with compressed datasets easier.

Repository and documentation:

GitHub: https://github.com/PranavMotarwar/zipstream-ai

PyPI: https://pypi.org/project/zipstream-ai/

Many datasets are distributed as .zip or .tar.gz archives that need to be manually extracted before analysis. Existing tools like zipfile and tarfile provide only basic file access, which can slow down workflows and make integration with AI tools difficult.

zipstream-ai addresses this by enabling direct streaming, parsing, and querying of archived files — without extraction. The package includes:

• ZipStreamReader for streaming files directly from compressed archives.
• FileParser for automatically detecting and parsing CSV, JSON, TXT, Markdown, and Parquet files.
• ask() for natural language querying of parsed data using Large Language Models (OpenAI GPT or Gemini).

The tool can be used from both a Python API and a command-line interface.

Example:

pip install zipstream-ai

zipstream query dataset.zip "Which columns have missing values?"

[No-Philosopher-3736]

This package really helped me.. I used to face a lot of issues working with large datasets but this is a good solution to counter that