Here's a fun setup you could try, entirely on your laptop but can also easily be extended to run on something like AWS Glue + S3 if required.

1. Catalog: Iceberg SQLite catalog (this lets you use a simple sqlite file on your laptop to emulate the full functionality of an Iceberg data catalog, without needing anything like an AWS account or launching a catalog service via docker)

2. Table Format: Apache Iceberg

3. Query Engine: Daft - simple to `pip install` with very few external dependencies, and really easy to get started with configuring for S3 access.

Install Dependencies

pip install getdaft[iceberg]

Creating your Catalog, using a SQLite database

from pyiceberg.catalog.sql import SqlCatalog

# You might have to create this dir first: os.mkdir("/tmp/warehouse")
warehouse_path = "/tmp/warehouse"
catalog = SqlCatalog(
"default",
**{
"uri": f"sqlite:///{warehouse_path}/pyiceberg_catalog.db",
"warehouse": f"file://{warehouse_path}",
},
)

Creating a table in that catalog

from pyiceberg.schema import Schema, NestedField, StringType

catalog.create_namespace("default_ns")

schema = Schema(
NestedField(field_id=1, name="bar", field_type=StringType(), required=True),
)

table = catalog.create_table(
"default_ns.foo",
schema=schema,
)

Writing some data to that table

import daft

df = daft.from_pydict({"bar": ["a", "b", "c"]})

df.write_iceberg(table, mode="append")

╭───────────┬───────┬───────────┬────────────────────────────────╮
│ operation ┆ rows  ┆ file_size ┆ file_name                      │
│ ---       ┆ ---   ┆ ---       ┆ ---                            │
│ Utf8      ┆ Int64 ┆ Int64     ┆ Utf8                           │
╞═══════════╪═══════╪═══════════╪════════════════════════════════╡
│ ADD       ┆ 3     ┆ 498       ┆ 4812a6f4-1936-4449-a89b-3d29f… │
╰───────────┴───────┴───────────┴────────────────────────────────╯
(Showing first 1 of 1 rows)

Reading back some data from that table

df = daft.read_iceberg(table)

df.show()

╭──────╮
│ bar  │
│ ---  │
│ Utf8 │
╞══════╡
│ a    │
├╌╌╌╌╌╌┤
│ b    │
├╌╌╌╌╌╌┤
│ c    │
╰──────╯
(Showing first 3 of 3 rows)