dlt (data load tool)

Use it when

• Hand-written Python scripts pulling from REST APIs keep breaking on pagination, typing, and upstream schema changes before loading into a warehouse
• You need nested JSON from APIs or files normalized into typed relational tables in DuckDB, BigQuery, Snowflake, or another destination by swapping one destination string

What it solves

Not the fit when

• Not a scheduler or orchestrator; it runs inside your existing runtime (Airflow, Lambda, notebooks)
• New destination plugins are unlikely to be merged upstream per the maintainers
• Python 3.14 support is experimental with some extras unavailable
• workflow scheduling and orchestration
• streaming or CDC replication engine
• BI dashboards and reporting

Install

pip install "dlt[duckdb]" pandas

Invoke

import dlt; declare a named source and run pipeline = dlt.pipeline(pipeline_name="pokemon", destination="duckdb", dataset_name="pokemon_data"); pipeline.run(source); read a known loaded table back with pipeline.dataset().pokemon.df()

Alternatives

No reviewed alternatives recorded yet.