How to Analyze Large CSV Files with DuckDB

When a CSV file has more than 100,000 rows, Excel struggles. Pivot tables slow down. Filters take seconds to respond. Saving the file can trigger a crash. This is not a machine performance problem. It is an Excel architectural problem: the spreadsheet format was built for interactive editing, not bulk data querying.

DuckDB solves this with a different approach. It is a database that runs entirely inside a program on your laptop, with no server, no login, and no configuration. You point it at a CSV file, write a SQL query, and get an answer in milliseconds. A benchmark demonstrated by the DuckDB community in early 2026 showed querying 1.3 million rows across eight CSV files in a single statement, faster than Excel can even open a file that size.

DuckDB is free, open source, and installs in under five minutes. You do not need to know Python. You do not need to set up a database server. You do not need cloud storage. You just need a terminal and a few SQL commands you can copy from this guide.

Install DuckDB on Mac or Windows

On a Mac:

Open Terminal (press Command + Space, type "Terminal", press Enter). Run this command:

brew install duckdb

If you do not have Homebrew installed, run this first:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Then re-run the brew command.

On Windows:

Open Command Prompt (press Windows key, type "cmd", press Enter). Run:

winget install DuckDB.cli

If winget is not available, download the DuckDB CLI binary directly from duckdb.org/docs/installation and place it in a folder on your PATH.

Launch DuckDB with a graphical interface:

DuckDB ships with a built-in web UI as of version 1.2. After installing, run:

duckdb -ui

This opens a browser-based interface at localhost:4213. You can write and run queries in a text editor, view results in a table, and export without touching the terminal again. This is the recommended starting point for analysts who prefer not to work in a command line.

Verify the install:

In the terminal, run duckdb --version. You should see something like v1.2.0. If you get "command not found," close and reopen the terminal and try again.

Load Your CSV File

You do not need to import or convert the file. DuckDB reads CSV files directly.

Place your CSV file in a folder you can find easily. For this guide, the file is called sales.csv and it sits in your Downloads folder.

In the DuckDB shell (launched by typing duckdb in the terminal), run:

SELECT * FROM read_csv('~/Downloads/sales.csv') LIMIT 5;

This previews the first five rows. DuckDB automatically detects column names and data types from the file header, with no schema definition needed.

To inspect the full column list and types:

DESCRIBE SELECT * FROM read_csv('~/Downloads/sales.csv');

To count the total number of rows:

SELECT COUNT(*) FROM read_csv('~/Downloads/sales.csv');

If your file has more than 500,000 rows, save it as a DuckDB table first:

CREATE TABLE sales AS SELECT * FROM read_csv('~/Downloads/sales.csv');

Now every query runs against the in-memory table instead of re-reading the file from disk, which is considerably faster.

Run Business Queries Without Python

The four queries below cover the most common analysis tasks for operations and finance teams. Each uses plain SQL. The syntax reads almost like English: SELECT which columns, FROM which table, WHERE some condition is true, GROUP BY a category, ORDER BY a value.

Query 1: Find the top products or categories by revenue

SELECT category, SUM(revenue) AS total_revenue
FROM sales
GROUP BY category
ORDER BY total_revenue DESC
LIMIT 10;

This returns the top ten categories sorted by total revenue. Replace category and revenue with your actual column names.

Query 2: Filter rows by date or amount

SELECT *
FROM sales
WHERE purchase_date >= '2025-01-01'
  AND purchase_amount > 500
ORDER BY purchase_date DESC;

This returns all transactions from 2025 onward where the amount exceeded 500. Adjust the date and amount to match your data.

Query 3: Summarize by month

SELECT
  strftime(CAST(purchase_date AS DATE), '%Y-%m') AS month,
  COUNT(*) AS total_orders,
  SUM(purchase_amount) AS total_sales
FROM sales
GROUP BY month
ORDER BY month;

The strftime function formats the date into year-month (e.g. 2025-03). This produces a clean monthly summary without needing a pivot table.

Query 4: Combine multiple CSV files in one query

If your data is split across multiple files (one per month, one per region), DuckDB reads them all at once:

SELECT * FROM read_csv('~/Downloads/sales_*.csv', union_by_name=true);

The * is a wildcard that matches any filename starting with sales_. The union_by_name=true option handles minor column differences between files without crashing. In a documented test using WhoTracksMe data, this approach handled 1.3 million rows across eight CSV files in a single command on a standard laptop.

Export Your Results

Once you have the query result you want, export it to a new CSV file:

COPY (
  SELECT category, SUM(revenue) AS total_revenue
  FROM sales
  GROUP BY category
  ORDER BY total_revenue DESC
) TO '~/Downloads/revenue_by_category.csv' (HEADER, DELIMITER ',');

This writes the query result directly to a new CSV file. You can open it in Excel, share it by email, or import it into any reporting tool.

To export to Parquet instead (smaller and faster for large datasets):

COPY (SELECT ...) TO '~/Downloads/output.parquet' (FORMAT 'PARQUET');

When DuckDB Beats Excel (and When to Stay)

DuckDB is the right tool when:

• Your CSV has more than 100,000 rows and Excel is slow or crashing
• You need to combine multiple files from different sources or time periods
• You want to apply the same analysis repeatedly without rebuilding a pivot table every time
• You are working with JSON exports, Parquet files, or data that Excel cannot open cleanly

Excel is still the right tool when:

• You are editing data by hand (correcting values, adding rows, formatting cells)
• You need to share a formatted report with stakeholders who expect a spreadsheet
• Your dataset fits comfortably in memory and a pivot table answers your question in under a minute

The useful comparison is not DuckDB vs. Excel on the same task. It is which tool makes the task take the least time. For data exploration and aggregation on files over 100,000 rows, DuckDB consistently wins. For final formatting and sharing, most teams still use Excel or Google Sheets as the last step.

If you want to skip the SQL setup entirely, VSLZ AI lets you upload a CSV and ask questions in plain English, returning charts and summaries from a single prompt with no SQL or terminal needed.

Where to Go Next

The fastest way to start is with the DuckDB UI. Install DuckDB, run duckdb -ui, upload your CSV, and run the four queries above. A working analysis takes under 15 minutes.

For files you return to regularly, save them as named DuckDB tables so queries stay fast. For teams that need results in a shared format, use the COPY command to export back to CSV after each session.

DuckDB does not replace a full data warehouse for teams managing millions of events per day. But for the analyst or operations manager who needs to answer a business question from a large file today, it is the fastest path from raw data to answer without cloud setup or engineering support.