feat: add first-class SELECT DISTINCT support to SQL AST

[paulteehan]

Summary

• Add distinct: bool = False to the SELECT AST node so callers can write SELECT(fields=[...], distinct=True) and get SELECT DISTINCT col1, col2 FROM ... natively
• _build_select_sql_lines emits SELECT DISTINCT when any incoming SELECT element has distinct=True
• Existing DISTINCT expression node and _build_distinct_sql are intentionally untouched — they remain correct for aggregate-level use (COUNT(DISTINCT x), SUM(DISTINCT x))

Why

SQL uses DISTINCT in two semantically different positions:

Feature	Example	Semantics
Set quantifier on SELECT	SELECT DISTINCT a, b FROM t	Dedupes the whole result set
Aggregate-level modifier	COUNT(DISTINCT a)	Per-aggregate input dedup

Today the AST only models the second. The first has no representation, and the workaround of nesting DISTINCT([a, b, c]) into SELECT.fields renders as SELECT DISTINCT(a, b, c) — accepted loosely by Postgres/MySQL but rejected by DB2, Athena/Presto/Trino, and BigQuery, where parens around the field list are interpreted as a row constructor and change semantics. The existing warning at sql_ast.py:61-67 flags this exact shape as wrong.

This was discovered during review of soda-extensions PR #356, which currently ships:

paginated_sql = data_source.sql_dialect.build_select_sql(statements)
paginated_sql = paginated_sql.replace(\"SELECT\", \"SELECT DISTINCT\", 1)

After this change lands, that caller (and any future ones) can be migrated to SELECT(fields=columns, distinct=True) and the string-replace hack deleted.

What changed

• soda-core/src/soda_core/common/sql_ast.py — SELECT gets distinct: bool = False; warning message updated to point callers at the new flag
• soda-core/src/soda_core/common/sql_dialect.py — _build_select_sql_lines reads select_element.distinct and prepends DISTINCT to the keyword. Continuation-line indent kept at 7 spaces (no realignment to SELECT DISTINCT 's width — still readable)
• soda-postgres/tests/unit/test_postgres_dialect.py — 6 new tests

Acceptance criteria (from internal plan)

• SELECT(fields=[\"a\", \"b\"], distinct=True) renders SELECT DISTINCT a, b (no parens)
• SELECT(fields=[\"a\"]) continues to render SELECT a unchanged
• COUNT(DISTINCT(expression=\"x\")) continues to render COUNT(DISTINCT(x))
• All existing call sites work without change (default distinct=False)
• Unit tests cover: single col, multiple cols, full paginated shape (WHERE/ORDER BY/LIMIT/OFFSET), aggregate-level DISTINCT preserved, default-false

Test plan

• uv run pytest soda-{postgres,bigquery,athena,databricks,duckdb,redshift,snowflake,sparkdf,sqlserver,synapse,trino,fabric}/tests/unit — 30/30 passed
• Full-suite collection: 905 tests collect cleanly, no import regressions
• Integration matrix (run in CI) — especially DB2, Athena, BigQuery, Trino once soda-extensions caller is migrated
• Follow-up: migrate soda-reconciliation/.../reference_diff_check.py once soda-extensions PR #356 merges, then grep -rn '.replace(\"SELECT\", \"SELECT DISTINCT\"' src/ should return zero hits

Out of scope

• Removing/renaming the existing DISTINCT expression node — it has legitimate aggregate-level users
• The TODO: refactor build_select_sql to use AST at sql_dialect.py:669 — independent and larger
• SELECT DISTINCT ON (...) (Postgres-specific) — not needed here

🤖 Generated with Claude Code

[Lepo00]

Thank you very much for this @paulteehan!

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[Niels-b]

Not sure if we should do it like this. We already have a DISTINCT in the AST, you should just use that, no?

In my opinion we should be able to do SELECT(DISTINCT([my_elements])). Open for discussion.

Also, what are we assuming will happen here? If we have a SELECT DISTINCT that the entire row is "distinct"?