Fix kde dask backends dropping points on descending-coordinate templates (#3627)#3633
Conversation
…eep their points (#3627) _filter_points_to_tile assumed positive dx/dy when computing the tile extent, so dask+numpy and dask+cupy dropped most or all points for descending-coordinate templates: all-zero output for compact kernels, partially wrong values for gaussian. Same bug class as #1198, one layer above the #1199 kernel fix. Also records the 2026-07-03 accuracy sweep of the kde module in the sweep state CSV.
brendancol
left a comment
There was a problem hiding this comment.
PR Review: Fix kde dask backends dropping points on descending-coordinate templates (#3627)
Blockers (must fix before merge)
- none
Suggestions (should fix, not blocking)
- xrspatial/tests/test_kde.py (TestDaskCupyDescending): the dask+cupy regression test covers only the descending-y quartic case. The filter is shared code, so this is enough to pin the bug, but a desc-x case would be cheap and would catch any future divergence between the two dask wrappers (they duplicate the tile loop).
Nits (optional improvements)
- xrspatial/kde.py:391-401:
tile_x1 = tile_x0 + tile_cols * dxovershoots the last pixel centre by one spacing, so the filter keeps a slightly wider band than strictly needed. That is the safe direction (never drops a contributing point) and matches the pre-existing behavior on ascending grids; worth a half-sentence in the comment if anyone touches this again.
What looks good
- The fix is minimal: order the tile edges with min/max before widening by the cutoff, exactly parallel to what #1199 did inside the kernels.
- Verified locally on a CUDA host: dask+numpy and dask+cupy now match numpy exactly (max abs diff 0 for quartic, fringe-only for gaussian) on desc-y, desc-x, and desc-both templates. Before the fix, compact kernels returned all zeros and gaussian returned partially wrong values.
- The new parametrized test (3 orientations x 3 kernels) fails on main and passes here; tolerance choices mirror the existing #1199 regression tests (exact for compact kernels, atol=1e-5 for the gaussian 4*bw box-cutoff fringe).
- The sweep-accuracy state CSV update rides along per the sweep contract; it does not touch code.
Checklist
- Algorithm matches reference/paper (filter now consistent with the kernels' own cutoff logic)
- All implemented backends produce consistent results (verified numpy / cupy / dask+numpy / dask+cupy on this host)
- NaN handling is correct (unchanged by this PR; tracked separately in #3628)
- Edge cases are covered by tests (3 orientations x 3 kernels + dask+cupy)
- Dask chunk boundaries handled correctly (per-tile origins already sign-correct; only the filter was wrong)
- No premature materialization or unnecessary copies
- Benchmark exists or is not needed (bug fix, no perf-relevant change: the filter does the same amount of work)
- README feature matrix updated (n/a, no API change)
- Docstrings present and accurate (helper docstring unchanged, still correct)
brendancol
left a comment
There was a problem hiding this comment.
PR Review follow-up (#3627)
Disposition of the earlier findings, after ae16e5e:
- Suggestion (dask+cupy desc-x coverage): fixed.
TestDaskCupyDescendingis now parametrized over desc-y and desc-x; 66 tests pass locally, both GPU variants executed on this host. - Nit (filter overshoot comment): fixed. The comment now says the tile edge overshoots the last pixel centre by one spacing and why that is the safe direction.
No new findings. No blockers.
brendancol
left a comment
There was a problem hiding this comment.
PR Review: Fix kde dask backends dropping points on descending-coordinate templates (#3627)
Re-review after merging current main (SHA 228fadf5). This pulls in the libjxl=0.11.* CI pin (#3639), which was the only reason CI was red. The merge was clean with no conflicts; kde.py was untouched by main.
Correctness
The fix in _filter_points_to_tile (xrspatial/kde.py:396-402) is right. With descending coordinates dx/dy go negative, so tile_x1 = tile_x0 + tile_cols*dx falls below tile_x0. The previous bounds xs >= tile_x0 - cutoff and xs <= tile_x1 + cutoff then described an inverted interval and masked out every contributing point. Ordering the edges with min/max before widening by cutoff restores a valid interval. The tile_x1/tile_y1 one-spacing overshoot past the last pixel centre keeps the filter conservative, so it never drops a real contributor.
Both dask backends route through this one helper (xrspatial/kde.py:436 for numpy, :475 for cupy), so the fix covers dask+numpy and dask+cupy together.
Tests
test_dask_matches_numpy_descending_coords covers all three descending combinations across gaussian/epanechnikov/quartic, with a sum() > 0 guard plus per-kernel tolerances. TestDaskCupyDescending adds the same check on the GPU path. Ran locally against the worktree:
xrspatial/tests/test_kde.py .................................................. 66 passed in 1.29s
The dask+cupy cases execute on this box (CUDA present), not skipped.
What looks good
- Change is scoped to the bug, nothing else touched.
- The comment explains why the ordering matters and cites the issue.
- Tests reproduce the original failure across kernels and both dask backends.
No blockers. Ready to merge once CI goes green with the main merge.
Closes #3627
_filter_points_to_tilecomputed the tile extent astile_x0 .. tile_x0 + cols*dx, which inverts when dx/dy are negative. Descending-coordinate templates (the normal north-up raster layout) lost most or all of their points before the kernel ran: compact kernels returned all zeros, gaussian returned partially wrong values. The fix orders the tile edges with min/max before widening by the cutoff.Backends: numpy and cupy unchanged (already correct); dask+numpy and dask+cupy fixed and now match numpy exactly on descending-y, descending-x, and both-descending templates.
Test plan:
test_dask_matches_numpy_descending_coords(3 orientations x 3 kernels) fails on main, passes with the fixTestDaskCupyDescendingruns the dask+cupy path on a GPU host (CUDA verified locally)test_kde.pysuite: 65 passed