Skip to content

NetCDF compression

Jump to bottom
André Castro edited this page Aug 27, 2024 · 1 revision

How to resolve issue with nccopy incompatible versions

We call netcdf-bin utility nccopy from modules/classes.py in class NetCDF to compress the output NetCDF. However the version 4.9.0 of netcdf-bin produces often errors when compressing the parsivel NetCDFs, while version 4.7.0 does handle them without errors.

This page describes how to set the Pi OS (Debian) to install version 4.7.0., by pinning that package to version 4.7.0

error

/usr/local/src/disdrodlv2/venv/bin/python /usr/local/src/disdrodlv2/export_disdrodlDB2NC.py -d $(date -u +"%Y-%m-%d" --date="yesterday") -c /usr/local/src/disdrodlv2/configs_netcdf/config_001_CABAUW.yml

compress: /data/disdroDL/202401/20240117_KNMI_Cabauw-CABAUW_PAR001.nc 
NetCDF: Filter error: bad id or parameters or duplicate filter
Location: file ; line 1152
Failed to compress /data/disdroDL/202401/20240117_KNMI_Cabauw-CABAUW_PAR001.nc. Error code:1

More info on netcd-bin rersion

apt-cache policy netcdf-bin

On @agoncalvesdeol laptop - Debian 11 ("bullseye") — current "oldstable" release

netcdf-bin:
  Installed: 1:4.7.4-1
  Candidate: 1:4.7.4-1
  Version table:
 *** 1:4.7.4-1 500
        500 http://deb.debian.org/debian bullseye/main amd64 Packages
        100 /var/lib/dpkg/status

On Pi - Debian 12 ("bookworm") — current "stable" release

netcdf-bin:
  Installed: 1:4.9.0-3+b1
  Candidate: 1:4.9.0-3+b1
  Version table:
 *** 1:4.9.0-3+b1 500
        500 http://deb.debian.org/debian bookworm/main arm64 Packages
        100 /var/lib/dpkg/status

Solution: pinning netcdf-bin to version 4.7.4 (Bullseye)

Package: netcdf-bin
Pin: release n=bullseye
Pin-Priority: 999

added to /etc/apt/preferences.d/netcdf-bin

deb http://deb.debian.org/debian bullseye main contrib appended to /etc/apt/sources.list

apt-cache policy netcdf-bin

netcdf-bin:
  Installed: (none)
  Candidate: 1:4.7.4-1
  Version table:
     1:4.9.0-3+b1 500
        500 http://deb.debian.org/debian bookworm/main arm64 Packages
     1:4.7.4-1 999
        500 http://deb.debian.org/debian bullseye/main arm64 Packages

install 1:4.7.4-1 version: apt-get install netcdf-bin=1:4.7.4-1

Clone this wiki locally