Re: Bit masks in PyNIO NetCDF?

Jesper Larsen wrote:
> Dear PyNGL people,
>
> For a shelf sea modeller GRIB has one killer feature. It allows for bit
> masking of values:
>
> http://www.wmo.ch/web/www/WDM/Guides/Guide-binary-2.html#Section3
>
> This means that it is only necessary to store values at actual sea points. In
> NetCDF it is common to store all values, including land (although as far as I
> understand the NetCDF standard allows for bit masking). The effect of this in
> my applications is that NetCDF files use more than ten times the storage
> space of GRIB files (I have written a simple grib2nc converter using PyNIO).
> Other oceanographic applications will experience similar problems (although
> probably in most cases to a lesser degree).
>
> Have you considered implementing NetCDF bit masking in NGL?
>
> Cheers,
> Jesper
>
>
Jesper: NetCDF-4 has built-in, on-the-fly zlib compression. My python
netcdf-4 module (http://code.google.com/p/netcdf4-python/) includes a
utility that uses PyNIO to convert GRIB files to netcdf-4 format, with
compression. In my experience, the files end up even smaller than the
original GRIB (even if the GRIB file has bit-masked fields).

-Jeff

-- 
Jeffrey S. Whitaker         Phone : (303)497-6313
NOAA/OAR/CDC  R/PSD1        FAX   : (303)497-6449
325 Broadway                Boulder, CO, USA 80305-3328
_______________________________________________
pyngl-talk mailing list
pyngl-talk_at_ucar.edu
http://mailman.ucar.edu/mailman/listinfo/pyngl-talk