Re: Bit masks in PyNIO NetCDF?

Jeff and Jesper,

We do have built in support for NetCDF 4 (classic mode only so far) in
PyNIO when it is built with the
NetCDF 4 library. There is an option that supports the various
compression levels. There has
been no announcement of this because we have been waiting for the final
release of NetCDF 4.
However, if anyone has a use for this feature and would like to test it
out, we would be happy to
supply a version built with NetCDF 4. I too have found that GRIB files
converted to compressed
NetCDF 4 files often become smaller.

Jesper: I have fixed the memory leak that you discovered in PyNIO.
Thanks for pointing it out.
  -dave

On Mar 2, 2007, at 7:03 AM, Jeff Whitaker wrote:

> Jesper Larsen wrote:
>> Dear PyNGL people,
>>
>> For a shelf sea modeller GRIB has one killer feature. It allows for
>> bit masking of values:
>>
>> http://www.wmo.ch/web/www/WDM/Guides/Guide-binary-2.html#Section3
>>
>> This means that it is only necessary to store values at actual sea
>> points. In NetCDF it is common to store all values, including land
>> (although as far as I understand the NetCDF standard allows for bit
>> masking). The effect of this in my applications is that NetCDF files
>> use more than ten times the storage space of GRIB files (I have
>> written a simple grib2nc converter using PyNIO). Other oceanographic
>> applications will experience similar problems (although probably in
>> most cases to a lesser degree).
>>
>> Have you considered implementing NetCDF bit masking in NGL?
>>
>> Cheers,
>> Jesper
>>
>>
> Jesper: NetCDF-4 has built-in, on-the-fly zlib compression. My
> python netcdf-4 module (http://code.google.com/p/netcdf4-python/)
> includes a utility that uses PyNIO to convert GRIB files to netcdf-4
> format, with compression. In my experience, the files end up even
> smaller than the original GRIB (even if the GRIB file has bit-masked
> fields).
>
> -Jeff
>
> --
> Jeffrey S. Whitaker Phone : (303)497-6313
> NOAA/OAR/CDC R/PSD1 FAX : (303)497-6449
> 325 Broadway Boulder, CO, USA 80305-3328
>
> _______________________________________________
> pyngl-talk mailing list
> pyngl-talk_at_ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/pyngl-talk

_______________________________________________
pyngl-talk mailing list
pyngl-talk_at_ucar.edu
http://mailman.ucar.edu/mailman/listinfo/pyngl-talk
Received on Mon Mar 05 2007 - 15:37:02 MST