Re: Masked arrays in PyNIO 1.3.0b1

Hi Mary,

I ran your test script, and Nio returned a masked
array. Here is the output from my IDLE session.

---->

Python 2.5 (r25:51918, Sep 19 2006, 08:49:13)
[GCC 4.0.1 (Apple Computer, Inc. build 5341)] on darwin
Type "help", "copyright", "credits" or "license" for more information. Python 2.5
(r25:51918, Sep 19 2006, 08:49:13)
Type "copyright", "credits" or "license" for more information.

IPython 0.8.4 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object'. ?object also works, ?? prints more.

In [1]: import Nio, Ngl
In [2]: dirc = Ngl.pynglpath("data")
In [3]: ufile = Nio.open_file(dirc + "/cdf/Ustorm.cdf","r")
In [4]: u = ufile.variables["u"][:]
In [5]: print type(u)
<class 'numpy.ma.core.MaskedArray'>

<----

I re-ran my "test case" using your idiom above:

var = myfile.variables['my_var'][:]

And Nio returns a masked array. What differs between
your usage and mine, is that I generally keep data
read in as a <class 'Nio.NioVariable'>, because all
the coordinates meta data are preserved. Thus, my
usage for your test case would be:

import Nio, Ngl
dirc = Ngl.pynglpath("data")
ufile = Nio.open_file(dirc + "/cdf/Ustorm.cdf","r")
u = ufile.variables["u"]
u_data = u.get_value()

Which returns a NumPy ndarray, rather than a masked
array. Granted, it is very straight forward to tran-
sform this to a masked array via:

u_data = ma.masked_values(u.get_value(), u._FillValue)

A safer way to do this (assuming no a priori knowledge
of a variable's attributes) is:

if hasattr(u, '_FillValue'):
   u_data = ma.masked_values(u.get_value(), u._FillValue)
else: u_data = u.get_value()

Perhaps I don't understand the intent of the get_value()
method.

Thanks for the example on the MaskedArrayOptions. They
key piece I was missing was the

   opt = Nio.options()
   opt.MaskedArrayMode = 'MaskedIfFillAtt'
   f = Nio.open_file(filename,options=opt)

Instead, I was simply doing this:

   f = Nio.open_file(filename, options='MaskedIfFillAtt')

Which through a pile of exceptions.

Daran

--
> You should indeed be getting a MaskedArray if your data contains a _FillValue or
missing_value attribute.
>
> Can you try a test for us?
>
> Run the short script below, and let me know what type "u" is:
>
> import Nio, Ngl
> dirc = Ngl.pynglpath("data")
> ufile = Nio.open_file(dirc + "/cdf/Ustorm.cdf","r")
> u = ufile.variables["u"][:]
> print type(u)
>
> It should return "<class 'numpy.ma.core.MaskedArray'>".
>
> If you don't have Ngl, then you can download the file from
> http://www.ncl.ucar.edu/Applications/Data/, and run the following script:
>
> import Nio
> ufile = Nio.open_file("Ustorm.cdf","r")
> u = ufile.variables["u"][:]
> print type(u)
>
> As for some examples for setting the MaskedArrayMode options, I
> believe Dave Brown is working on some. Meanwhile, you can try the attached test
program.
>
> --Mary
>
> On Sat, 30 Aug 2008, Daran L. Rife wrote:
>
>> Hi,
>> This didn't seem to get through last time round.
>> I Recently installed the latest version of PyNIO 1.3.0b1
>> for Mac OS X Leopard (Intel) running Python 2.5:
>>     PyNIO-1.3.0b1.macosx-10.5-i386-2.5.tar.gz
>> I was excited to learn that when reading variables from
>> netCDF files, Nio now returns a masked array for any var-
>> iable having the _FillValue or missing_value attribute
>> set within the input file.
>> When I try this, Nio still seems to return a numpy.ndarray,
>> instead of the desired masked array. Attached below is out-
>> put from my IDLE session. Could someone please help me un-
>> derstand what I am doing wrong?
>> Also, are there examples showing how to properly set the
>> new MaskedArrayMode options?
>> Thanks in advance for your help,
>> Daran
>> --
>> --->
>> Python 2.5 (r25:51918, Sep 19 2006, 08:49:13)
>> Type "copyright", "credits" or "license" for more information.
>> IPython 0.8.4 -- An enhanced Interactive Python.
>> ?         -> Introduction and overview of IPython's features.
>> %quickref -> Quick reference.
>> help      -> Python's own help system.
>> object?   -> Details about 'object'. ?object also works, ?? prints more. In [1]:
import Nio as nio
>> In [2]: f = nio.open_file('composite.20080523.L3m_DAY_SST_4.nc', 'r') In [3]: sst
= f.variables['sst_data']
>> In [4]: print sst
>> Variable: sst_data
>> Type: float
>> Total Size: 3110400 bytes
>>             777600 values
>> Number of Dimensions: 3
>> Dimensions and sizes:        [time | 1] x [latitude | 720] x [longitude | 1080]
Coordinates:
>>             time: [14022..14022]
>>             latitude: [24.52084..54.47918]
>>             longitude: [-98.47916..-53.52083]
>> Number of Attributes: 5
>>          long_name :        MODIS 8-day composite SST
>>          standard_name :        sea_surface_temperature
>>          units :        degC
>>          valid_range :        [-1.979918837547302, 31.75502777099609] _FillValue
:
>>      1e+20
>> In [5]: s = sst.get_value()
>> In [6]: type(s)
>> Out[6]: <type 'numpy.ndarray'>
>> In [8]: from numpy import ma
>> In [9]: s = ma.masked_values(sst.get_value(), sst._FillValue) # Force   data into
ma
>> In [10]: type(s)
>> Out[10]: <class 'numpy.ma.core.MaskedArray'>
>> <---
>> _______________________________________________
>> pyngl-talk mailing list
>> pyngl-talk_at_ucar.edu
>> http://mailman.ucar.edu/mailman/listinfo/pyngl-talk