Re: PyNIO: Assigning data to 3-d NetCDF variable

Hi Sasha,

It worked! I now have the correct shape when reading the the 'hi' variable back off the netCDF. I tried this method long ago, but had no luck at the time. Perhaps it now works because of the resolved datatype issue that you helped me fix.

Thank you for your time,
Greg

On Nov 18, 2013, at 2:22 PM, Oleksandr Huziy wrote:

> Just do it like this, have you tried it? without looping
>
> f_out.variables['hi'].assign_value(Clime_hi)
>
>
> 2013/11/18 Gregory Deemer <gjdeemer@alaska.edu>
> Sasha,
>
> Yep, I pulled the trigger too quickly on the resolution. The file size alone is an indicator that 2-D array was only assigned once. It is simply overwritten 365 times given the loop that I have set up.
>
>
> On Nov 18, 2013, at 1:53 PM, Oleksandr Huziy wrote:
>
>> Are you sure this works, if i understand correctly, here you are assigning a 2d array to a 3d var, it should have complained.... Or it is writing all the time to the first time slice of the hi variable... check the output with ncview for other time steps...
>>> for i in range(0, Clime_hi.shape[0]):
>>> f_out.variables['hi'].assign_value(Clime_hi[i,:,:])
>>
>> If it is OK, then I am very surprised))
>>
>> Cheers
>> --
>> Oleksandr (Sasha)
>>
>>
>>
>> 2013/11/18 Gregory Deemer <gjdeemer@alaska.edu>
>> Hi Oleksandr,
>>
>> Thank you for your help today, I think we have reached a solution. Declaring the variable as double, and reassigning my _FillValue from np.float32 to np.double to match this new type was the trick. It is a bit puzzling though, as I know the data that was ingested came from a single-precision floating point flat binary file.
>>
>> The logic in my value assignment statement:
>>> for i in range(0, Clime_hi.shape[0]):
>>> f_out.variables['hi'].assign_value(Clime_hi[i,:,:])
>>
>> Was of the correct form and is now functioning.
>>
>> Once again, thank you!
>> Greg
>>
>>
>> O
>> n Nov 18, 2013, at 1:03 PM, Oleksandr Huziy wrote:
>>> The ESG site is not letting me in yet))
>>> See here: http://www.pyngl.ucar.edu/FAQ/#ReadNetCDF
>>> item 23.
>>>
>>> Cheers
>>>
>>>
>>> 2013/11/18 Gregory Deemer <gjdeemer@alaska.edu>
>>> Hi Oleksandr,
>>>
>>> The double data type did not work out.
>>>
>>> fatal:NetCDF: Not a valid data type or _FillValue type mismatch: error attempting to write variable (hi) to file
>>>
>>> Perhaps my _FillValue being -9999.0 is causing the type error mismatch? I have written data to a NetCDF using similar methods, but my _FillValue was a positive int in those cases.
>>>
>>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>>
>>> Gregory J. Deemer
>>> Graduate Research Assistant
>>> Department of Atmospheric Sciences
>>> University of Alaska Fairbanks
>>> IARC 338K
>>> Office: (907) 474-5430
>>> Cell: (907) 750-1063
>>>
>>>
>>>
>>> On Mon, Nov 18, 2013 at 12:39 PM, Oleksandr Huziy <guziy.sasha@gmail.com> wrote:
>>> Gregory:
>>>
>>> I am trying to install nio now. Try to create the variable as double:
>>>
>>> # ICE THICKNESS
>>> f_out.create_variable('hi', 'd', ('time', 'y', 'x'))
>>>
>>>
>>> 2013/11/18 Gregory Deemer <gjdeemer@alaska.edu>
>>> Oleksandr,
>>>
>>> Thank you for the quick reply. I attempted the code examples that you suggested and PyNIO came back with errors.
>>>
>>> f_out.variables['hi'][i,:,:] = Clime_hi[i,:,:]
>>> self._obj[xsel] = value
>>> NIOError: type or dimensional mismatch writing to variable (hi)
>>>
>>>
>>> f_out.variables['hi'][:] = Clime_hi
>>> self._obj[xsel] = value
>>> NIOError: type or dimensional mismatch writing to variable (hi)
>>>
>>> I will try a few other permutations of your suggestion by replacing '=' with .assign_value()
>>>
>>> Thanks again,
>>> Greg
>>>
>>>
>>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>>
>>> Gregory J. Deemer
>>> Graduate Research Assistant
>>> Department of Atmospheric Sciences
>>> University of Alaska Fairbanks
>>> IARC 338K
>>> Office: (907) 474-5430
>>> Cell: (907) 750-1063
>>>
>>>
>>>
>>> On Mon, Nov 18, 2013 at 12:14 PM, Oleksandr Huziy <guziy.sasha@gmail.com> wrote:
>>> Hi Gregory:
>>>
>>> Does it work like this?
>>>
>>> # Write values to file
>>> for i in range(0, Clime_hi.shape[0]):
>>> f_out.variables['hi'][i,:,:] = Clime_hi[i,:,:]
>>>
>>>
>>> or
>>>
>>> f_out.variables['hi'][:] = Clime_hi
>>>
>>>
>>>
>>>
>>> 2013/11/18 Gregory Deemer <gjdeemer@alaska.edu>
>>> Hello,
>>>
>>> I'm wanting to assign values to a 3-D variable consisting of time, y, x. The time dimension is unlimited but is assigned 366 values. I have seen some examples of how to approach this problem with other scripting languages where a loop is applied to write the 2-D slice per timestep, and that is the approach I have taken here, but I am receiving the error message:
>>>
>>> NIOError: type or dimensional mismatch writing to variable (hi)
>>>
>>> For reference, the shapes of pertinent variables in the code snippet to follow are:
>>>
>>> Clime_hi.shape = (366, 120, 360)
>>> time_dat.shape = (366,) # values spanning 0 - 365
>>> lat_datR.shape = (120, 360)
>>> lon_datR.shape = (120, 360)
>>>
>>>
>>>
>>> # Create new variable dimensions
>>> f_out.create_dimension('time', None)
>>> f_out.create_dimension('y', 120)
>>> f_out.create_dimension('x', 360)
>>>
>>> # TIME
>>> f_out.create_variable('time', 'd', ('time',))
>>> # Attributes of time
>>> setattr(f_out.variables['time'], 'standard_name', 'Valid Time')
>>> setattr(f_out.variables['time'], 'units', 'hours since 2001-01-01 00:00:00')
>>> setattr(f_out.variables['time'], 'time_origin', '2001-01-01 00:00:00')
>>> setattr(f_out.variables['time'], 'calendar', 'gregorian')
>>> setattr(f_out.variables['time'], 'axis', 'T')
>>> # Write values to file
>>> f_out.variables['time'].assign_value(time_dat)
>>>
>>> # LATITUDE
>>> f_out.create_variable('lat', 'd', ('y','x'))
>>> # Attributes of latitude
>>> setattr(f_out.variables['lat'], 'standard_name', 'latitude')
>>> setattr(f_out.variables['lat'], 'units', 'degrees_north')
>>> setattr(f_out.variables['lat'], 'axis', 'Y')
>>> # Write values to file
>>> f_out.variables['lat'].assign_value(lat_datR)
>>>
>>> # LONGITUDE
>>> f_out.create_variable('lon', 'd', ('y', 'x'))
>>> # Attributes of longitude
>>> setattr(f_out.variables['lon'], 'standard_name', 'longitude')
>>> setattr(f_out.variables['lon'], 'units', 'degrees_east')
>>> setattr(f_out.variables['lon'], 'axis', 'X')
>>> # Write values to file
>>> f_out.variables['lon'].assign_value(lon_datR)
>>>
>>> # ICE THICKNESS
>>> f_out.create_variable('hi', 'f', ('time', 'y', 'x'))
>>> # Create variable attributes
>>> setattr(f_out.variables['hi'], 'standard_name', 'sea_ice_thickness')
>>> setattr(f_out.variables['hi'], 'units', 'm')
>>> setattr(f_out.variables['hi'], '_FillValue', _FillandMiss)
>>> setattr(f_out.variables['hi'], 'missing_value', _FillandMiss)
>>> # Write values to file
>>> for i in range(0, Clime_hi.shape[0]):
>>> f_out.variables['hi'].assign_value(Clime_hi[i,:,:])
>>>
>>> Again, the error message that I am given once the loop as finished its iteration is this:
>>>
>>> for i in range(0, Clime_hi.shape[0]):
>>> NIOError: type or dimensional mismatch writing to variable (hi)
>>>
>>> Please let me know if more information is needed to solve this error.
>>>
>>> Thank you,
>>> Greg
>>>
>>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>>>
>>> Gregory J. Deemer
>>> Graduate Research Assistant
>>> Department of Atmospheric Sciences
>>> University of Alaska Fairbanks
>>> IARC 338K
>>> Office: (907) 474-5430
>>> Cell: (907) 750-1063
>>>
>>>
>>> _______________________________________________
>>> pyngl-talk mailing list
>>> List instructions, subscriber options, unsubscribe:
>>> http://mailman.ucar.edu/mailman/listinfo/pyngl-talk
>>>
>>>
>>>
>>>
>>> --
>>> Sasha
>>>
>>>
>>>
>>>
>>> --
>>> Sasha
>>>
>>>
>>>
>>>
>>> --
>>> Sasha
>>
>>
>>
>>
>> --
>> Sasha
>
>
>
>
> --
> Sasha

_______________________________________________
pyngl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/pyngl-talk
Received on Mon Nov 18 16:37:29 2013