Re: problem with PyNIO dimension sizes

Hi Zach,

OK, I understand. I will be looking into this immediately and
hopefully will have a fix soon.
  -dave

On Dec 14, 2008, at 1:49 PM, Zach DuFran wrote:

> David-
> Thanks for your response.
>
> Due to my only providing excerpts from my code, it was unclear what
> I was doing.
> 1. I was creating a new NetCDF file with each iteration of the t
> loop, so I wasn't redeclaring dimensions for the same file.
> 2. I did not want to write multiple times to the same file. I was
> writing one time to each of the 41 (nt) files I was generating from
> this script.
> 3. The levels and record dimensions are simply dummy dimensions
> that are required so that my NetCDF file format meets a specific
> format that is required for my purposes.
>
> Sorry for the confusion.
>
> Let me know when you have a bug fix for the dimension problem.
> Thanks!
> Zach
>
> ________________________________________
> From: David Brown [dbrown_at_ucar.edu]
> Sent: Sunday, December 14, 2008 2:13 PM
> To: Zach DuFran
> Cc: pyngl-talk_at_ucar.edu Talk
> Subject: Re: problem with PyNIO dimension sizes
>
> Hi Zach,
>
> Yes it does appear that something is not working right when some of
> the dimensions have single elements. However, for what I think you
> are trying to do there is a work-around. The code you posted and as
> edited by Mary below has some problems. First of all, you can only
> create a dimension or a variable in a file once, so you are sure to
> get an exception the second time through the loop. Second you are
> creating the level dimension with 1 element, yet your use of the loop
> counter t suggests that you really mean it to have 'nt' elements.
>
> Assuming this is your intention, it should work with the following
> modifications:
>
> import numpy
> import Nio
> import os
> ncols = 1024
> nrows = 1024
> nt = 41
>
> # Ingest data
> #databin = array.array('f')
> #databin.read(fileobj, nt*nrows*ncols)
> #prcp = numpy.array(databin, dtype='float32')
> prcp = numpy.zeros(nt*nrows*ncols,dtype='float32')
> prcp = prcp.reshape(nt,nrows,ncols)
>
> os.system("rm -f test.nc")
> ncfile = Nio.open_file("test.nc","c")
>
> ncfile.create_dimension('x',ncols)
> ncfile.create_dimension('y',nrows)
> ncfile.create_dimension('levels',nt) ###
> ncfile.create_dimension('record',1)
>
> d4 = ('record','levels','y','x')
>
> prcp_NC = ncfile.create_variable('prcp','f',d4)
>
> # either of the following work for me (using PyNIO 1.3.0b1)
>
> for t in range(nt):
> prcp_NC[0,t,:,:] = prcp[t,:,:]
>
> # or most simply
>
> prcp_NC[:] = prcp[:]
>
> ---------
>
> However, note that if the level dimension 'nt' is actually equal to
> 1, then neither of the methods work. This is definitely a bug and
> will be addressed as soon as possible.
> -dave
>
>
> On Dec 12, 2008, at 3:25 PM, Mary Haley wrote:
>
>> Hi Zach,
>>
>> This looks like a bug, but I'm not certain.
>>
>> I took your code snippet, and made it into code I could actually
>> run, and I got the same results as you:
>>
>> import numpy, os, Nio
>>
>> ncols = 1024
>> nrows = 1024
>> nt = 41
>>
>> prcp = numpy.zeros([nt,nrows,ncols], dtype='float32')
>>
>> filename = "test.nc"
>>
>> for t in range(nt):
>> if os.path.exists(filename): os.remove(filename)
>> ncfile = Nio.open_file(filename, 'c')
>> ncfile.create_dimension('x',ncols)
>> ncfile.create_dimension('y',nrows)
>> ncfile.create_dimension('levels',1)
>> ncfile.create_dimension('record',1)
>> d4 = ('record','levels','y','x')
>> prcp_NC = ncfile.create_variable('prcp','f',d4)
>> print prcp_NC.shape
>> print prcp.shape
>> print prcp_NC[0,0,:,:].shape
>> print prcp[t,:,:].shape
>> prcp_NC[0,0,:,:] = prcp[t,:,:]
>>
>> Hopefully Dave can take a look at this and see a work-around. I
>> tried a few things with no luck.
>>
>> --Mary
>>
>> On Fri, 12 Dec 2008 10:48:16 -0600
>> Zach DuFran <zdufran_at_wdtinc.com> wrote:
>>> I am trying to write a 4D variable to a NetCDF file and I am
>>> receiving a fatal error. The line where the error occurs is
>>> prcp_NC[0,0,:,:] = prcp[t,:,:]. I have printed out the shapes of
>>> each variable and they match. When I switch the order in prcp_NC
>>> to be prcp_NC[:,:,0,0] the script will run succesfully, but the
>>> data in the NetCDF file is all blank.
>>> If I assign prcp_NC as a 2D variable (only x and y) and do prcp_NC
>>> [:,:] = prcp[t,:,:] the data is written correctly.
>>> The error message for the attached script is:
>>> fatal:Dimension sizes of left hand side do not match
>>> right hand side
>>> The only way I can get the assignment to a 4D variable to work is
>>> to add 2 loops so the code reads:
>>> for i in xrange(nrows):
>>> for j in xrange(ncols):
>>> prcp_NC[0,0,i,j] = prcp[t,i,j]
>>> Of course, this method is considerably slower.
>>> I would appreciate any help you can provide-
>>> Zach DuFran
>>
>> _______________________________________________
>> pyngl-talk mailing list
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/pyngl-talk

_______________________________________________
pyngl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/pyngl-talk
Received on Mon Dec 15 2008 - 10:57:33 MST