Re: PyNIO and NetCDF file sizes - File too large

Carl,

Thanks for providing the file offline.

Using the PyNIO-1.3.0b1.linux-i686-gcc412-2.5.tar tarfile, along with
Python 2.5.4 and numpy 1.3.0, I was able to open this file just fine:

% python2.5
Python 2.5.4 (r254:67916, Jan 20 2010, 12:39:42)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-46)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import Nio
>>> Nio.__version__
'1.3.0b1'
>>> a = Nio.open_file('ocean_his-T1.nc')
>>> print(a)
Nio file: ocean_his-T1.nc
    global attributes:
       file : ocean_his.nc
       format : netCDF-3 classic file
       Conventions : CF-1.0
. . .
    dimensions:
       xi_rho = 1801
       xi_u = 1800
       xi_v = 1801
       xi_psi = 1800
       eta_rho = 1201
       eta_u = 1201
       eta_v = 1200
       eta_psi = 1200
       boundary = 4
       ocean_time = 241 // unlimited
       Nuser = 25
    variables:
       integer ntimes
          long_name : number of long time-steps
       integer ndtfast
          long_name : number of short time-steps

  . . .

Can you double-check that you have the same binary as me? Typing
"Nio" in the python prompt should give you the path:

>>> Nio
<module 'Nio' from ''/usr/local/lib/python2.5/site-packages/PyNIO/
Nio.pyc'>

Then you can exit python and type:

    ls -l /usr/local//lib/python2.5/site-packages/PyNIO/nio.so

Ours says:

-rwxr-xr-x 1 haley contribl 7066696 Jan 20 12:46 nio.so*

I believe that every 1.3.0b1 PyNIO binary should have been built with
large file support.

Maybe Dave Brown has a test you can run to double-check your PyNIO
for this.

--Mary

On Jan 19, 2010, at 12:11 PM, Carl Drews wrote:

> Mary and Dave -
>
> Thanks for checking. I installed PyNIO from your binary, not from
> source code. How can I check for large file support within my
> NetCDF install?
>
> The offending file is on its way to you.
>
> Carl
>
> Mary Haley wrote:
>> Hi Carl,
>> Dave Brown thinks that the version of PyNIO you have should work
>> on a file this large, unless you built PyNIO from source code and
>> didn't link it against a NetCDF library that had large file
>> support turned on.
>> If you are using the PyNIO binary from our website, then we're not
>> sure why it's not working. Would it be possible for you to put the
>> file somewhere that we could download it?
>> Thanks,
>> --Mary
>> On Jan 19, 2010, at 10:35 AM, Carl Drews wrote:
>>> I am attempting to open a 17 Gigabyte NetCDF file:
>>>
>>> 16949113596 Jan 18 04:51 ocean_his-T1.nc
>>>
>>> using this Python code:
>>>
>>> import Nio
>>> import matplotlib
>>> from pylab import *
>>> import numpy
>>> ...
>>> # open the NetCDF file for reading
>>> print "Nio version ", Nio.__version__
>>> file = Nio.open_file(romsFile, 'r')
>>>
>>> Here is the script output:
>>>
>>> ROMS file = ocean_his-T1.nc
>>> Nio version 1.3.0b1
>>> fatal:File too large
>>> fatal:Could not open (ocean_his-T1.nc)
>>> Traceback (most recent call last):
>>> File "waterLand.py", line 188, in <module>
>>> file = Nio.open_file(romsFile, 'r')
>>> File "/usr/local/python/lib/python2.5/site-packages/PyNIO/
>>> Nio.py",
>>> line 512, in open_file
>>> file = _Nio.open_file(filename,mode,options,history,format)
>>> NIOError: Unable to open file
>>>
>>> Is there a limit on the NetCDF file size? I need to read several
>>> variables from this model output. Do I have to split the
>>> ocean_his-T1
>>> file into multiple variables? If so, what maximum size is allowed?
>>>
>>> Here is my system information (uname -a):
>>>
>>> Linux acd-xxxxxx 2.6.18-164.6.1.el5.centos.plus #1 SMP Wed Nov 4
>>> 09:36:36 EST 2009 i686 i686 i386 GNU/Linux
>>>
>>> Thanks!
>>>
>>> Carl
>>> _______________________________________________
>>> pyngl-talk mailing list
>>> List instructions, subscriber options, unsubscribe:
>>> http://mailman.ucar.edu/mailman/listinfo/pyngl-talk
>

_______________________________________________
pyngl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/pyngl-talk
Received on Wed Jan 20 13:15:13 2010