Re: Writing speed depends on indexed dimension

Update: it seems that only writing the first block takes a lot of time.
So conc_var[0] = random((25,180,360)) takes more than 5 minutes, but
conc_var[1] = random((25,180,360)) right after that takes less than a
second. So it's not as bad as I thought it was, but I'm still confused
as to why.

-Sourish

On 10/04/2010 12:04 PM, Sourish Basu wrote:
> Hi All,
>
> I'm trying to write a very large array to a variable
> var[2922,25,180,360] in a netcdf file with the option LargeFile set.
> The file size /per se/ is not a problem. I'm computing and writing it
> in chunks of 25x180x360. The problem is that the write speed depends
> on the length of the first index. For example, the following
>
> opt = options()
> opt.Format = 'LargeFile'
> fid = open_file('test.nc', 'w', options=opt)
> fid.create_dimension('days',2922)
> fid.create_dimension('pres_divisions',25)
> fid.create_dimension('pres_levels',26)
> fid.create_dimension('latitude',180)
> fid.create_dimension('longitude',360)
> conc_var = fid.create_variable('mixing_ratio', 'd', ('days',
> 'pres_divisions', 'latitude', 'longitude'))
> conc_var[0] = random((25,180,360))
>
> takes a very long time, whereas if I just change the definition of
> 'days' to
>
> fid.create_dimension('days',2)
>
> then the writing of conc_var[0] takes less than a second. I want to
> write the blocks sequentially, i.e., conc_var[0], then conc_var[1],
> then conc_var[2] and so on. Is there any way to speed up the writing
> (in the case where 'days' is 2922) by using that fact?
>
> Cheers,
>
> -Sourish
>
>
> _______________________________________________
> pyngl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/pyngl-talk
>

_______________________________________________
pyngl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/pyngl-talk