Drexel dragonThe Math ForumDonate to the Math Forum



Search All of the Math Forum:

Views expressed in these public forums are not endorsed by Drexel University or The Math Forum.


Math Forum » Discussions » Software » comp.soft-sys.matlab

Topic: Memory leak writing NetCDF4 files but not NetCDF3 files
Replies: 2   Last Post: May 24, 2013 2:47 PM

Advanced Search

Back to Topic List Back to Topic List Jump to Tree View Jump to Tree View   Messages: [ Previous | Next ]
Jeff

Posts: 6
Registered: 2/2/09
Memory leak writing NetCDF4 files but not NetCDF3 files
Posted: May 2, 2013 11:14 PM
  Click to see the message monospaced in plain text Plain Text   Click to reply to this topic Reply

One of my scripts used to write many large NetCDF3 files (80-200 MB each). For storage reasons, I changed this script to write to NetCDF4 files with compression:
Old: ncid=netcdf.create([outputfolder fname_nc], 'CLOBBER');
New: ncid=netcdf.create([outputfolder fname_nc], 'NETCDF4');
Unfortunately, after the new script runs, MATLAB doesn't free the memory used during the time the script runs. For example:

NetCDF3 - Before function runs:
memory; gui_1('RAXPOL-20130417-225616.dat'); memory
Maximum possible array: 11806 MB (1.238e+010 bytes) *
Memory available for all arrays: 11806 MB (1.238e+010 bytes) *
Memory used by MATLAB: 1730 MB (1.815e+009 bytes)
Physical Memory (RAM): 16278 MB (1.707e+010 bytes)

NetCDF3 - After function runs:
Maximum possible array: 11359 MB (1.191e+010 bytes) *
Memory available for all arrays: 11359 MB (1.191e+010 bytes) *
Memory used by MATLAB: 1740 MB (1.825e+009 bytes)
Physical Memory (RAM): 16278 MB (1.707e+010 bytes)

NetCDF4 - Before function m-file runs:
>> memory; gui_1('RAXPOL-20130417-225616.dat'); memory
Maximum possible array: 12936 MB (1.356e+010 bytes) *
Memory available for all arrays: 12936 MB (1.356e+010 bytes) *
Memory used by MATLAB: 701 MB (7.355e+008 bytes)
Physical Memory (RAM): 16278 MB (1.707e+010 bytes)

NetCDF4 - After function m-file runs:
Maximum possible array: 11757 MB (1.233e+010 bytes) *
Memory available for all arrays: 11757 MB (1.233e+010 bytes) *
Memory used by MATLAB: 1716 MB (1.799e+009 bytes)
Physical Memory (RAM): 16278 MB (1.707e+010 bytes)

This is very repeatable and doesn't depend upon the particular files created. If I write to a NetCDF4 file, the memory usage grows and grows until Windows complains about low memory usage and Matlab crashes. It doesn't matter if I run the following:

close all
clear all
clear java
clear classes
clear functions
clear mex
pack

The memory used by MATLAB isn't freed, and I eventually need to restart Matlab to clear the memory. This only happens when writing NetCDF4 files.

I'm using Matlab 2011a on 64-bit Windows 7 with 16 GB of RAM. I preallocate all arrays, though that shouldn't matter since the memory should be cleared after the function "ends".

Any help? I don't want to have to restart Matlab so darn frequently just to free memory that should be freed automatically!



Point your RSS reader here for a feed of the latest messages in this topic.

[Privacy Policy] [Terms of Use]

© Drexel University 1994-2014. All Rights Reserved.
The Math Forum is a research and educational enterprise of the Drexel University School of Education.