The Math Forum

Search All of the Math Forum:

Views expressed in these public forums are not endorsed by NCTM or The Math Forum.

Math Forum » Discussions » Software » comp.soft-sys.matlab

Notice: We are no longer accepting new posts, but the forums will continue to be readable.

Topic: Problem with memory usage (R2009a)
Replies: 11   Last Post: Oct 21, 2009 10:20 AM

Advanced Search

Back to Topic List Back to Topic List Jump to Tree View Jump to Tree View   Messages: [ Previous | Next ]

Posts: 18
Registered: 7/14/09
Problem with memory usage (R2009a)
Posted: Oct 20, 2009 7:42 AM
  Click to see the message monospaced in plain text Plain Text   Click to reply to this topic Reply

Hi all,

I have a big CSV file (50K lines) that I want to load into memory and
then save to a .mat file so that I can access the data randomly. The
CSV file is about 25MB and the resulting .mat file is about 10MB.
However, during execution memory usage is much higher (about 300MB)
and when I load the .mat file into memory, I also use up 300MB of
memory, and I don't understand why. Can you help me with this? Is
there a way to reduce memory usage?

Here's the script:


function bars = loadHistory(symbol)

filepath = strcat('C:\forexdata\', upper(symbol), '.csv');
fid = fopen(filepath);

if fid == -1
fprintf('Error: could not open file %s\n', filepath);

bars = struct('date', [], 'time', [], 'open', [], 'high', [], 'low',
[], 'close', [], 'volume', []);
bars = repmat(bars, [1 500000]);

i = 1;
commas = ones(1, 6);
line = fgets(fid);
if line == -1
j = 1;
for cnt = 1:length(line)
if line(cnt) == ','
commas(j) = cnt;
j = j+1;

bars(i).date = datenum(line(1:commas(1)-1), '');
hr = str2double(line(commas(1)+1:commas(1)+2));
mn = str2double(line(commas(1)+4:commas(1)+5));
bars(i).time = hr*60 + mn;
bars(i).open = str2double(line(commas(2)+1:commas(3)-1));
bars(i).high = str2double(line(commas(3)+1:commas(4)-1));
bars(i).low = str2double(line(commas(4)+1:commas(5)-1));
bars(i).close = str2double(line(commas(5)+1:commas(6)-1));
bars(i).volume = str2double(line(commas(6)+1:max(size

i = i+1;


Point your RSS reader here for a feed of the latest messages in this topic.

[Privacy Policy] [Terms of Use]

© The Math Forum at NCTM 1994-2018. All Rights Reserved.