Drexel dragonThe Math ForumDonate to the Math Forum



Search All of the Math Forum:

Views expressed in these public forums are not endorsed by Drexel University or The Math Forum.


Math Forum » Discussions » Software » comp.soft-sys.matlab

Topic: Downsampling very large text file
Replies: 7   Last Post: Apr 18, 2013 9:02 AM

Advanced Search

Back to Topic List Back to Topic List Jump to Tree View Jump to Tree View   Messages: [ Previous | Next ]
dpb

Posts: 8,231
Registered: 6/7/07
Re: Downsampling very large text file
Posted: Apr 18, 2013 9:02 AM
  Click to see the message monospaced in plain text Plain Text   Click to reply to this topic Reply

On 4/18/2013 3:53 AM, bram wrote:
> "bram " <b.schrier@student.tudelft.nl> wrote in message
> <kkob28$3qg$1@newscl01ah.mathworks.com>...

>> Oops its not working like it should.
>>
>> Im getting all NaN's
>>
>> Firstly i made a mistake in the formatting of the txt file:
>>
>> It is actually formatted:
>>
>> Value1 Value2 Value1 Value3 Value1 Value4 ... Value1 Value9
>> Pairs have one space in between, 3 tabs between pairs
>>
>> Will changing '\t' to ' ' work or will i have to enter all formatting?
>>
>> Secondly does %f handle values like this: 1.896875e-001?
>>
>> Thanks for the help so far,
>>
>> Bram

>
> Ive tried removing the delimiter part since the default is all types of
> white spaces but it has no effect.
>
> Further: the script seems to not stop at the ens of the text file. i'm
> testing with a smaller file but it doesnt stop...
>
> code:
>
> NtoAvg = 20; % how many records for each average
> m=zeros(20,16);
> fmt=repmat('%f',1,16);
> i=0;
> fid=fopen('exportrun12_short.txt','rt');
> while ~feof(fid)
> C=textscan(fid,fmt,NtoAvg,'collectoutput',1);
> i=i+1;
> m(i,:)=mean(C{:})
> end
> fid=fclose(fid);


A) OK for default white space

B) Use the 'multipledelimiters...' parameter to skip the repeated tabs

C) Yes %f converts any decimal-based numeric field on input and
interprets exponents correctly -- doc fscanf and friends or a C man page

D) to only read a single record and not all in a file or until a
conversion fails, specify the N argument as 1 for the number of times to
use the format expression in the call. I forget whether there were
groups of data, maybe, too? If so, unless you know there are always an
even number or a fixed number you'll be forced to read a record at a
time or have an error handler to fix up when run into the next one.

If isn't a section then you've got the choice of reading some number as
I (think I recall) originally suggested that is the number you wanted to
average over and just accept the end if not an evenly divisible number
or do the precompute record-counting and counted loop Tideman(?)
suggested...

W/O the 'N' argument, textscan reads until it fails or runs out of data
so in your loop it reads the file and then as I pointed out before since
the eof flag isn't set until the next read attempt it then dies.

--



Point your RSS reader here for a feed of the latest messages in this topic.

[Privacy Policy] [Terms of Use]

© Drexel University 1994-2014. All Rights Reserved.
The Math Forum is a research and educational enterprise of the Drexel University School of Education.