Date: Feb 20, 2013 10:26 PM
Author: Bill Rowe
Subject: Re: Obtaining Random LIne from A file
On 2/19/13 at 1:00 AM, firstname.lastname@example.org (Ramiro) wrote:
>Thank you so much for the reply. My files are 50MB each, I don't
>think ReadList would work for my purposes, it would be too slow.
Have you tried and found ReadList to be too slow? If you haven't
you should try it. You might well find it is faster than you think.
Specifically, I was using Mathematica version 7 running on OS X
10.5 using a MacBook Pro with 8GB of installed memory and a 2.53
GHz Intel Core Duo processor to analyze files consisting of 12
channels of 32 bit data sampled at 200KSPS over a typical
sampling period of 5 sec. If you do the math, you will see these
were 48MB data files. Each test run would consist of 10-20 such files.
I was able to complete the analysis needed in 1-2 hours. The
time was not spent waiting for my MacBook Pro to read the data.
Instead, it was spent reviewing plots etc and adjusting analysis
steps based on the plots and other results. The analysis
involved a rather complex non-linear fitting routine which could
not be relied on to find an optimal solution without some tweaking.