Search All of the Math Forum:
Views expressed in these public forums are not endorsed by
NCTM or The Math Forum.



Re: Obtaining Random LIne from A file
Posted:
Feb 20, 2013 10:26 PM


On 2/19/13 at 1:00 AM, ramiro.barrantes@gmail.com (Ramiro) wrote:
>Thank you so much for the reply. My files are 50MB each, I don't >think ReadList would work for my purposes, it would be too slow.
Have you tried and found ReadList to be too slow? If you haven't you should try it. You might well find it is faster than you think.
Specifically, I was using Mathematica version 7 running on OS X 10.5 using a MacBook Pro with 8GB of installed memory and a 2.53 GHz Intel Core Duo processor to analyze files consisting of 12 channels of 32 bit data sampled at 200KSPS over a typical sampling period of 5 sec. If you do the math, you will see these were 48MB data files. Each test run would consist of 1020 such files.
I was able to complete the analysis needed in 12 hours. The time was not spent waiting for my MacBook Pro to read the data. Instead, it was spent reviewing plots etc and adjusting analysis steps based on the plots and other results. The analysis involved a rather complex nonlinear fitting routine which could not be relied on to find an optimal solution without some tweaking.



