Two approaches spring to mind. One is judicious use of ReadList, array packing & memory cleanup. Another is through database link and a SQL lite database.
Other authors have apparently had success using a Java-based approach.
On 2/04/2013 15:25, Brentt wrote: > Hi, I am writing a package which requires for its use access to a 80 mb > data set. > > Is there a standard way to associate a package with a set of data? I > figured out a way that involves pulling the data from dat files available > online and parsing them into a more readily available mathematica form. But > I'd rather be able to have the user just have the data with the package. > What is the best way to go about this? I thought .mx files but I noticed > the docs say that they don't work across platforms. > >