I have a CSV file that is >100 GB which I need to convert to a .mat. Unfortunately, no one machine has enough memory to read the entire file into disk, and then write out to a .mat.
Is there a way to more or less convert a CSV a few lines at a time into a .mat a "little bit at a time"?
On a side note, I find saving a large dataset as a .mat (e.g. a cell array of 20x1e7 strings), which is around 40 GB of data, is MUCH slower than reading the same dataset in from a CSV file. How can this be? Is there something I can do about it?