"Chuck37 " <email@example.com> wrote in message news:firstname.lastname@example.org... > I have a big piece of never changing data that I'd like to be used by all > the workers in a parallel setting. Since I'm only working with local > workers, I don't understand why I have to eat the huge latency associated > with transferring the data to workers each time the parfor loop is called. > Can someone explain? > > I tried to use spmd to send data once at the beginning and have it stay > there, but the data is kind of big (~2 GB going to 10 workers), and I got > "error during serialization" anyway. Is there a solution to this problem > with local workers where they can all access the data from the same > memory? Accesses are infrequent, so simultaneous access shouldn't cause a > big slowdown. > > Any ideas would be great. > > My setup is something like this by the way: > > M = big data; > for x = 1:m > stuff > parfor y = 1:n > a(y) = function(M,a(y)); > end > stuff > end > > Parfor is presently worse than 'for' because of the overhead from sending > M every time.
Can you use PARFOR for the loop over x instead of the loop over y?