> Edric M Ellis <firstname.lastname@example.org> wrote in message <email@example.com>... >> That's basically it. Do you have some figures for how long it takes >> against number of workers? What cluster type are you using? > > The problem space is: > > N=total number of input vectors that needs to be processed > n<=N number of input vector in a slice > W= number of parallel workers > > N=1000, n=10, W=50, then the pool closing time is ~60 seconds. > N=1000, n=10, W=100, then the pool closing time is ~140 seconds. > N=22,260, n=10, W=100, then the pool closing time is ~150 seconds. > N=50,000, n=100, W=100, then the pool closing time is ~140 seconds. > N=100,000, n=100, W=100, then the pool closing time is ~530 seconds.
Hm, I'm surprised that the pool closing time is dependent on the workload. This suggests to me that perhaps you're running low on memory somewhere. I tried the following simple experiment (I only have 16 cores on my machine):
parpool('local', 16); data = rand(50000,100); parfor idx = 1:100, x(idx) = mean(data(:,idx)); end tic, delete(gcp('nocreate')); toc
and observed a pool closing time of 4.1 seconds. Then, this
showed a pool closing time of 4.0 seconds. Perhaps I'm missing something about what you're doing.
> Earlier in the thread we discussed how each worker transfers results > back to client. Understanding is that each worker finishes processing > all the slices before sending the result set to client via in-memory > transfer.
Workers in PARFOR loops get sent a chunk of slices - they work on that chunk and then send the results back immediately using a socket connection to the client. The PARFOR machinery attempts to split the loop up into roughly 3 * NumWorkers chunks of slices.
> Now the question is around transfer of input variables from client to a > worker. The client passes a large input variable (~1 GB) to each slice given to > a worker. This variable contains static data. Does Parfor pass this static > variable to the worker each time a slice is given to the worker?
No - a PARFOR 'broadcast' variable is sent once to each worker at the start of the loop.
> That will have performance impact in transferring the large data over > the network (does the transfer takes place in-memory without hitting > i/o?).
All PARFOR transfers are currently done over a socket connection.