Search All of the Math Forum:
Views expressed in these public forums are not endorsed by
Drexel University or The Math Forum.
|
|
Chuck37
Posts:
84
Registered:
2/23/10
|
|
Re: spmd overhead and sharing small amounts of data
Posted:
Dec 20, 2012 10:44 AM
|
|
Thanks, that will help I think.
I'm using a dual hex core machine. It has 12 real processors but shows as 24 on Linux due to hyperthreading. I'm using 10 workers so I can have them to myself while other people use the others for small stuff. I do see them go to 100% sometimes, just not during most of my loop going in and out of spmd.
By the way, is it possible to plot while still in the spmd? I like to put up a plot as I go to monitor progress.
Edric M Ellis <eellis@mathworks.com> wrote in message <ytw623xcglg.fsf@uk-eellis0l.dhcp.mathworks.com>...
>
> What size MATLABPOOL did you open? One thing to note: MATLAB doesn't
> consider hyperthreaded cores when choosing how many processes to make
> the default for the local scheduler - so even if all the workers were
> completely busy, you might not see more than 50% utilisation. (That's
> what happens here on my system - the OS thinks it has 12 cores, but
> really it has 6 hyperthreaded cores, so the default is for MATLAB to
> launch 6 local workers).
>
> > Any alternatives? For one, is there a way to gather directly worker
> > to worker without exiting spmd and bringing it all to the master?
>
> You can use functions like either labBroadcast to send data from one
> worker to all the others, and gcat or gop to perform 'reduction'
> operations. Very briefly, here's how they work
>
> % assume matlabpool size 4
> spmd
> data = rand();
>
> % broadcast from lab 1, x1 gets the value from 'data' on lab 1.
> x1 = labBroadcast(1, data);
>
> % concatenation - each lab gets x2 = [1 2 3 4];
> x2 = gcat(labindex);
>
> % general reduction - in this case, each lab gets
> % x3 = 1 + 2 + 3 + 4
> x3 = gop(@plus, labindex);
> end
>
> > The data is small and functions cheap, so maybe I'd be ahead to let
> > everyone do the same computations just to keep things flowing.
>
> That sounds like a very good idea.
>
> Cheers,
>
> Edric.
|
|
|
|
