Drexel dragonThe Math ForumDonate to the Math Forum



Search All of the Math Forum:

Views expressed in these public forums are not endorsed by Drexel University or The Math Forum.


Math Forum » Discussions » Software » comp.soft-sys.matlab

Topic: PCT output data transfer
Replies: 2   Last Post: Jun 27, 2014 12:31 PM

Advanced Search

Back to Topic List Back to Topic List Jump to Tree View Jump to Tree View   Messages: [ Previous | Next ]
Edric Ellis

Posts: 697
Registered: 12/7/04
Re: PCT output data transfer
Posted: Jun 26, 2014 3:29 AM
  Click to see the message monospaced in plain text Plain Text   Click to reply to this topic Reply

"Sujit Das" <SujitRDas@gmail.com> writes:
> Edric M Ellis <eellis@mathworks.com> wrote in message <ytwvbrpe99e.fsf@uk-eellis-deb7-64.dhcp.mathworks.com>...
>> "Sujit Das" <SujitRDas@gmail.com> writes:
>>

>> > Interesting. Yes, we are using PARFOR in our application. The size of
>> > result set that each worker (Linux) needs to return to client (Wintel
>> > or Linux) could be as large as 1 to 2 GB. How does the worker return
>> > this result set (which are in-memory Matlab variables) to client? Is
>> > the transfer taking place directly from worker memory to client memory
>> > (RMI??) without ever hitting the file or disk I/O unlike what happens
>> > in the batch jobs?

>>
>> Yes, when results are returned from a PARFOR loop, a type of RMI is used
>> to transfer the data from the workers back to the client, no on-disk
>> storage is used. However, note that there is some overhead to doing this
>> - the MATLAB data needs to be turned into a serialized form for
>> transmission - this is part of the same process that happens when
>> variables are saved to disk, so if for example you were transmitting an
>> object with a very slow SAVEOBJ, then you'd see that slowness.
>>

> Thanks Edric. That is very insightful. Yes, we realize that data
> serialization does happen during transfer. We have also encountered
> the limit of 2 GB ( I believe) in 64-bit workers during serialization.


If you have 64-bit workers and a 64-bit client, the limit no longer
applies (it was removed in R2013a).

Cheers,

Edric.



Point your RSS reader here for a feed of the latest messages in this topic.

[Privacy Policy] [Terms of Use]

© Drexel University 1994-2014. All Rights Reserved.
The Math Forum is a research and educational enterprise of the Drexel University School of Education.