The Math Forum

Search All of the Math Forum:

Views expressed in these public forums are not endorsed by NCTM or The Math Forum.

Math Forum » Discussions » Software » comp.soft-sys.matlab

Notice: We are no longer accepting new posts, but the forums will continue to be readable.

Topic: PCT output data transfer
Replies: 2   Last Post: Jun 27, 2014 12:31 PM

Advanced Search

Back to Topic List Back to Topic List Jump to Tree View Jump to Tree View   Messages: [ Previous | Next ]
Edric Ellis

Posts: 721
Registered: 12/7/04
Re: PCT output data transfer
Posted: Jun 26, 2014 3:29 AM
  Click to see the message monospaced in plain text Plain Text   Click to reply to this topic Reply

"Sujit Das" <> writes:
> Edric M Ellis <> wrote in message <>...
>> "Sujit Das" <> writes:

>> > Interesting. Yes, we are using PARFOR in our application. The size of
>> > result set that each worker (Linux) needs to return to client (Wintel
>> > or Linux) could be as large as 1 to 2 GB. How does the worker return
>> > this result set (which are in-memory Matlab variables) to client? Is
>> > the transfer taking place directly from worker memory to client memory
>> > (RMI??) without ever hitting the file or disk I/O unlike what happens
>> > in the batch jobs?

>> Yes, when results are returned from a PARFOR loop, a type of RMI is used
>> to transfer the data from the workers back to the client, no on-disk
>> storage is used. However, note that there is some overhead to doing this
>> - the MATLAB data needs to be turned into a serialized form for
>> transmission - this is part of the same process that happens when
>> variables are saved to disk, so if for example you were transmitting an
>> object with a very slow SAVEOBJ, then you'd see that slowness.

> Thanks Edric. That is very insightful. Yes, we realize that data
> serialization does happen during transfer. We have also encountered
> the limit of 2 GB ( I believe) in 64-bit workers during serialization.

If you have 64-bit workers and a 64-bit client, the limit no longer
applies (it was removed in R2013a).



Point your RSS reader here for a feed of the latest messages in this topic.

[Privacy Policy] [Terms of Use]

© The Math Forum at NCTM 1994-2018. All Rights Reserved.