> Edric M Ellis <firstname.lastname@example.org> wrote in message <email@example.com>... >> "Sujit Das" <SujitRDas@gmail.com> writes: >> >> > We are using PCT and experience slowness in transfer of result data >> > from workers to clients. Does anyone know how Matlab transfers output >> > data from workers to clients in PCT? Does it save the output data to >> > (temp) files and then use FTP or some other file transfer mechanism to >> > transfer to client? >> >> Which cluster type are you using? >> >> MJS (jobmanager) stores result data in a database on the host where the >> jobmanager itself is running, and the data is sent over the network. All >> other schedulers use the file-system in one way or another. (Most save >> results directly to disk, but some of the third-party scheduler >> integrations using the "generic" cluster can use SCP to transfer data). >> > Thanks Edric. We use both MJS (for Wintel) and LFS for linux grid. Is this > mechanism of transfer documented somewhere by Mathworks? For large results the > mechanism of writing to disk (files or DB) and then transferring over the > networks could be very inefficient. Another question: Does a worker send back > the results back to the client as soon as a slice is done? I assume that is the > case. If not, then all the workers will have to be done producing the results > before the results are sent back to the client.
The format is not explicitly documented - although for everything other than MJS, standard .MAT files are used.
Also, note that this storage is used only for results of running a BATCH job (or other job created using one of the create*Job methods). If you're talking about parallel pools and PARFOR etc., then there is no on-disk / database storage of results at all (you mentioned 'slices' which made me think that's what you're referring to).
In the PARFOR case, workers are given a group of slices to work on together, and return the results of the entire group rather than one slice at a time.