createJob

Create independent job on cluster

Syntax

job = createJob(myCluster)

job = createJob(myCluster,Name,Value)

Description

job = createJob(myCluster) creates an independent job on the identified cluster and an independent job object on the client.

The job's data is stored in the location specified by the cluster's JobStorageLocation property.

job = createJob(myCluster,Name,Value) also specifies the Profile or sets the independent job properties using one or more optional name-value arguments when creating the job. For a listing of the valid properties, see the parallel.Job object reference page.

In you specify a Profile and other properties, the values you specify in these property name-value pairs can override the values in the profile.

Examples

Create and Run Job

Create and run a job on a cluster.

Create an independent job object using the default profile.

c = parcluster;
j = createJob(c);

Add tasks to the job.

for i = 1:10
    createTask(j,@rand,1,{10});
end

Run the job.

submit(j);

Wait for the job to finish, and retrieve the job results.

wait(j);
out = fetchOutputs(j);

Display the random matrix returned from the third task.

disp(out{3})

    0.9730    0.1454    0.7662    0.9601    0.0836    0.9843    0.2656    0.8780    0.1748    0.2112
    0.7104    0.6426    0.9654    0.9145    0.6432    0.4296    0.4141    0.6507    0.7349    0.9729
    0.3614    0.3250    0.9843    0.8676    0.5964    0.1125    0.1687    0.2207    0.8327    0.7751
    0.2934    0.8229    0.9601    0.1388    0.8854    0.5494    0.1708    0.6235    0.0617    0.1119
    0.1558    0.8728    0.1856    0.2842    0.8472    0.5912    0.8205    0.3761    0.1292    0.8711
    0.3421    0.2005    0.9495    0.4687    0.8467    0.1968    0.7528    0.0899    0.8172    0.3892
    0.6071    0.9987    0.2639    0.0828    0.7554    0.7506    0.6331    0.0035    0.8602    0.4818
    0.5349    0.8446    0.1578    0.1702    0.6472    0.0087    0.8810    0.3619    0.6499    0.8299
    0.4118    0.9079    0.4784    0.5140    0.5163    0.5712    0.1639    0.7487    0.6486    0.7195
    0.1020    0.0982    0.2994    0.2809    0.4726    0.4986    0.3940    0.0096    0.3214    0.6705

Delete the job.

delete(j);

Create a Job with Attached Files

Create an independent job and attach files in addition to those specified in the default profile.

c = parcluster;
j = createJob(c,'AttachedFiles',...
        {'myapp/folderA','myapp/folderB','myapp/file1.m'});

Input Arguments

myCluster — Cluster

parallel.Cluster object

Cluster, specified as a parallel.Cluster object that represents cluster compute resources. To create the parallel.Cluster object, use the parcluster function.

Example: myCluster = parcluster; job = createJob(myCluster);

Data Types: parallel.Cluster

Name-Value Arguments

Specify optional pairs of arguments as Name1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Before R2021a, use commas to separate each name and value, and enclose Name in quotes.

Example: job = createJob(myCluster, 'AdditionalPaths',["/additional/path1","/additional/path2"]) creates an independent job on the cluster represented by myClusterand adds path1 and path2 folders to all the workers MATLAB^® search path.

Standard Name-Value Arguments

Profile — Cluster profile with properties to apply to job

profile name

Cluster profile with property values to apply to the job object, specified as the comma-separated pair consisting of 'Profile' and the profile name as a character vector or string. If you do not specify a profile, and the cluster has a value specified in its 'Profile' property, MATLAB automatically applies the cluster's profile. For more details about defining and applying profiles, see Discover Clusters and Use Cluster Profiles.

Example: createJob(myCluster,'Profile',"myProfileName")

Data Types: char | string

Property Name-Value Arguments

AdditionalPaths — Paths to add to MATLAB search path of workers

character vector | string scalar | string array | cell array

Paths to add to MATLAB search path of the workers that execute the job, specified as character vector, string, string array, or cell array of character vectors.

If the client and workers have different paths to the same folder, you must specify the folder using the path on the workers. For example, if the path to the folder is /shared/data on the client and /organization/shared/data on the workers, specify "/organization/shared/data".

If you specify relative paths such as "../myFolder", MATLAB resolves the paths relative to the current working directory on the workers.

Example: "AdditionalPaths",["/path/to/folder1","path/to/folder2"]

Data Types: char | string | cell

AttachedFiles — Files and folders to attach to job

character vector | string scalar | string array | cell array

Files and folders to attach to the job object, specified as a character vector, string, string array, or cell array of character vectors. This argument makes the identified files and folders available for the workers executing the tasks in the job.

If you specify the Profile property, and the profile or the myCluster profile already has values for the AttachedFiles property, MATLAB appends the files and folders you specify here to the existing AttachedFiles values.

Example: "AttachedFiles", {'data_set.m','folder_for_workers'}

Data Types: char | string | cell

Note

The properties listed here are only a subset. For a full list of valid properties, see the parallel.Job object reference page.

Tips

• To develop and test your code, you can run batch job on a local cluster on your client machine. If you close your MATLAB session, any batch jobs using the local cluster also stop immediately.

• When you offload work to a remote cluster, you can close the MATLAB client session while the job is processing and retrieve information from a batch job later or in a new client session.

• When you offload computations to workers, any files that the client needs for computations must also be available on workers. By default, the client attempts to detect and attach these files. To turn off automatic detection, set the AutoAttachFiles property to false. If the software cannot find all the files, or if sending files from client to worker is slow, use one of these options.

  • If the files are in a folder that is not accessible on the workers, set the AttachedFiles property. The cluster copies each file you specify from the client to the workers.

  • If the files are in a folder that is accessible on the workers, you can set the AdditionalPaths property instead. Use the AdditionalPaths property to add paths to the MATLAB search path for each worker and avoid copying files unnecessarily from the client to the workers.

Version History

Introduced before R2006a