MATLAB

MATLAB is a very powerful tool used for the creation, analysis, and visualization of data. The MATLAB language is an easy-to-learn, high-level programming language. This documentation will not cover how to install or use the basics of MATLAB; there are many tutorials online that can teach you MATLAB. Here is MATLAB's official interactive, online tutorial and a link to their documentation. In this document we will cover how to run MATLAB with and without parallel code.

Running MATLAB

The following instructions are for running MATLAB without any parallel code. This is for programs that require interaction or are too small to benefit from parallel computing.

With GUI

The only place to run MATLAB with a GUI is by connecting to our system through FastX.

Load the MATLALB module: module load matlab
Launch MATLAB by running: matlab

Without GUI

If your MATLAB program needs to be interactive you need to run it through a sinteractive job. MATLAB is too intensive to be run on head node of SPORC, i.e., where you are as soon as you log into sporcsubmit.rc.rit.edu.

Load the MATLALB module: module load matlab
Create a sinteractive job by running: sinteractive
Launch MATLAB by running: matlab -nodisplay -nosplash

If your MATLAB program does not need to be interactive and is too small to benefit from making it run with parallel code, it can be run from the command line. If you are on SPORC this means you will have to create a small job:

#!/bin/bash -l

#NOTE the -l flag!

#SBATCH -J matlab_example #Name of the job

#Standard out and Standard error output files

#SBATCH -o matlab_example_multiple.out

#SBATCH -e matlab_example_multiple.err

# To send emails and notify when done

#SBATCH --mail-user abc1234@rit.edu

#SBATCH --mail-type=END

#Set up time, partition, and memory

#SBATCH -t 1:00:00

#SBATCH -p tier3

#SBATCH --mem=2000M

#Load my environment

module load matlab

#Run the file with no graphical display

matlab -nodisplay -nosplash -singleCompThread -r "my_matlab_file"

Notes

Notice in the last line the option -singleCompThread, this helps to increase the performance of the MATLAB program. Some functions in MATLAB will try to run in multiple threads, but this can cause issues or slow down performance so it is best to stick to one thread. If your code is supposed to run in parallel, do not include this argument as it will make your code not parallel.
To run a job from the terminal not on SPORC, just run the last two lines in the job file above. Load the module and then type matlab command just as it is in the job file.
If your program uses parameters your MATLAB file must wrapped with a function that is the same name as the file. See the parallel code examples below.

Running MATLAB with Parallel Code

The rest of this document covers parallelization of your MATLAB code, i.e., breaking up long processes into chunks that can be processed at the same time. In MATLAB these chunks that process code in parallel are called workers. These workers can be spread out across multiple nodes and cores to utilize the Research Computing resources. This means you can get your results much faster. To do this we will be using MATLAB's Parallel Computing Toolbox.

Parfor loop

MATLAB'S Parallel Computing Toolbox comes with the parfor loop construction. It splits the work done in the for loop among workers to complete the work in parallel. To use, simply replace the for loop in your code with parfor. However, there are some rules:

Parfor loops cannot be nested within each other.
Parfor loop iterations must be independent of each other, that is, the results of one iteration cannot depend on the results of any other iteration. (No recursion!)
Parfor loop variables must be consecutive, increasing integers. The step used for the loop cannot be any number but 1. To work around this, use iValues as the step you would use in the regular for loop:
```
iValues = 0:0.2:1; 
```
```
parfor idx = 1:numel(iValues)
```
```
    i = iValues(idx);
```
```
     ... 
```
```
end
```
You cannot break out of a parfor loop early, that means no break or return statements in your parfor loop. If this creates an issue with your loop try parfeval.

Example Parallel MATLAB Code

The following is a simple MATLAB program to demonstrate the parfor loop. We will be using this code for the rest of the examples in this document and the expected output is based on this MATLAB program.

function [a]=matlab_example(nloop, jobid)

% ===============================================

% Example function to demonstrate parfor loop and parallel code

% nloop: number of iterations

% jobid: Slurm job id used to save output to specific file

% ===============================================

if ischar(nloop) %checking data type of first input

    nloop=str2num(nloop);

end

%preallocate output array in a nloop x 1 matrix

a=zeros(nloop, 1);

% TIME CONSUMING LOOP

tic; %beginning of timing the loop

parfor i=1:nloop

    a(i)=IntensiveFunction();

end

time=toc %end of timing the loop

%save the output to its own file

save(['example_' num2str(jobid) '.out'], 'time', 'a', '-ascii')

delete(gcp)

end

function result=IntensiveFunction()

% Computation intensive calculation

    result = max(abs(eig(rand(500))));

end

Download matlab_example.m.

Running Your Code on One Node

Running in Parallel on One Node

Now this is our first real taste of parallelization. You will notice in this example the keyword parpool. This function creates the workers that process the chunks created by parfor. In the example we ask for one node ( --nodes=1) and that twenty tasks be done on it ( --ntasks-per-node=20). The number of tasks we ask for per node is the number of workers - 1 we want. One task is dedicated to running the MATLAB instance that manages the worker which is why we only request 19 with SLURM_NTASKS-1. The pool of workers is created in a separate file setPool.m. It can be substituted for parpool('local', ${numWorkers}), but a separate file gives us the flexibility we want.

setPool.m

%================================================

% Create a parpool for a MATLAB parallel program

%================================================

pc = parcluster('local')    %use SPORC the local cluster

parpool(pc, str2num(getenv('numWorkers')))

Download setPool.m

matlab_parallel_example.sh

#!/bin/bash -l

#NOTE the -l flag!

#SBATCH -J matlab_parallel #Name of the job

#Standard out and Standard error output files

#%A is the job id and %a is the array index

#SBATCH -o matlab_parallel.out

#SBATCH -e matlab_parallel.err

#Send an email when done

#SBATCH --mail-user abc1234@rit.edu

#SBATCH --mail-type=END

#Set time, partition, and memory

#SBATCH -t

#SBATCH -p tier3

#SBATCH --mem=15000M

#Run on 1 node and make 19 workers

#SBATCH --nodes=1

#SBATCH --ntasks-per-node=20

#Load my environment

module load matlab

# Run the file with no graphical display

# Create a pool of workers on the 'local' SPORC cluster.

# It must be one less than the ntasks-per-nodes because one of those tasks run the

# MATLAB instance itself and cannot be a worker

numWorkers=$((SLURM_NTASKS-1))

export numWorkers

# Run the MATLAB program

matlab -nodisplay -nosplash -r "setPool;matlab_example(200,${SLURM_JOB_ID});exit;"

Expected Output

There will be three created from this example: example_JOBID.out, parforTest.err, and parforTest.out. If everything goes right parforTest.err is empty. The results from all the workers are aggregated into the example_JOBID.out file. The parforTest.out file contains product information about MATLAB and then the statement: Starting parallel pool (parpool) using the 'local' profile ... connected to 19 workers, and then end with the time it took to complete the job. However, if you want to see parallel computing in action, in the body of the parfor loop add: fprintf('Loop %d completed\n', i) and run the job again. You will now see the iterations of the loop were not completed consecutively.

Download matlab_parallel_example.sh

Parallel Computing with Slurm Array on a Single Node

Now we will be using multiple workers and repeatedly running a MATLAB script on one node. The .sh file will look very similar to the previous two. The Slurm Array can be used with code that is not parallel, too.

#!/bin/bash -l

#NOTE the -l flag!

#SBATCH -J multiple_array_test #Name of the job

#Standard out and Standard error output files %A is the job id and %a is the array index

#SBATCH -o multiple_array_%A_%a.out

#SBATCH -e multiple_array_%A_%a.err

# To send emails and notify when done

#SBATCH --mail-user abc1234@rit.edu

#SBATCH --mail-type=END

#SBATCH -t 1:00:00 # Request 1 hour MAX for the job

#Run on tier3

#SBATCH -p tier3

#SBATCH --mem=15000M #memory requirement of jobs in MB

#SBATCH --ntasks=20 #Number of workers -1 to create on each node

#Load my environment

module load matlab

#create ID to identify job

slurmArrayID="${SLURM_ARRAY_JOB_ID}${SLURM_ARRAY_TASK_ID}"

export slurmArrayID

# get number for setPool

numWorkers=$((SLURM_NTASKS-1))

export numWorkers

# Run MATLAB parallel program

matlab -nodisplay -nosplash -r "setPool;matlab_example(200,${slurmArrayID});exit;"

Expected Output

The output for each file will look very similar to the the output for matlab_parallel_example, except there will be 10 different .out and .err files, one for each index in the array.

Download parallel_array_example.sh

Additional Information

MATLAB's Parallel Computing Toolbox comes with many more tools besides the parfor loop. For example:

Single Program Multiple Data (spmd)

spmd gives more control over each of the workers. In the parfor loop, each worker received relatively equal amounts of one larger job. With spmd, each worker can get a unique, specific job assigned to them based on their rank. If you have ten different data sets to analyze, you can have ten workers analyze each set individually in parallel with spmd. Like the parfor loop, spmd cannot be nested within each other and you can break out of the spmd block.

%===========================================

% Example of the spmd command

% Each worker will print a message

% SetPool must be ran before this program

%===========================================

spmd

  lab_count = numlabs;   %number of total workers

  workerid = labindex;   %rank of the worker

  if (workerid == 1)

    fprintf('There are %d workers. \n', lab_count);

end

  fprintf('I am worker %d\n', workerid);

end

%delete the pool and exit matlab

delete(gcp);

exit;

Expected Output

The bash script used for this example is very similar to the one used for Running in Parallel on One Node except ntasks was 4 instead of 20. When completed the out put file for the output file will contain the MATLAB version information, followed by information about the cluster and the pool. Next is the results of the jobs:

Lab 1:
 There are 3 workers
 I am worker 1
Lab 2:
 I am worker 2
Lab 3:
 I am worker 3

As you can see each worker is aware of what rank they are. A simple way to utilize this would be to name your data sets somethings along the lines of dataset_1, dataset_2, ..., etc. and have each worker use

load(['dataset_' num2str(labindex) '.ascii']).

Download spmd_example.m.

Download spmd_example.sh.

parfeval

parfeval is useful for when you want to run a loop that you can stop early. For example, if you are analyzing a very large data set, you may want to stop when the results are 'good enough' instead of waiting for the entire set to be completed. parfeval is also useful for running functions in the background because it doesn't block MATLAB from continuing to work. parfeval will split up the workers in the pool itself.

Syntax

f = parfeval(fcn,numout,in1,in2,...)

fnc: the function to execute

numout: expected number of outputs from the function

in1, in2: the parameters for the function

f: a future object. By itself, it doesn't mean a lot, the data has to be extracted from it when it's ready with fetchNext(f).

If you want to break out of a loop using parfeval use cancel(f) to stop the evaluation of the future object.

Example

This example shows how you can use parfeval to evaluate a function an get the results as they are available.

%=================================

% Simple example of parfeval

% From MATLAB documentation

% must run setPool before this

%=================================

% evaluate the magic function 10 times

for idx = 1:10

  f(idx) = parfeval(@magic, 1, idx);

end

% preallocate place to store results

magicResults = cell(1,10);

% get the results and put them in the array

for idx = 1:10

  [completedIdx, value] = fetchNext(f);

  magicResults{completedIdx} = value

  fprintf('Got results with index %d.\n', completedIdx);

end

%clean up the pool and exit

delete(gcp);

exit;

Expected Output

The bash script for this example is identical to the script for previous spmd example. The output file has MATLAB's version information, followed by the cluster and pool properties. The actual results of the MATALAB should be 'Got result with index: 1', then 2, then 3, ... etc., up to 10. If this was a much larger job, then the indexes may not be in order; it would all depend on which future object was ready for fetchNext(f) first.

Download parfeval_example.m

Download parfeval_example.sh

Quick Guide

Brief explanation of terms to know when using MATLAB's Parallel Computing Toolbox.

worker	The MATLAB computational engine that processes the code. Can also be called a lab. Each worker is assigned a number called its rank.
numlabs	Returns the total number of workers available.
labindex	Returns the rank of the worker.
parpool	The parallel pool of workers. It is created in the MATLAB program with parpool('local', #ofWorkers). The number of workers is the number of cpus requested on the node - 1.
gcp	MATLAB function that will get the current pool. At the end of the parallel code using delete(gcp) will neatly shutdown all the workers.
parfor	The parallel for loop. Splits the iterations of the for loop among the workers to be done in parallel. The step of the iteration must be +1, the iterations cannot rely on one another, parfor loops cannot be nested, and you cannot break out of the loop early.
spmd	Single Program Multiple Data. Allows for control over each worker. Use the worker's rank to assign jobs. Useful for when you want to do the same thing to different data sets. Like parfor: cannot nest spmd blocks in each other and cannot break out of them.
parfeval	Parallel Function Evaluation (not official, just assuming that what it stands for) will allow you to run functions in parallel without having MATLAB be blocked from running other things. Call parfeval as many times as you want the function to run in a loop and call fetchNext to get the results.

Trouble Shooting

Sometimes things just don't go right, here are some tips to help. If none of these tips help email us at rc-help@rit.edu.

Check the .err files for your job, they will help you the most. Often it will explicitly state the error that occurred in a specific file on a specific line. If you don't understand the error message try to look it up online. Always double check your spelling and capitalization.
Run sacct. This command tells you the state of each of your jobs and the exit code. You might get a state of OUT_OF_ME+ and an exit code of 125. This means you need to increase the amount of memory you request in your .sh file. Remember #SBATCH --mem uses megabytes by default. Append M, G, or T to the end of the number to request megabytes, gigabytes, or terabytes.
Your job might not be running because there are no resources available. Run squeue and look under the REASON column. If you see (Resources) that means you need to wait for other jobs to finish before yours can run. You can also see this by running sinfo. If you look in the rows for the tier you are trying to run your job in, you'll see none with the STATE idle or mix.

MATLAB

Running MATLAB

With GUI

Without GUI

Notes

Running MATLAB with Parallel Code

Parfor loop

Example Parallel MATLAB Code

Running Your Code on One Node

Running in Parallel on One Node

setPool.m

matlab_parallel_example.sh

Expected Output

Parallel Computing with Slurm Array on a Single Node

Expected Output

Additional Information

Single Program Multiple Data (spmd)

Expected Output

parfeval

Syntax

Example

Expected Output

Quick Guide

Trouble Shooting

More Reading