INTERACTIVE JOBS - MATLAB client on the cluster

Verify in MATLAB that you have the Statistics and Machine Learning toolbox, Parallel computing toolbox, Optimization toolbox e FSDA toolbox installed.

Access the MATLAB ADDONS installation and download the Parallel Computing Toolbox plugin for MATLAB Parallel Server with Slurm, (see the screenshot below).

To verify if the setup is working (the FSDA toolbox that contains the functions that configure and instantiate the remote cluster must be installed as well)

During the installation process of Slurm Plugin you are asked to create a profile to connect to the cluster, but for Windows OS is not (yet) available, so click on "Cancel".

At the end of the installation process the screen below should appear.

The first step, is to download an SSH client to access HPC resources: MobaXterm https://mobaxterm.mobatek.net/ is suggested.

The installer edition allows the user to install the Mobaxterm software on your system automatically.

The next step is the creation of the asymmetric SSH keys (public and private)

In a Windows environment the procedure is the same as in Linux, remember to use the double quotes instead of single quotes. More precisely after running (Windows + R) cmd.exe

in the cmd console enter the following command:

ssh-keygen -t rsa -P "" -f %USERPROFILE%\.ssh\id_rsa

The result under the folder .ssh …

C:\Users\<user>\.ssh\

… will be the following (in the screenshot below it is assumed that the <user> is 'Marco').

Before proceeding further be sure that you are connected to the unipr.it domain with a cable connection inside your UNIPR Department or alternatively you are using a VPN connection.

proceed to copy the public key contained in the file id_rsa.pub inside the file authorized_keys under the folder .ssh in the Linux HPC account space.

For this purpose start MobaXterm and after logging into your account on the server gui.hpc.unipr.it type the following in the console:

 cd .ssh
 nano authorized_keys
 

see screenshot below:

Open the nano editor in the console and with CTRL+V paste inside this file the string copied from the local file id_rsa.pub (see screenshot below).

To save the file authorized_keys, type CTRL+Z, Y e then ENTER.

At this point you need to install the following open-source apps to share the cluster file system on the client machine

These apps are available at the following address:

https://github.com/billziss-gh/sshfs-win

Next, you need to create a local share, so start connecting a network drive and map it with a letter.

After double clicking on "This PC" choose the "Map Network Drive" option

In the screen that appears choose a letter in the drop-down box called Unit (for example H:)

and in the "Folder" textbox enter the following path

\\sshfs\<nome utente>@gui.hpc.unipr.it

To make this share permanent, you need to tick the "Reconnect at login" checkbox.

After selecting "This PC" the network share H: should be set up as follows:

The UNIPRconfigCluster and UNIPRrunCluster functions of the FSDA toolbox allow respectively to: create the cluster profile of the MATLAB Paraller Server and to connect the client machine to the remote cluster to perform parallel computing on the remote server.

For example, by invoking the function UNIPRconfigCluster with the provided username in MATLAB we are able to create a remote cluster with all the default options:

  UNIPRconfigCluster('mirko.anello')
  

The function UNIPRconfigCluster automatically calls the companion UNIPRrunCluster function, developed by Fabio Spataro and Fausto Pagani of the UO Centro di Calcolo that creates a special tunnelling connection between the local machine and a machine belonging to the HPC cluster: the two black windows shells, state that everything is fine.

If the first time that you use this function a message appears with the login request, simply enter the password associated with your personal unipr account.

To verify that the cluster profile was created correctly, from the MATLAB HOME tab select the option Parallel → Create and Manage Clusters:

beside the local cluster you should see another cluster named as the relase version of MATLAB, see screenshot below.

If needed, you can customize these parameters using the MATLAB GUI, manually editing the cluster profile until you find the optimum configuration.

Alternatively, all these elements can be set up programmatically by calling the functionUNIPRconfigCluster via namepairs, (you can find the detailed HTML documentation and syntax of this function at the following web address: http://rosa.unipr.it/FSDA/UNIPRconfigCluster.html).

For example the command below:

 
  UNIPRconfigCluster('mirko.anello','tasksxnode', 1, 'memxcpu','4G', 'NumWorkers', 8, 'NumThreads', 1)
  

creates a remote cluster, with 1 node, 1 thread per worker (i.e. CPU core) 8 workers and 4 GBytes per worker for username mirko.anello .

At this point, through the command:

 parpool

the parallel pool is instantiated and started on the HPC cluster

After a while, (please be patient and remember that resources take some time to allocate!) in the MATLAB command window you should see the following image:

The cluster is now ready: let's see a simple example:

FSRmdrrs_example.m

   rng('default')
   b1=[1 1];
   b2=[1 2.6];
   n1=40;
   n2=50;
   s1=0.1;
   s2=0.1;
   X1=rand(n1,1);
   X2=rand(n2,1);
   y1=randn(n1,1)*s1+b1(1)+b1(2)*X1;
   y2=randn(n2,1)*s2+b2(1)+b2(2)*X2;
   hold('on')
   plot(X1,y1,'o');
   plot(X2,y2,'o');
   title('Two simulated regression lines')
   y=[y1;y2];
   X=[X1;X2];
   figure
   [out]=FSRmdrrs(y,X,'constr','','nsimul',1000,'init',10,'plots',1,'cleanpool',false,'msg',0)  

After a few seconds of processing, the following figures should appear:

please check all the details about the syntax of the FSRmdrrs function of the FSDA toolbox on:

http://rosa.unipr.it/FSDA/FSRmdrrs.html

go back one level