INTERACTIVE JOBS

Questa pagina è in sola lettura. Puoi visualizzare il sorgente, ma non puoi modificarlo. Contatta l'amministratore se pensi che ci sia un errore.
====== INTERACTIVE JOBS - MATLAB client on the cluster ======
===== MATLAB PREREQUISITES=====

Verify in MATLAB that you have the **Statistics and Machine Learning toolbox**, **Parallel computing toolbox**, **Optimization toolbox** e **FSDA toolbox** installed.


{{calcoloscientifico:userguide:matlab:parallel:installed_toolboxes.png?direct&600|}} 

Access the MATLAB ADDONS installation and download the **Parallel Computing Toolbox plugin for MATLAB Parallel Server with Slurm**, (see the screenshot below).



{{calcoloscientifico:userguide:matlab:parallel:matlab_addons.png?600|}} \\


{{calcoloscientifico:userguide:matlab:parallel:slurm_toolbox.png?800|}} \\


To verify if the setup is working (the FSDA toolbox that contains the functions that configure and instantiate the remote cluster must be installed as well)

During the installation process of Slurm Plugin you are asked to create a profile to connect to the cluster, but for Windows OS is not (yet) available, so click on "Cancel".

At the end of the installation process the screen below should appear.

{{calcoloscientifico:userguide:matlab:parallel:slurm_profile.png?600|}} \\




===== CREATE THE ASYMMETRIC SSH KEYS ON THE CLIENT =====

The first step, is to download an SSH client to access HPC resources: 
MobaXterm [[https://mobaxterm.mobatek.net/]] is suggested.

{{calcoloscientifico:userguide:matlab:parallel:mobaxterm.png?direct&600|}}

The installer edition allows the user to install the **Mobaxterm** software on your system automatically.


===== Creation of the asymmetric SSH keys=====


The next step is the creation of the asymmetric SSH keys (public and private)

In a Windows environment the procedure is the same as in Linux, remember to use the double quotes instead of single quotes.
More precisely after running (Windows + R) cmd.exe

in the cmd console enter the following command:


  ssh-keygen -t rsa -P "" -f %USERPROFILE%\.ssh\id_rsa


The result under the folder ''.ssh'' ...

  C:\Users\<user>\.ssh\


... will be the following (in the screenshot below it is assumed that the <user> is 'Marco').



{{calcoloscientifico:userguide:matlab:parallel:local_ssh_keys.png?direct&600|}}


__**Before proceeding further be sure that you are connected to the unipr.it domain with a cable connection inside your UNIPR Department or alternatively you are using a VPN connection.**__

proceed to copy the public key contained in the file ''id_rsa.pub'' inside the file ''authorized_keys'' under the folder ''.ssh'' in the Linux HPC account space.

For this purpose start **MobaXterm** and after logging into your account on the server ''gui.hpc.unipr.it'' type the following in the console:

   cd .ssh
   nano authorized_keys
   
see screenshot below:  

{{calcoloscientifico:userguide:matlab:parallel:moba_login_ssh.png?600|}}

Open the ''nano'' editor in the console and with ''CTRL+V'' paste inside this file the string copied from the local file ''id_rsa.pub'' (see screenshot below).


{{calcoloscientifico:userguide:matlab:parallel:moba_public_key.png?600|}}

To save the file ''authorized_keys'', type ''CTRL+Z'', ''Y'' e then ''ENTER''.


At this point you need to install the following open-source apps to share the cluster file system on the client machine

These apps are available at the following address:

[[https://github.com/billziss-gh/sshfs-win]] 


{{calcoloscientifico:userguide:matlab:parallel:sshfs_download.png?direct&800|}}


{{calcoloscientifico:userguide:matlab:parallel:sshfs_asset_big.png?direct&800|}}

===== Windows 11 requires at least the following version of winfsp =====


{{calcoloscientifico:userguide:matlab:parallel:win_fsp_asset.png?direct&800|}}


Next, you need to create a local share, so start connecting a network drive and map it with a letter.

After double clicking on "This PC" choose the "Map Network Drive" option

{{calcoloscientifico:userguide:matlab:parallel:connetti_unita_rete.png?600|}}



In the screen that appears choose a letter in the drop-down box called ''Unit'' (for example ''H:'')

and in the "Folder" textbox enter the following path


  \\sshfs\<nome utente>@gui.hpc.unipr.it
  
To make this share permanent, you need to tick the "Reconnect at login" checkbox.
 
  
{{calcoloscientifico:userguide:matlab:parallel:connect_disk_h.png?direct&600|}}




After selecting "This PC" the network share ''H:'' should be set up as follows:


{{calcoloscientifico:userguide:matlab:parallel:disk_h_share.png?direct&600|}}



=====The functions UNIPRconfigCluster and UNIPRrunCluster===== 

The ''UNIPRconfigCluster'' and ''UNIPRrunCluster'' functions of the **FSDA toolbox** allow respectively to: create the cluster profile of the **MATLAB Paraller Server** and to connect the client machine to the remote cluster to perform //parallel computing// on the remote server.


For example, by invoking the function ''UNIPRconfigCluster'' with the provided username in MATLAB we are able to create a remote cluster with all the default options:

    UNIPRconfigCluster('mirko.anello')
    
The function ''UNIPRconfigCluster'' automatically calls the companion ''UNIPRrunCluster'' function, developed by **Fabio Spataro** and **Fausto Pagani** of the **UO Centro di Calcolo** that creates a special **tunnelling** connection between the local machine and a machine belonging to the HPC cluster: the two black windows shells, state that everything is fine.


{{calcoloscientifico:userguide:matlab:parallel:tunnelling_enabled.png?direct&600|}}

If the first time that you use this function a message appears with the login request, simply enter the password associated with your personal ''unipr'' account.

To verify that the cluster profile was created correctly, from the MATLAB HOME tab select the option Parallel -> Create and Manage Clusters:

beside the local cluster you should see another cluster named as the relase version of MATLAB, see screenshot below.


{{calcoloscientifico:userguide:matlab:parallel:cluster_profile.png?direct&600|}}

If needed, you can customize these parameters using the MATLAB GUI, manually editing the cluster profile until you find the optimum configuration.

Alternatively, all these elements can be set up programmatically by calling the function''UNIPRconfigCluster'' via //namepairs//, (you can find the detailed HTML documentation and syntax of this function at the following web address: [[http://rosa.unipr.it/FSDA/UNIPRconfigCluster.html]]).

For example the command below:
   
    UNIPRconfigCluster('mirko.anello','tasksxnode', 1, 'memxcpu','4G', 'NumWorkers', 8, 'NumThreads', 1)
    
creates a remote cluster, with 1 node, 1 thread per worker (i.e. CPU core) 8 workers and 4 GBytes per worker for username ''mirko.anello'' .

At this point, through the command:

   parpool

the **parallel pool** is instantiated and started on the **HPC cluster**

After a while, (please be patient and remember that resources take some time to allocate!) in the MATLAB command window you should see the following image:


{{calcoloscientifico:userguide:matlab:parallel:cluster_object.png?direct&600|}}

The cluster is now ready: let's see a simple example:
<code matlab FSRmdrrs_example.m>
   rng('default')
   b1=[1 1];
   b2=[1 2.6];
   n1=40;
   n2=50;
   s1=0.1;
   s2=0.1;
   X1=rand(n1,1);
   X2=rand(n2,1);
   y1=randn(n1,1)*s1+b1(1)+b1(2)*X1;
   y2=randn(n2,1)*s2+b2(1)+b2(2)*X2;
   hold('on')
   plot(X1,y1,'o');
   plot(X2,y2,'o');
   title('Two simulated regression lines')
   y=[y1;y2];
   X=[X1;X2];
   figure
   [out]=FSRmdrrs(y,X,'constr','','nsimul',1000,'init',10,'plots',1,'cleanpool',false,'msg',0)  
</code>   

After a few seconds of processing, the following figures should appear:
   
{{calcoloscientifico:userguide:matlab:parallel:fsrmdrrs.png?direct&600|}}   
    
please check all the details about the syntax of the ''FSRmdrrs'' function of the **FSDA toolbox** on:

[[http://rosa.unipr.it/FSDA/FSRmdrrs.html]]

/*
 * external link (replaced by internal link)
[[https://www.hpc.unipr.it/dokuwiki_test/doku.php?id=calcoloscientifico:userguide#matlab_jobs|go back on level]]
 */

[[calcoloscientifico:userguide:matlab:parallel|go back one level]]