Tuesday, October 3, 2023

CloudComputing: How can I execute the Matlab code in CCDB

CCDB(https://ccdb.alliancecan.ca/, aka Compute Canada Database)
First, you should debug the code locally to ensure it runs smoothly. Also, make sure that all required custom functions are included in the directory you plan to upload.
Secondly, download Globus for transferring code and data.
Third, establish a connection with cedar using the built-in Command Prompt in Windows.
1
2
ssh -Y username@cedar.computecanada.ca
password
Create an executable file. First, navigate to the directory where the starting code for execution is located using the "cd" command.
Below is the template for the executable file. Prepare it in advance, and after creation, simply copy and paste it in. To save, use Ctrl+O, then Enter, and then Ctrl+X.
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
#!/bin/bash
#SBATCH --time=20-00:00
#SBATCH --nodes=1
#SBATCH --cpus-per-task=12
#SBATCH --account=def-XXXXX
#SBATCH --mem-per-cpu=250G
#SBATCH --output=job_output.log


module load matlab/2023a
srun matlab -nodisplay -singleCompThread -r "MAIN_SPI_RASTER"
or
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
#!/bin/bash
#SBATCH --time=20-00:00
#SBATCH --gres=gpu:4
#SBATCH --cpus-per-task=12
#SBATCH --account=def-XXXX
#SBATCH --mem-per-cpu=16384M
#SBATCH --output=job_output.log


module load matlab/2023a
srun matlab -nodisplay -singleCompThread -r "MAIN_SPI_RASTER"
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
$ module spider matlab
 Versions:
        matlab/2014a
        matlab/2016b
        matlab/2017a
        matlab/2018a
        matlab/2018b
        matlab/2019a
        matlab/2019b
        matlab/2020a
        matlab/2020b
        matlab/2020b.4
        matlab/2020b.6
        matlab/2021a.1
        matlab/2021a.5
        matlab/2021b.3
        matlab/2022a
        matlab/2022b.2
        matlab/2023a.3
Python
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
#!/bin/bash
#SBATCH --time=1-00:00
#SBATCH --gres=gpu:1
#SBATCH --cpus-per-task=1
#SBATCH --account=def-XXXX
#SBATCH --mem-per-cpu=6384M
#SBATCH --output=job_output.log

module load python/3.11.2
python cc.py
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
$ module spider python
  Versions:
        python/2.7.14
        python/2.7.18
        python/3.5.4
        python/3.6.3
        python/3.6.10
        python/3.7.0
        python/3.7.4
        python/3.7.7
        python/3.7.9
        python/3.8.0
        python/3.8.2
        python/3.8.10
        python/3.9.6
        python/3.10.2
        python/3.11.2
        python/3.11.5
After successfully submitting the job, you can check its status at any time using the command "sq". Once the job has completed, the system won't provide any notifications, so it's important to keep an eye on its status.
Use scancel with the job ID to cancel a job:
1
$ scancel <jobid>
1
2
/*Check the running status.*/
$ diskusage_report

Notes

Name Storage Max Duration Globus endpoint
Cedar 1T 28 days computecanada#cedar-globus
Graham 1T 7 days computecanada#graham-globus
Beluga 1T 7 days computecanada#beluga-dtn
Narval 1T 7 days Compute Canada - Narval
When running on CC, never use absolute paths; always use relative paths.
The CC platform has a maximum file limit of 500K, so it's important to use resources efficiently and reduce the number of files during runtime.
1
2
% An absolute path
/project/usernumber/username/Mat/SPEI/input/prec/prec_192404.tif
CC is a UNIX system and cannot directly run .EXE files. To run an .EXE file on CC, you first need to check if Wine is installed on CC. You can do this by typing 'wine' in the Command Prompt. If it returns the following, it means Wine is already installed.
wine
Usage: wine PROGRAM [ARGUMENTS...]   Run the specified program
       wine --help                   Display this help and exit
       wine --version                Output version information and exit

References

No comments:

Post a Comment