Slurm User Guide

Essential Linux Commands for Slurm Users

Before diving into Slurm-specific commands, it's crucial to know some basic Linux commands. Here are some essential ones you should be familiar with when using a Slurm cluster:

1. File and Directory Operations

  • ls: List directory contents
  • ls -l: Long format listing
  • ls -a: Show hidden files
  • pwd: Print working directory
  • cd: Change directory
  • cd ..: Move up one directory
  • cd ~: Go to home directory
  • mkdir: Create a new directory
  • rm: Remove files or directories
  • rm -r: Remove directories and their contents recursively
  • cp: Copy files or directories
  • mv: Move or rename files or directories
  • touch: Create an empty file or update file timestamps

    • 2. File Viewing and Editing

  • cat: Display file contents
  • less: View file contents page by page
  • head: Display the beginning of a file
  • tail: Display the end of a file
  • tail -f: Follow file changes in real-time
  • nano or vim: Text editors

    • 3. File Permissions and Ownership

  • chmod: Change file permissions
  • chown: Change file ownership

    • 4. Process Management

  • ps: Display current processes
  • ps aux: Show all processes for all users
  • top or htop: Interactive process viewers
  • kill: Terminate processes

    • 5. System Information

  • df: Report file system disk space usage
  • du: Estimate file space usage
  • free: Display amount of free and used memory

    • 6. Text Processing

  • grep: Search text using patterns
  • sed: Stream editor for filtering and transforming text
  • awk: Pattern scanning and processing language

    • 7. Network Commands

  • ssh: Secure shell for remote login
  • scp: Securely copy files between hosts on a network
  • wget or curl: Retrieve files from the web

    • 8. File Compression and Archiving

  • tar: Tape archiver, used for creating and extracting archives
  • gzip, gunzip: Compress or expand files
  • zip, unzip: Package and compress (archive) files

    • 9. File Transfer

  • rsync: Fast, versatile file copying tool

    • 10. Miscellaneous

  • man: Display the manual page for a command
  • history: Show command history
  • which: Locate a command

    • Using These Commands in Slurm Context

    1. File Management: You'll use ls, cd, mkdir, cp, and mv to navigate and manage your files and directories on the cluster.
    2. Job Script Creation: Use text editors like nano or vim to create and edit your Slurm job scripts.
    3. File Permissions: Use chmod to ensure your job scripts are executable.
    4. Process Monitoring: Commands like ps and top can be useful for monitoring your jobs on the compute nodes (if you have access).
    5. Output Examination: Use cat, less, head, and tail to view your job output files.
    6. Data Transfer: Use scp or rsync to transfer files to and from the cluster.
    7. Text Processing: Commands like grep, sed, and awk are invaluable for parsing and analyzing job outputs.