Skip to content

Getting Started: HPC Crash Course

Welcome to the Crash Course Café! ☕ Pull up a chair and let's get you productive on QUT Aqua without the usual learning curve headaches.

Course Overview

What This Course Covers

This isn't your typical academic introduction to high-performance computing. This is a practical, no-nonsense guide to getting real work done on QUT's Aqua system. We'll focus on the essential skills you need to submit jobs, avoid common mistakes, and have real work running in your first week.

Who This Is For

  • New to HPC/PBS: You've heard scary stories about command lines and job schedulers
  • New to QUT Aqua: You've used other HPC systems but Aqua has its own quirks
  • Impatient learners: You want to be productive today, not after reading 200 pages of documentation

Course Philosophy

Quick Wins Over Comprehensive Theory

We'll teach you the 20% of knowledge that handles 80% of your daily tasks. Deep theory can wait until you actually need it.

Practical Solutions Over Academic Explanations

Every concept comes with copy-paste ready examples and real-world context from QUT Aqua.

QUT Aqua-Specific Focus

Generic PBS tutorials are everywhere. This course is tailored specifically to QUT's configuration, quirks, and common issues.

What You'll Learn

By the end of this crash course, you'll be able to:

  • Connect to Aqua and run an interactive job without breaking anything
  • Develop and measure on a compute node interactively — before you submit anything unattended
  • Submit your first PBS batch job — and find its output
  • Diagnose common failures before panicking
  • Right-size resource requests — cores, memory, walltime, GPU
  • Scale beyond a single job — arrays for many runs, chained jobs and checkpoints for work longer than one walltime
  • Know when to dig deeper into the specialized guides

What You Need

  • Basic command line familiarity: You can navigate directories and edit files
  • QUT account access: You can SSH into Aqua
  • A specific task in mind: Having real work to do makes learning stick

What You DON'T Need

  • Deep understanding of parallel computing
  • Experience with other job schedulers
  • Perfect knowledge of every PBS directive
  • Python, strictly speaking. The examples are Python because that is what most readers run, but PBS does not care what is inside a job: every mechanic taught here applies to any command

Course Structure

Time Commitment

  • Total time: 2 to 3 hours across 8 focused lessons
  • Lesson format: 15–20 minutes each
  • Hands-on from Lesson 1: First SSH connection and an interactive job within the opening lesson
  • Follow-up learning: Each lesson connects to detailed guides for when you need more

This crash course is your launching pad. When you hit specific challenges, you'll know exactly which detailed guide to consult:

Next Steps After Completion

  1. Practice with your real work — apply what you've learned immediately
  2. Bookmark the troubleshooting guides — for when things inevitably go wrong
  3. Join the QUT HPC community — share your own discoveries and pain points

Prerequisites

Required before starting the lessons:

  • QUT HPC access: Confirmed SSH access to Aqua
  • Essential Linux commands: Comfortable with cd, ls, pwd, mkdir, cp, mv, rm, chmod
  • Linux file system: Understanding absolute vs relative paths, directory structure, home directory (~), root (/), permissions
  • File transfer method: SCP, rsync, or remote editing setup complete

Need help with setup? See the Prerequisites Checklist for verification commands, then Cmd+Opt+Remote for tooling options.

Course Outline

Phase 1, the core loop (Lessons 1 to 4). Connect, set up Python, run work by hand on a compute node, then hand that same loop to a script.

Lesson 1: Welcome to Aqua (15 min)

"From your laptop to a real compute cluster — and back — in 15 minutes."

  • Cluster mental model — file systems, login vs compute nodes
  • First SSH connection and a 5-minute interactive job
  • Where to put your files (home / scratch / work / TMPDIR)

→ Lesson 1

Lesson 2: Tooling Setup (15–20 min)

"One tool, two files, and an environment you can throw away and rebuild in a second."

  • Install uv with one command, no QUT module needed
  • Create a project: pyproject.toml for what you asked for, uv.lock for what you got, then rebuild the environment from them
  • The one filesystem rule for environments on Aqua; conda as a branch for non-PyPI packages

→ Lesson 2

Lesson 3: Working Interactively (15–20 min)

"Interactive is where you develop. Batch is where you run."

  • Size an interactive request — CPU vs GPU interactive queues and their caps
  • Run a small training script on a compute node by hand
  • tmux for surviving disconnects; when a session should become a batch job

→ Lesson 3

Lesson 4: Your First Batch Job (15–20 min)

"Write down what you want, hand it to PBS, and go do something else."

  • A job script: #PBS lines for the request, shell for the work, and what a job's shell starts with
  • Fine-tune a language model to judge film reviews, on a GPU with nobody watching; results in files, logs beside them
  • Find jobs with qstat, take one back with qdel

→ Lesson 4

Phase 2, reliable operation (Lessons 5 and 6). Read a failure before panicking, then size requests from measurements instead of guesses.

Lesson 5: When Jobs Fail (15–20 min)

"Reading PBS tea leaves and error messages"

  • Where a failure leaves its evidence: the end of .e, the summary in .o, qstat -xf
  • Who stopped the job: your program, PBS at a limit, or PBS refusing it, and the next move for each
  • Test small in an interactive session before you queue long

→ Lesson 5

Lesson 6: Right-sizing Requests (15–20 min)

"How much computer do I actually need?"

  • What a request costs, too little and too much, and why a GPU job is sized GPU first
  • Read what a finished job used on QUT's HPC Monitoring Dashboard
  • Turn each reading into a line of the next request, and save the job's record

→ Lesson 6

Phase 3, scaling (Lessons 7 and 8). Many jobs from one script, and work that outlives a single walltime.

Lesson 7: Job Arrays (15–20 min)

"One script, many runs, and one number to tell them apart."

  • An array, a loop or separate jobs: what each costs you, and the one #PBS -J line that makes a job out of every run
  • Run eight MCMC chains as one array, the index choosing each chain's seed and output file
  • Watch the array as one thing with qstat -t, find the runs that failed, and rerun only those

→ Lesson 7

Lesson 8: Long Jobs (15–20 min)

"When one walltime ceiling isn't enough"

  • Two shapes of work that outgrow a walltime: stages chained with -W depend=afterok, or one job PBS restarts with -c
  • Train a hopping robot for twenty minutes on a ten-minute walltime, saving as it goes and resuming where it stopped, scored by a job that waited for it
  • Read a broken link: a failed stage and the job deleted behind it, a restart that starts from scratch

→ Lesson 8

Ready to dive in? Start with the Prerequisites Checklist, then head to Lesson 1 and get from zero to productive in record time! 🚀