Interacting with Schedulers (SchedulerIO)#

The SchedulerIO classes provide the bridge between qtoolkit’s high-level objects and the low-level CLI commands of specific queue managers.

For third-party software that wants to handle command execution independently, SchedulerIO offers a decoupled way to generate commands and parse their output.

Generating Commands#

Each SchedulerIO subclass (e.g., SlurmIO, PBSIO) implements methods to generate strings for specific actions.

Note that the third-party client is responsible for executing the commands, verifying the exit codes, and parsing the output.

Job Submission Script#

The get_submission_script method generates the full content of a shell script, including shebang, scheduler headers (directives), and the actual commands.

from qtoolkit.io.slurm import SlurmIO
from qtoolkit.core.data_objects import QResources

slurm_io = SlurmIO()
resources = QResources(job_name="test", nodes=1)
# Using QResources (recommended)
script = slurm_io.get_submission_script(commands="echo 'hello'", options=resources)

# Or using a dictionary for scheduler-specific options
options_dict = {"job_name": "test_dict", "nodes": 1, "partition": "standard"}
script = slurm_io.get_submission_script(commands="echo 'hello'", options=options_dict)
print(script)

Command Line Strings#

You can also generate the specific CLI strings for various operations:

# Get submission command (e.g., 'sbatch submit.sh')
submit_cmd = slurm_io.get_submit_cmd("submit.sh")

# Get cancellation command (e.g., 'scancel 12345')
cancel_cmd = slurm_io.get_cancel_cmd("12345")

# Get job info command (e.g., 'scontrol show job -o 12345')
job_cmd = slurm_io.get_job_cmd("12345")

# Get jobs list command (e.g., 'squeue --noheader -o ...')
list_cmd = slurm_io.get_jobs_list_cmd(user="myuser")

Parsing Output#

After executing a command generated by SchedulerIO, you can pass its exit code, stdout, and stderr back to the SchedulerIO instance to parse the results into standardized objects.

# Assume we executed a submission command and got these results
exit_code = 0
stdout = "Submitted batch job 12345"
stderr = ""

result = slurm_io.parse_submit_output(exit_code, stdout, stderr)
print(f"Status: {result.status}, Job ID: {result.job_id}")

# Similarly for job status
stdout = "JobId=12345 JobName=test JobState=RUNNING ..."
job = slurm_io.parse_job_output(exit_code, stdout, stderr)
print(f"State: {job.state}") # Returns QState.RUNNING

This decoupled approach ensures that your application doesn’t need to know the regex patterns or exit code meanings for every different scheduler.

Supported Schedulers#

  • qtoolkit.io.slurm.SlurmIO

  • qtoolkit.io.pbs.PBSIO

  • qtoolkit.io.sge.SGEIO