API Reference

Run notebooks via kbatch, store output in S3.

kbatch_papermill.kbatch_papermill(notebook: Path, s3_dest: str, job_name: str = 'papermill', *, s3_code_dir: str = '', code_dir: str | None = None, profile_name: str = 'default', env: dict[str, str] | None = None, parameters: dict[str, Any] | None = None) → str[source]

Run a notebook with Papermill and store the result in S3.

Parameters:

  • notebook (Path) – Path to the notebook.

  • s3_dest (str) – S3 URL where the notebook should be stored (e.g., s3://bucket/path/to/notebook.ipynb).

  • job_name (str, optional) – Name prefix for the kbatch job. Defaults to “papermill”.

  • profile_name (str, optional) – Name of the profile to run with (specifies resource requirements). Defaults to “default”.

  • env (dict[str, str], optional) – Additional environment variables to set (other than “AWS_” ones). Defaults to None.

  • parameters (dict[str, Any], optional) – Papermill parameters to pass. Defaults to None.

Returns:

Name of the kbatch job.

Return type:

str

kbatch_papermill.print_job_status()[source]

Print the status of all kbatch jobs as a nice table.

kbatch_papermill.wait_for_jobs(*job_names, stop_on_failure=True, failure_logs=True)[source]

Wait for one or more jobs by name.

Parameters:

  • job_names (list[str], optional) – Job names. Defaults to all names.

  • stop_on_failure (bool, optional) – Whether to stop waiting on the first failure. Defaults to True.

  • failure_logs (bool, optional) – Whether to print the logs of failed jobs. Defaults to True.