Examples

Motoko ships an example workflow in the repository’s examples/ directory. It contains three BlackDynamite studies and one orchestrator:

examples/
  motoko.yaml
  orchestrator.py
  mult/
    bd.yaml
    launch.sh
    doIt.py
  add/
    bd.yaml
    launch.sh
    doIt.py
  norm/
    bd.yaml
    launch.sh
    doIt.py

Workflow configuration

The example motoko.yaml registers three task managers:

task_managers:
  mult:

  add:

  norm:

orchestrator: orchestrator.main

The mult, add, and norm task managers are located in matching subdirectories. Each subdirectory is a valid BlackDynamite study. The orchestrator is loaded from examples/orchestrator.py and Motoko calls its main function.

Studies

The mult study accepts one floating-point job parameter:

study: mult

job:
  x: float

run:

Its doIt.py script multiplies the input by coefficients and stores the result on the run as run.y. The first run produces two values; later runs produce one value.

The add study also accepts one floating-point job parameter:

study: add

job:
  x: float

run:

Its doIt.py script adds 0.56 to the input and pushes the result as a scalar quantity named scalar.

The norm study accepts a list of mult run ids:

study: norm

job:
  mult_ids: list

run:

Its doIt.py script resolves dependencies from the workflow, reads scalar quantities from the add runs, computes L1 and L2 norm-like values, and pushes them as a vector quantity named vector.

Orchestrator flow

The example orchestrator uses event actions:

  1. init creates the initial mult tasks from --inputs.

  2. mult_finished reacts to the first finished mult runs and creates add tasks from their output values.

  3. need_launching_norm waits until the required mult and add runs are in the expected states, then creates a norm task.

  4. norm_run_finished reacts to the finished norm run and creates additional mult tasks from the computed vector values.

  5. finish terminates the workflow once the expected number of runs exists and all states match the completion condition.

The orchestrator also registers an error handler:

workflow.add_error_handler(event="state = FAILED", f=report_error)

If a run fails, the handler prints information about the failed run and its output files.

Running the example

From the example workflow directory:

cd examples
motoko create .
motoko info
motoko launcher
motoko info --verbose

Inspect the orchestrator’s workflow-specific command-line arguments:

motoko orchestrator start --help

Start the example workflow with two input values:

motoko orchestrator start --inputs 2.1 3.1

When the workflow finishes, inspect the final state:

motoko info

Stop all daemons when done:

motoko kill