ROSE as a Service¶

This page describes ROSE's service model: how the active-learning/RL loop you define stays under your control while the actual simulation and training work executes on HPC, and what that does and does not require from you today.

Core idea: BYOF — Bring Your Own Workflow¶

ROSE does not own the science. It owns the orchestration loop: submit a task, wait for it, decide what runs next, check a stop criterion, repeat — until max_iter or the criterion is met.

You bring:

• A simulate / train / active_learn (or environment / update, or prediction / uncertainty) implementation, as a plain Python function or a shell command.
• The environment that implementation needs (packages, data, scripts).
• The decision of where it should run — a laptop, a login node, or a leadership-class HPC allocation.

ROSE brings the orchestration: dependency-correct task submission, iteration bookkeeping, stop-criterion evaluation, checkpoint-safe state, and tracking integration. The YAML Spec API is the declarative form of this contract — function: tasks:simulate is a pointer to your code, not a hook into ROSE's.

This is the same boundary that makes ROSE usable across domains without ROSE having to understand any of them: it sees *args, **kwargs in, a return value out, nothing about what happened in between.

Two places work can happen¶

A ROSE learner doesn't run anything itself — it submits tasks through whichever asyncflow execution backend you hand it. Two backends matter for the "as a Service" framing:

The first model is "submit a job, then run my loop inside it." The second is "run my loop here; dispatch tasks to a job running somewhere else." That second model is what makes ROSE service-like: the loop's control plane (your learner.start() call, deciding when to stop, talking to MLflow/ClearML) is decoupled from the compute plane (the HPC allocation actually executing simulate/train).

How ROSE operates within a job¶

The orbit backend connects two things over the ORBIT broker:

1. An orbit endpoint, running inside an HPC job allocation. Historically this job had to be requested through whatever your site normally uses to get an allocation, outside of ROSE. rose run <yaml> --remote now automates this step too: given a remote.target block in the spec (see the YAML Spec API), it submits the bootstrap job itself — via IRI, NERSC's Superfacility API (SFAPI), or PsiJ on an already-connected login node — and waits for the endpoint to register before handing off to step 2. Once the job starts, the endpoint comes up inside it and stays alive for the allocation's lifetime, ready to accept task descriptions.
2. Your orchestration process, running anywhere — your laptop, a long-lived service host, a CI runner. It builds the learner from your spec (LearnerBuilder(cfg, asyncflow).build()), starts the loop (learner.start(...)), and for every simulation/training/active_learn/... task it submits, the asyncflow engine forwards that task description over the broker to the endpoint, waits for the result, and resumes the loop.

The practical effect: you submit a ROSE workflow to HPC without an interactive session on the cluster, and without your laptop needing to stay connected to the scheduler — only to the broker. The job allocation is what's expensive and scheduler-queued; your control process is cheap and can be restarted independently of it. Stop-criterion checks, set_next_config() decisions, and tracking calls all happen on your side of the broker, on every iteration, with no per-iteration job resubmission.

No standalone broker required either. remote.embedded: true (see the YAML Spec API) hosts the broker inside your own orchestration process — nothing to deploy or keep running separately. Combined with target.kind: psij, running rose run --remote directly on a login node needs no bootstrap connection at all: PsiJ submits the compute-node job from the same process that's coordinating the loop. This is the fullest "as a Service" story so far — one command, one process, no infrastructure to stand up first (you still need an operator-placed broker cert/token under ~/.radical/orbit, same as any broker).

This is additive to the model in Target Resources, not a replacement — both go through the same WorkflowEngine/LearnerBuilder plumbing. Which backend you choose only changes where the edge lives; the spec, the learner, and the loop semantics are identical either way.

What ROSE automates for you today¶

• The loop itself — iterate, await, check criterion, repeat, with no boilerplate beyond defining your tasks and the threshold.
• Preemption-safe state — every completed iteration is durable before the next starts, so a killed job loses at most the in-flight iteration.
• Tracking — tracking.backend: mlflow | clearml in the spec wires a tracker once; every iteration's metrics, params, and lifecycle events are reported automatically, with no tracking code inside your task functions.
• Heterogeneous dispatch — the same loop drives CPU-only, GPU, MPI, or shell-executable tasks, and (via the learners: block) distinct task implementations per parallel learner.

What ROSE does not yet automate: data movement¶

Today, getting a simulation's output into the training task's hands is entirely your task code's responsibility:

• type: python slots pass the previous task's return value in-memory, as the first positional argument — this is implicit in the task type, not something you declare.
• type: shell slots pass nothing automatically; your command's stdout becomes the next task's input only if your scripts agree on a file path or convention outside the spec (this is exactly the pattern in the M3DC1 use case, where simulate/train/active_learn hand-roll a namespaced directory of files to communicate).

Planned: a DataExchangeProtocol field in the YAML spec — FileBased or MemoryBased — that makes this an explicit, ROSE-managed choice instead of an implicit consequence of task type:

• MemoryBased formalizes what type: python already does today — in-process object passing between tasks on the same worker.
• FileBased would let ROSE own staging a per-iteration (and per-learner, for parallel runs) working directory, instead of every use case reimplementing its own namespace/path convention by hand inside task code.

This isn't implemented yet — it's the next piece of surface area on the spec, and the natural place to close the "no environment/output contract" gap without ROSE needing to know anything about the science moving through it.

What you bring, concretely¶

None of this is unique to the service model — it's the same BYOF boundary as running ROSE locally. What the service model changes is where that boundary sits physically: your task code and its dependencies need to be reachable from the edge's job allocation, not from your laptop.