Skip to content

Active Learning (MPI + Dragon DDict)

Location: example/active-learn/cpp-mpi-with-ddict

This example combines an MPI-parallel C++ simulation with a Python-driven sequential active-learning loop, using a Dragon DDict as the shared data-exchange layer between them. The goal of this workflow is to create a trained surrogate model (a Gaussian-Process model specifically) based on data generated by a toy simulation. The active-learner queries the surrogate model trainer to understand which points to sample next.

Architecture

  • A shared Dragon DDict is visible to every Dragon-managed process.
  • The MPI simulation writes per-rank samples under keys like sim_rank_{r}_iter_{i}, and a rank-0 sentinel key sim_meta_iter_{i} that every task uses to self-detect the current iteration.
  • Python-side tasks (training, active learning) read those samples and write back model_iter_{i}, mse_iter_{i}, and query_points_iter_{i} (as Dragon Serializables) for the next iteration.

Key Design Points

  • DragonExecutionBackend transforms function task to ComputeTasks handled by RHAPSODY which then uses DragonHPC to execute the task
  • The MPI simulation reads the DDict descriptor from the ROSE_DDICT_DESCRIPTOR environment variable and uses the cross C++/Python Serializable format to exchange data with the Python-based tasks.
  • The convention used throughout the example: pass ddict_descriptor into every task via acl.start(initial_config=...) kwargs, and have each task open its own DDict.attach(ddict_descriptor) connection, deriving the current iteration by counting sim_meta_iter_* sentinel keys.

Running

cd example/active-learn/cpp-mpi-with-ddict
dragon driver.py

See driver.py and simulation.cpp for the full implementation, and requirements.txt in the same directory for the Python dependencies (surrogate model, active-learning loop, etc.).