Control your analyses, anywhere
Follow every pipeline run on your HPC cluster or in the cloud, learn what each step really needs, and plan the heavy ones for when the energy is cleaner.
Open the appIllustrative data, shown with the app's own planner. Line: grid carbon intensity over 36 hours. Areas: power drawn by each run. Pick a run and move the slider to see how its CO₂ changes with the start time.
Bio-Pype
- Open-source pipeline framework
- Runs on your machines; compute.bio only receives the run's progress
- Reproducible, portable analyses
$ pip install bio-pypeEmission-aware computation
- Time, CPU and memory of every step
- Energy estimate and CO₂ footprint per run
- Carbon intensity of the grid at run time
HPC backends
- Submit and follow jobs on the scheduler you already have
SLURM
PBS
LSF
Runtime models (alpha)
- Learn what each step needs from finished runs
- Suggest better memory and walltime requests
- Rate a finished run against what was expected
- Opt-in: your runs are analysed only if you agree
Requested
64 GB
Used
22 GB
Suggested
Illustrative data28 GB
Low-carbon scheduling (planned)
- Predict a run's duration and energy before it starts
- Suggest the cleanest time window to run it
Cloud
- Scaleway — EU sovereign cloud
- Not just HPC
- More providers to come
Grid carbon intensity
- Current and forecast, from free public sources
- Europe: ENTSO-E
- Great Britain: carbonintensity.org.uk
- Australia: OpenElectricity and AEMO
Illustrative data
Where this is going
Bioinformatics workloads are energy-hungry, but they are also flexible: most analyses are not urgent, and the carbon intensity of the grid changes by the hour. compute.bio connects the two: it records what every step of a run used, learns from finished runs what a workflow needs, and tracks the carbon intensity of the grid the run is drawing from.
Today that means monitoring and per-run energy and CO₂ accounting, plus early models that suggest better memory and walltime requests and tell you how a finished run did against what was expected. The models are alpha and get better as more runs are analysed. Next: predicting a run before it starts, and suggesting a low-carbon window to run it in.