Skip to content. | Skip to navigation

EUROfusion

Abstract of ITM-IPS Project

Some of the workflows envisaged in the ITM-TF (and by others, including ITER) require significant computer resources for at least some components of the workflow. The IPS framework, developed by a US Fusion Simulation Project in the past years, has been evaluated in 2012 by the HLST, highlighting some interesting features in view of optimisation of the HPC resources. The main one is the Task Pool, a system that allows submitting dynamically new batch jobs to a set of pre-reserved CPUs. For some workflows, this system can be efficiently used to maximise the usage of the HPC resources while avoiding the queuing time for modules as they are being launched during the execution of the workflow.

It is thought that this feature could be implemented in a more generic way, without being tied to the IPS Framework, to make it usable by other workflow managers or even simple job submission scripts. This proposal asks the HLST to study how this Intelligent Batch Submission system could be made generic and implementable outside of the IPS Framework and implement a prototype that would be used from another workflow engine on a few Use Cases.

This proposal is made by key ITM-TF members in view of augmenting the capabilities of the ITM-TF Framework, but the outcome of the work could be used by the whole HPC community, even outside fusion.