Last August 15, 2022 we gave a demonstration on how to use Coffea using the Work Queue executor. Coffea is a framework for pulling together all the typical needs of a high-energy collider physics (HEP) experiment analysis using the scientific python ecosystem. Using Coffea on top of Work Queue, we can automatically manage the resources (such as cores, memory, and disk) to maximize the number of concurrent tasks that the application can run. Further, since Work Queue tasks can be created dynamically once the application is running, we can shape the size of tasks (in this case, the number of collision of particle events) so that they better fit the resources available.
For this demo, we also showcased a new status display for Work Queue when used inside a jupyter notebook:
No comments:
Post a Comment