Creating Better Force Fields on Distributed GPUs with Work Queue

ForceBalance is an open source software tool for creating accurate force fields for molecular mechanics simulation using flexible combinations of reference data from experimental measurements and theoretical calculations. These force fields are used to simulate the dynamics and physical properties of molecules in chemistry and biochemistry.

The Work Queue framework gives ForceBalance the ability to distribute computationally intensive components of a force field optimization calculation in a highly flexible way. For example, each optimization cycle launched by ForceBalance may require running 50 molecular dynamics simulations, each of which may take 10-20 hours on a high end NVIDIA GPU. While GPU computing resources are available, it is rare to find 50 available GPU nodes on any single supercomputer or HPC cluster. With Work Queue, it is possible to distribute the simulations across several HPC clusters, including the Certainty HPC cluster at Stanford, the Keeneland GPU cluster managed by Georgia Tech and Oak Ridge National Laboratories, and the Stampede supercomputer managed by the University of Texas. This makes it possible to run many simulations in parallel and complete the high level optimization in weeks instead of years.

 - Lee-Ping Wang, Stanford University

CCTools 4.3 released

The Cooperative Computing Lab is pleased to announce the release of version 4.3.0 of the Cooperative Computing Tools, including Parrot, Chirp, Makeflow, WorkQueue, Weaver, DeltaDB, SAND, All-Pairs, and other software. This release has some important changes:

• Peter Bui's Weaver is included. Weaver is a high level interface to Makeflow which allows to describe workflows using python. For more information see cctools/doc/man/weaver.1 and cctools/weaver/examples in the distribution.
• This is also the first release to include DeltaDB, written by Peter Ivie and Douglas Thain. DeltaDB implements a model for time-varying schema-free data and underlies the query engine for the CCTools catalog server.
• Backwards compatibility of master and workers pre-4.3 is broken. Workers from 4.3 cannot connect to masters pre-4.3, and masters from 4.3 will not accept connection from workers pre-4.3. The API did not change, thus unless you want to take advantage of new features, you should not need to modify your code.
• The interface to work_queue_pool has been simplified, and all options have to be specified at the command line. Please see cctools/doc/man/work_queue_pool.1 for more information.
• Undefined environment variables used in Makeflow are no longer allowed by the parser.
• Binaries for 32bit architectures are not being distributed as part of this release. Please let us know if you need them.

Other highlights

• [WorkQueue] Perl object oriented bindings have been added. See perldoc Work_Queue::Queue [B. Tovar]
• [WorkQueue] A priority per task can now be specified. [D. Thain, B. Tovar]
• [WorkQueue] --single-shot option added to workers to exit quickly after the master disconnects [D. Thain].
• [WorkQueue] Hierarchy statistics when using foremen are now available. [B. Tovar, M. Wolf]
• [WorkQueue] work_queue_pool code cleanup. [D. Thain, B. Tovar]
• [Makeflow] New lexer and parser with cleaner semantics and error reporting. [B. Tovar]
• [Parrot] Bug fix that allows parrot's temp-dir to be on GPFS. [P. Donnelly]
• [Parrot] Several fixes to better support executables with threads. [P. Donnelly]
• [Parrot] Update to use the newer ptrace API. [P. Donnelly]
• [Parrot] Several updates to parrot_package_run. See cctools/doc/man/parrot_package_run. [H. Meng]
• [Parrot] iRODS 4.x support. [D. Thain]

You can download the software here: cctools download

Thanks goes to the contributors and testers for this release: Peter Bui, Patrick Donnelly, Nick Hazekamp, Peter Ivie, Kangkang Li, Haiyan Meng, Peter Sempolinski, Douglas Thain, Ben Tovar, Lee-Ping Wang, Matthias Wolf, Anna Woodard, and Charles Zheng

Enjoy!