Haipeng Cai Defends Ph.D.

Haipeng Cai successfully defended his dissertation, "Cost-effective Dependence Analyses for Reliable Software Evolution", which studied methods for efficiently determining the scope of complex software system that is affected by a given change.

Haipeng will be taking a postdoctoral research position at Virginia Tech under the supervision of Prof. Barbara Ryder.

Congratulations to Dr. Haipeng Cai!

CCTools 5.1.0 released

The Cooperative Computing Lab is pleased to announce the release of version 5.1.0 of the Cooperative Computing Tools including Parrot, Chirp, Makeflow, WorkQueue, SAND, All-Pairs, Weaver, and other software.

The software may be downloaded here:
download

This minor release adds a couple of small features, and fixes the following
issues of version 5.0.0:

• [Prune]     Fix installation issue. (Haiyan Meng)
• [Umbrella]  Fix installation issue. (Haiyan Meng)
• [WorkQueue] Worker's --wall-time to specify maximum period of time a worker may be active. (Andrey Tovchigrechko, Ben Tovar)
• [WorkQueue] work_queue_status's --M to show the status of masters by name. (Names may be regular expressions). (Ben Tovar)
• [WorkQueue] Fix missing priority python binding.
• [WorkQueue] Fix incorrect reset of workers when connecting to different masters. (Ben Tovar)
• [WorkQueue] Fix segmentation fault when cloning tasks. (Ben Tovar)
• [WQ_Maker]  Cleanup, and small fixes. (Nick Hazekamp)

Thanks goes to our contributors:

Nicholas Hazekamp
Haiyan Meng
Ben Tovar
Andrey Tovchigrechko

Please send any feedback to the CCTools discussion mailing list:

mailing list

Enjoy!

~

CCTools 5.0.0 released

The Cooperative Computing Lab is pleased to announce the release of version 5.0.0 of the Cooperative Computing Tools including Parrot, Chirp, Makeflow, WorkQueue, SAND, All-Pairs, Weaver, and other software.
The software may be downloaded here: CCTools download
This is a major release that incorporates the preview of three new tools:

• [Confuga] An active storage cluster file system built on top of Chirp. It is used as a collaborative distributed file system and as a platform for execution of scientific workflows with full data locality for all job dependencies. (Patrick Donnelly)
• [Umbrella] A tool for specifying and materializing comprehensive execution environments. Once a task is specified, Umbrella determines the minimum mechanism necessary to run it such as, direct execution, a system container, a local virtual machine, or submission to a cloud or grid environment. (Haiyan Meng).
• [Prune] A system for executing and precisely preserving scientific workflows. Collaborators can verify research results and easily extend them at a granularity determined by the user. (Peter Ivie)

This release adds several features and several bug fixes. Among them:

• [AllPairs] Support for symmetric matrices. (Haiyan Meng)
• [Chirp] Perl and python bindings. (Ben Tovar)
• [Chirp] Improvements to the job interface. (Patrick Donnelly)
• [Makeflow] Improved Graphviz's dot output. (Nate Kremer-Herman)
• [Makeflow] Support for command wrappers. (Douglas Thain)
• [Parrot] Several bug fixes for CVMFS-based applications. (Jakob Blomer, Patrick Donnelly)
• [Parrot] Valgrind support. (Patrick Donnelly)
• [Resource Monitor] Library for polling resources. (Ben Tovar)
• [WorkQueue] Signal handling bug fixes. (Andrey Tovchigrechko)
• [WorkQueue] Log visualizer. (Ryan Boccabella)
• [WorkQueue] work_queue_worker support for Docker. (Charles Zheng)
• [WorkQueue] Improvements to perl bindings. (Ben Tovar)
• [WorkQueue] Support to blacklist workers. (Nick Hazekamp)

Incompatibility warnings: Workers from 5.0 do not work with masters pre 5.0.
Thanks goes to the contributors for many features and bug fixes: Matthew Astley, Jakob Blomer, Ryan Boccabella, Peter Bui, Patrick Donnelly, Nathaniel Kremer-Herman, Victor Hawley, Nicholas Hazekamp, Peter Ivie, Kangkang Li, Haiyan Meng, Douglas Thain, Ben Tovar, Andrey Tovchigrechko, and Charles Zheng.
Please send any feedback to the CCTools discussion mailing list: mailing list
Enjoy!

Preservation Framework for Computational Reproducibility at ICCS 2015

Haiyan Meng presented our work on Preservation Framework for Computational Reproducibility at the International Conference on Computational Science (ICCS) in Reykjavik, Iceland. This is a collaborative work between University of Notre Dame and University of Chicago for the DASPOS project both of these two universities are working on.

The preservation framework proposed in this paper includes three parts: 

• First, how to use light-weight application-level virtualization techniques to create a reduced package which only includes all the necessary dependencies; 
• Second, how to organize the data storage archive to preserve these packages; 
• Third, how to distribute applications through standard software delivery mechanisms like Docker and deploy applications through flexible deployment mechanisms such as Parrot, PTU, Docker, and chroot.

Haiyan Meng, Rupa Kommineni, Quan Pham, Robert Gardner, Tanu Malik and Douglas Thain, An Invariant Framework for Conducting Reproducible Computational Science, Journal of Computational Science 9 (2015): 137-142. DOI: 10.1016/j.jocs.2015.04.012