Congrads to Ph.D Graduates

Congratulations to all of our 2017 Ph.D. graduates in Computer Science and Engineering,
and especially to Dr. Haiyan Meng who is moving on to a position at Google, Inc.

Announcement: CCTools 6.1.0. released

The Cooperative Computing Lab is pleased to announce the release of version 6.1.0 of the Cooperative Computing Tools including Parrot, Chirp, Makeflow, WorkQueue, Umbrella, Prune, SAND, All-Pairs, Weaver, and other software.

The software may be downloaded here:
http://ccl.cse.nd.edu/software/download

This is a major which adds several features and bug fixes. Among them:

• [General]  IPv4 and IPv6 mode handling fixes. (Tim Shaffer)
• [Grow]  Fuse module for GROW-FS. (Tim Shaffer)
• [Makeflow]  Updated manual. (Douglas Thain)
• [Makeflow]  Support for Mesos. (Charles Zheng)
• [Makeflow]  Support for Singularity. (Kyle Sweeney)
• [Makeflow]  --shared-fs option. (Nick Hazekamp)
• [Makeflow]  --preserve option for per rule caching. (Pierce Cunneen)
• [Parrot]  Fix ld.so and exec bugs. (Tim Shaffer)
• [Parrot]  Fix handling of namespace symlinks. (Tim Shaffer)
• [Parrot]  Bind/connect on AF_UNIX sockets. (Tim Shaffer)
• [Parrot]  Several bug fixes. (Tim Shaffer)
• [Parrot]  parrot_namespace to use parrot inside parrot. (Tim Shaffer)
• [Parrot]  Add fixed and warped PIDs. (Douglas Thain)
• [Prune]     Several bug fixes. (Peter Ivie)
• [ResourceMonitor]  Resource per task snapshots, --snapshot-file. (Ben Tovar)
• [Umbrella]  Several bug fixes. (Haiyan Meng)
• [WorkQueue]  Resource per task snapshots, q.enable_monitoring_snapshots. (Ben Tovar)
• [WorkQueue]  Several bug fixes. (Ben Tovar)
• [WorkQueue]  Custom environments for wq factory. (Kyle Sweeney)
• [WorkQueue]  Several fixes to wq factory. (Nate Kremer-Herman)
• [WQ_Maker]   Several bug fixes, update to latest version of maker. (Nick Hazekamp)

Thanks goes to the contributors for many features, bug fixes, and tests:

• Jakob Blomer
• Pierce Cunneen
• Patrick Donnelly
• Nathaniel Kremer-Herman
• Nicholas Hazekamp
• Peter Ivie
• Haiyan Meng
• Tim Shaffer
• Douglas Thain
• Ben Tovar
• Kyle Sweeney
• Chao Zheng

Please send any feedback to the CCTools discussion mailing list:

http://ccl.cse.nd.edu/community/forum

Enjoy!

Makeflow and Mesos Paper at CCGrid 2017

Charles Zheng will present the paper Deploying High Throughput Scientific Workflows on Container Schedulers with Makeflow and Mesos at the 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2017) in May 15, 2017 at Madrid, Spain. In this paper we consider how to launch workflow system on container schedulers with minimal performance loss and higher system efficiency. As examples of current technology, we use Makeflow,  Work Queue, Resource Monitor and Mesos. We observe that using Work Queue and Resource Monitor not only reduces the task turnaround time but also achieves higher resource usage rate. Following is the system architecture.

Workflow Reproducibility Paper at ICCS 2017

Haiyan Meng will present a paper titled Facilitating the Reproducibility of Scientific Workflows with Execution Environment Specifications at the International Conference on Computational Science (ICCS) 2017 this June at Zurich, Switzerland. This paper explores the challenges in reproducing scientific workflows, and proposes a framework for facilitating the reproducibility of scientific workflows at the task level by giving scientists complete control over the execution environments of the tasks in their workflows and integrating execution environment specifications into scientific workflow systems.