Ph.D. Defense: Nick Hazekamp

Nick Hazekamp successfully defended his Ph.D. dissertation titled "Methods Enabling Portability of Scientific Workflows".  He has recently started a job at Atomic Object in Muskegon, Michigan.  Congratulations, Dr. Hazekamp!

Announcement: CCTools version 7.0.21 released

The Cooperative Computing Lab is pleased to announce the release of version 7.0.21 of the Cooperative Computing Tools including Parrot, Chirp, JX, Makeflow, WorkQueue, and other software.

The software may be downloaded here:
http://ccl.cse.nd.edu/software/download

This is a minor release which adds some bug fixes. Among them:

• [Makeflow] Fix segfault when using --submit-safe-mode. (Ben Tovar)
• [WorkQueue] Fix memory leak error when accumulating capacity statistics. (Ben Tovar)
• [WorkQueue] Do not count the factory as an available worker. (Ben Tovar)

Thanks goes to the contributors for many features, bug fixes, and tests:


T.J. Dasso
Nathaniel Kremer-Herman
Nicholas Hazekamp
Tanner Juedeman 
Ryker McIntyre
Tim Shaffer
Zoe Surma
Douglas Thain
Ben Tovar
Yifan Yu


Please send any feedback to the CCTools discussion mailing list:

http://ccl.cse.nd.edu/community/forum

Enjoy!

ACIC Tutorial on Makeflow and Work Queue

We enjoyed giving a tutorial on Building Scalable Applications with Makeflow and Work Queue for the Advanced Cyberinfrastructure taught by Nirav Merchant at the University of Arizona.  The midterm exam is to scale up a bioinformatics website using Makeflow and Work Queue! 

ReadTheDocs Preview

We are migration our documentation to the online ReadTheDocs service,
which offers an improved style and better navigation.  Check out the
preview here: cctools.readthedocs.io

Announcement: CCTools 7.0.17 released

The Cooperative Computing Lab is pleased to announce the release of version 7.0.17 of the Cooperative Computing Tools including Parrot, Chirp, JX, Makeflow, WorkQueue, SAND, All-Pairs, Weaver, and other software.

The software may be downloaded here.

This is a minor release which adds some bug fixes. Among them:

• [General] Fix for linking issue when using a conda environment. (Tim Shaffer)
• [General] Fix for compilation issue with python3 and old versions of swig. (Ben Tovar)
• [Chirp] Fix for the perl test. (Zoe Surma)
• [Makeflow] Fix typos in documentation. (Ryker Campbell)
• [Parrot] Fix a PATH issue with local execution. (Ben Tovar)
• [Parrot] LOCAL keyword in a mountlist acts now like --disable-service. (Tim Shaffer)

Thanks goes to the contributors for many features, bug fixes, and tests:

• Ryker Campbell
• Nathaniel Kremer-Herman
• Nicholas Hazekamp
• Tim Shaffer
• Zoe Surma
• Douglas Thain
• Ben Tovar
• Chao Zheng

Please send any feedback to the CCTools discussion mailing list:

http://ccl.cse.nd.edu/community/forum

Enjoy!

Ph.D. Defense: Chao "Charles" Zheng

Congratulations to Dr. Chao "Charles" Zheng, who defended his Ph.D. thesis on "The Challenges of Scaling Up High Throughput Workflows with Container Technology".  He will shortly be leaving for California to join Alibaba, Inc.  Here is Charles (second from right) after the successful defense:

Summer REU Projects

In summer 2019, REU students TJ Dasso and Eamon Marmion worked at the CCL on large scale distributed applications.  They completed the integration between the Parsl workflow language and the Work Queue execution framework, demonstrating Python applications running on thousands of cores, and also streamlined our software installation process via Pip and Conda.  They presented their work a the summer REU poster symposium in Jordan Hall:

PhD Proposal: Tim Shaffer

Congrads to Tim Shaffer on passing the PhD candidacy proposal stage:
"Proactive Storage Management for High Throughput Scientific Workloads"

Shrinkwrap Containers at CERN

Tim Shaffer attended the 2019 CVMFS Workshop and presented "Shrinkwrap: Creating HPC Containers", work done together with Nick Hazekamp.  Shrinkwrap is a tool that profiles applications using the CVMFS filesystem and generates a minimal container image with only the parts of the global filesystem that were actually used:

Since the LHC is currently shut down between runs, attendees were able to tour  the experiment hall and see the CMS detector up close!

Announcement: CCTools 7.0.11 released

The Cooperative Computing Lab is pleased to announce the release of version 7.0.11 of the Cooperative Computing Tools including Parrot, Chirp, JX, Makeflow, WorkQueue, Umbrella, Prune, SAND, All-Pairs, Weaver, and other software.

The software may be downloaded here:
http://ccl.cse.nd.edu/software/download

This is a minor release which adds several bug fixes. Among them:

• [General] Fix out-of-date man pages.
• [Catalog] Add merge event to catalog database. (Douglas Thain)
• [Catalog] Fix unary operator constant predicate bug. (Tim Shaffer)
• [Makeflow] Specify custom SGE resources. (Nick Hazekamp)
• [Makeflow] Use scheduler universe in condor_submit_makeflow. (Ben Tovar)
• [WorkQueue] Compilation with python3.7 and anacodna. (Ben Tovar)
• [WorkQueue] Specify port ranges when creating queue in python. (Ben Tovar)

Thanks goes to the contributors for many features, bug fixes, and tests:

• Nathaniel Kremer-Herman
• Nicholas Hazekamp
• Tim Shaffer
• Douglas Thain
• Ben Tovar
• Chao Zheng

Please send any feedback to the CCTools discussion mailing list:

http://ccl.cse.nd.edu/community/forum

Enjoy!

Ph.D. Proposal: Nate Kremer-Herman

Congrads to Nate Kremer-Herman who passed his Ph.D. proposal, titled "Troubleshooting Distributed Applications Using a Graph Representation"