Tuesday, July 10, 2018

Halfway Through 2018 Summer REU


We are a little more than halfway through the 2018 edition of our summer Data Intensive Scientific Computing REU program at the University of Notre Dame.  This summer, our students are working on projects in network science, genome analysis, workflow systems, data visualization, and more.  At this point in the summer, students are finalizing their results and starting to work on posters and videos to present what they have learned for our summer research symposium.

Thursday, July 5, 2018

Announcement: CCTools 7.0.0 released

The Cooperative Computing Lab is pleased to announce the release of version 7.0.0 of the Cooperative Computing Tools including Parrot, Chirp, JX, Makeflow, WorkQueue, Umbrella, Prune, SAND, All-Pairs, Weaver, and other software.

The software may be downloaded here:
http://ccl.cse.nd.edu/software/download

This is a major release which adds several features and bug fixes. Among them:

  • [General] Catalog updates compressed, and via TCP. (Douglas Thain, Nick Hazekamp, Ben Tovar)
  • [JX] Bug fixes to JX, a superset of JSON to dynamically describe workflows, see doc/jx-tutorial.html. (Tim Shaffer, Douglas Thain)
  • [Makeflow] Formally define and implement hooks to workflow rules. Hooks may be used to wrap rules with containers (e.g. singularity), a monitoring tool, etc.  (Nick Hazekamp, Tim Shaffer)
  • [Makeflow] Rule execution as Amazon Lambda functions and S3 objects. (Kyle Sweeney, Douglas Thain)
  • [Makeflow] Efficient shared file system access. (Nick Hazekamp)
  • [Makeflow] Several bug fixes for rules executing in Mesos. (Chao Zheng)
  • [ResourceMonitor] Several bug fixes. (Ben Tovar)
  • [WorkQueue] Add user-defined features to workers and tasks. (Nate Kremer-Herman)
  • [WorkQueue] Fixes for python3 support. (Ben Tovar)

Thanks goes to the contributors for many features, bug fixes, and tests:

  • Nathaniel Kremer-Herman
  • Nicholas Hazekamp
  • Tim Shaffer
  • Douglas Thain
  • Ben Tovar
  • Kyle Sweeney
  • Chao Zheng

Please send any feedback to the CCTools discussion mailing list:

http://ccl.cse.nd.edu/community/forum

Enjoy!