Tuesday, February 16, 2016

CCTools 5.4.0 released

The Cooperative Computing Lab is pleased to announce the release of version 5.4.0 of the Cooperative Computing Tools including Parrot, Chirp, Makeflow, WorkQueue, Umbrella, SAND, All-Pairs, Weaver, and other software.

The software may be downloaded here:


This minor release adds several features and bug fixes. Among them:

  • [Catalog]  Catalog server communication is now done using JSON encoded queries and replies. (Douglas Thain)
  • [Makeflow] --skip-file-check added to mitigate overhead on network file systems. (Nick Hazekamp)
  • [Makeflow] Added amazon batch job interface. (Charles Shinaver)
  • [Resource Monitor] Network bandwidth, bytes received, and sent are now recorded. (Ben Tovar)
  • [Work Queue] Tasks may be grouped into categories, for resource control and fast abort. (Ben Tovar)
  • [Work Queue] work_queue_pool was renamed to work_queue_factory. (Douglas Thain)
  • [Work Queue] --condor-requirements to specify arbitrary HTCondor requirements in worker_queue_factory. (Chao Zheng)
  • [Work Queue] --factory-timeout to terminate worker_queue_factory when no master is active. (Neil Butcher)
  • [Work Queue] Compile-time option to specify default local settings in    sge_submit_workers. (Ben Tovar)
  • [Umbrella]   Several bugfixes. (Haiyan Meng, Alexander Vyushkov)
  • [Umbrella]   Added OSF, and S3 communication. (Haiyan Meng)
  • [Umbrella]   Added EC2 execution engine. (Haiyan Meng)
  • [Parrot] Several bug-fixes for memory mappings. (Patrick Donnelly)
  • [Parrot] All compiled services are shown under / (Tim Shaffer)
  • [Parrot] POSIX directory semantics. (Tim Shaffer)
  • [Parrot] Added new syscalls from Linux kernel 4.3. (Patrick Donnelly)

Thanks goes to the contributors for many features, bug fixes, and tests:

  • Jakob Blomer
  • Neil Butcher
  • Patrick Donnelly
  • Nathaniel Kremer-Herman
  • Nicholas Hazekamp
  • Peter Ivie
  • Kevin Lannon
  • Haiyan Meng
  • Tim Shaffer
  • Douglas Thain
  • Ben Tovar
  • Alexander VyushkovRodney Walker
  • Mathias Wolf
  • Anna Woodard
  • Chao Zheng

Please send any feedback to the CCTools discussion mailing list:



Wednesday, February 3, 2016

Preservation Talk at Grid-5000

Prof. Thain gave a talk titled Preservation and Portability in Distributed Scientific Applications at the Grid-5000 Winter School on distributed computing in Grenoble, France.  I gave a broad overview of our recent efforts, including distributing software with Parrot and CVMFS, preserving high energy physics applications with Parrot, specifying environments with Umbrella, and preserving workflows with PRUNE.