Summer REU in DISC at Notre Dame

REU in Data Intensive Scientific Computing (DISC) at the University of Notre Dame
DISC combines big data, big science, and big computers at the University of Notre Dame. The only thing missing is you!

We invite outstanding undergraduates to apply for a summer research experience in DISC at the University of Notre Dame.  Students will spend ten weeks learning how to use high performance computing and big data technologies to attack scientific problems.  Majors in computer science, physics, biology, and related topics are encouraged to apply.

For more information:
http://disc.crc.nd.edu

CCTools 5.3.0 released

The Cooperative Computing Lab is pleased to announce the release of version 5.3.0 of the Cooperative Computing Tools including Parrot, Chirp, Makeflow, WorkQueue, Umbrella, SAND, All-Pairs, Weaver, and other software.

The software may be downloaded here:
http://www.cse.nd.edu/~ccl/software/download

This minor release adds several features and bug fixes. Among them:

• [Makeflow]  Several enhancements in garbage collection. (Nick Hazekamp)
• [Makeflow]  Better task state handling when recovering execution log. (Nick Hazekamp)
• [Parrot]    Correct handling of multi-threaded programs. (Patrick Donnelly)
• [Parrot]    Adds parrot_mount, to set arbitrary mount points while parrot is executing. (Douglas Thain)
• [Parrot]    Add --fake-setuid option, for executables that request setuid. (Tim Shaffer)
• [Parrot]    Update cvmfs uri to new convention. (Jakob Blomer)
• [Parrot]    Add --whitelist to restrict filesystem access. (Tim Shaffer)
• [Parrot]    Make special file descriptors invisible to tracee. (Patrick Donnelly)
• [Resource Monitor] Compute approximations of shared resident memory. (Ben Tovar)
• [Resource Monitor] Remove resource_monitorv for static binaries. resource_monitor now handles all cases.  (Ben Tovar)
• [Resource Monitor] Working directories are not tracked by default anymore. Use --follow-chdir instead.  (Ben Tovar)
• [Umbrella]  Support for curateND. (Haiyan Meng)
• [Umbrella]  Support for installing software from package managers. (Haiyan Meng)
• [Work Queue] Adds option -C to read a JSON configuration file. (Ben Tovar)
• [Work Queue] Several bug fixes regarding task/workflow statistics. (Ben Tovar)
• [Work Queue] Master's shutdown option now correctly terminates  workers. (Ben Tovar)
• [Work Queue] Adds --sge-paremeter to ./configure script to personalize the sge_submit_workers script. (Ben Tovar)
• [Work Queue] Adds the executable disk_allocator, to restrict disk usage at the workers. (Nate Kremer-Herman)

Incompatibility warnings:

• Workers from this release do not work correctly with masters from previous releases.

Thanks goes to the contributors for many features, bug fixes, and tests:

• Jakob Blomer
• Neil Butcher
• Patrick Donnelly
• Nathaniel Kremer-Herman
• Nicholas Hazekamp
• Peter Ivie
• Kevin Lannon
• Haiyan Meng
• Tim Shaffer
• Douglas Thain
• Ben Tovar
• Mathias Wolf
• Anna Woodard

Please send any feedback to the CCTools discussion mailing list:

http://ccl.cse.nd.edu/software/help/

Enjoy!

Analyzing LHC Data on 10K Cores with Lobster

Prof. Thain gave a talk titled Analyzing LHC Data on 10K Cores with Lobster at the Workshop on Data Intensive Computing in the Clouds at Supercomputing 2015.  The talk gave an overview of our collaboration with members of the CMS experiment at Notre Dame.  Together, we have built a data analysis system which can deploy the complex CMS computing environment on large clusters of non-dedicated machines.

Global Filesystems Paper in IEEE CiSE

Our latest paper, in collaboration with Jakob Blomer and the CVMFS team at CERN, describes the evolution of global-scale filesystems to serve the needs of the world-wide LHC experiment collaborations:

• Jakob Blomer, Predrag Buncic, Rene Meusel, Gerardo Ganis, Igor Sfiligoi and Douglas Thain, The Evolution of Global Scale Filesystems for Scientific Software Distribution, IEEE/AIP Computing in Science and Engineering, 17(6), pages 61-71, December, 2015. DOI: 10.1109/MCSE.2015.111 

Delivering complex software across a worldwide distributed system is a major challenge in high-throughput scientific computing. The problem arises at different scales for many scientific communities that use grids, clouds, and distributed clusters to satisfy their computing needs. For high-energy physics (HEP) collaborations dealing with large amounts of data that rely on hundreds of thousands of cores spread around the world for data processing, the challenge is particularly acute. To serve the needs of the HEP community, several iterations were made to create a scalable, user-level filesystem that delivers software worldwide on a daily basis. The implementation was designed in 2006 to serve the needs of one experiment running on thousands of machines. Since that time, this idea evolved into a new production global-scale filesystem serving the needs of multiple science communities on hundreds of thousands of machines around the world. 

Preservation Talk at iPres 2015

Prof. Thain gave a talk titled "Preserving Scientific Software Executions: Preserve the Mess or Encourage Cleanliness" at the 2015 Conference on Digital Preservation.  This talk gives a high level overview of our work on preservation, encompassing packaging with Parrot, environment specification with Umbrella, and workflow preservation with Prune.

CMS Case Study Paper at CHEP

Our case study work on how to preserve and reproduce a high energy physics (HEP) application with Parrot has been accepted by Journal of Physics: Conference Series (JPCS 2015).

The HEP application under investigation is called TauRoast, and authored by our physics collaborator, Matthias. TauRoast is a complex single-machine application having lots of implicit and explicit dependencies: CVS, github, PyYAML websites, personal websites, CVMFS, AFS, HDFS, NFS, and PanFS. The total size of these dependencies is about 166.8TB.

To make TauRoast reproducible, we propose one fine-grained dependency management toolkit based on Parrot to track the really used data and create a reduced package which gets rid of all the unused data. By doing so, the original execution environment with the size of 166.8TB is reduced into a package with the size of 21GB. The correctness of the preserved package is demonstrated in three different environments - the original machine, one virtual machine from the Notre Dame Cloud Platform and one virtual machine from the Amazon EC2 Platform.

Haiyan Meng, Matthias Wolf, Peter Ivie, Anna Woodard, Michael Hildreth and Douglas Thain, A Case Study in Preserving a High Energy Physics Application with Parrot, Journal of Physics: Conference Series, December, 2015.

 

OpenMalaria Preservation with Umbrella

Haiyan worked together with Alexander from CRC, successfully preserved and reproduced a C++ application, openMalaria, using Umbrella. The data dependencies of openMalaria include packages from yum repositories, OS images from the CCL websites, software and data dependencies from curateND. Through a JSON-format specification, Umbrella allows the user to specify the complete execution environment for his application: hardware, kernel, OS, software, data, packages supported by package managers, environment variables, and command. Each dependency in an Umbrella specification also comes with its metadata information - size, format, checksum, downloading urls, and mountpoints. During runtime, Umbrella tries to construct the execution environment specified in the specification with the help of different sandboxing techniques such as Parrot and Docker, and run the user's task.

DAGVz Paper at Visual Performance Analysis Workshop

An Huynh will be presenting a paper on the visualization of task-parallel programs at the Visual Performance Analysis workshop at Supercomputing 2015.  (He is a student of Kenjiro Taura at the University of Tokyo who spent a semester working with us in the Cooperative Computing Lab.)

An developed DAGViz, a tool for exploring the rich structure of task parallel programs, which requires associating DAG structure with performance measures at multiple levels of detail.  This allows the analyst to zoom in to trouble spots and view exactly where and how each basic block of a program ran on a multi-core machine.

An Huynh, Douglas Thain, Miquel Pericas, and Kenjiro Taura, DAGViz: A DAG Visualization Tool for Analzying Task Parallel Program Traces, Workshop on Visual Performance Analysis (VPA) at ACM Supercomputing (SC), November, 2015.

Virtual Wind Tunnel in IEEE CiSE

Some of our recent work on a system for collaborative engineering design was recently featured in the September issue of IEEE Computing in Science and Engineering focused on "Open Simulation Laboratories"  This project was part of a collaboration between faculty in the computer science and civil engineering departments, Open Sourcing the Design of Civil Infrastructure.

CCL grad student Peter Sempolinski led the design and implementation of an online service enabling collaborative design and evaluation of structures, known as the "Virtual Wind Tunnel".  This service enables structural designs to be uploaded and shared, then evaluated for performance via the OpenFOAM CFD package.  The entire process is similar to that of collaborative code development, where the source (i.e. a building design) is kept in a versioned repository, automated builds (i.e. building simulation) are performed in a consistent and reproducible way, and test results (i.e. simulation metrics) are used to evaluate the initial design.  Designs and results can be shared, annotated, and re-used, making it easy for one engineer to build upon the work of another.

The prototype system has been used in a variety of contexts, most notably to demonstrate the feasibility of crowdsourcing design and evaluation work via Amazon Turk.

• Peter Sempolinski, Daniel Wei, Ahsan Kareem, and Douglas Thain, Adapting Collaborative Software Development Techniques to Structural Engineering, IEEE Computing in Science and Engineering, 17(51), pages 27-34, September, 2015. DOI: 10.1109/MCSE.2015.88

Three Papers at IEEE Cluster in Chicago

This week, at the IEEE Cluster Computing conference in Chicago, Ben Tovar will present some of our work on automated application monitoring:

Gideon Juve, Benjamin Tovar, Rafael Ferreira da Silva, Dariusz Krol, Douglas Thain, Ewa Deelman, William Allcock, and Miron Livny, Practical Resource Monitoring for Robust High Throughput Computing, Workshop on Monitoring and Analysis for High Performance Computing Systems Plus Applications at IEEE Cluster Computing, September, 2015.

Matthias Wolf will present our work on the Lobster large scale data management system:

Anna Woodard, Matthias Wolf, Charles Mueller, Nil Valls, Ben Tovar, Patrick Donnelly, Peter Ivie, Kenyi Hurtado Anampa, Paul Brenner, Douglas Thain, Kevin Lannon and Michael Hildreth, Scaling Data Intensive Physics Applications to 10k Cores on Non-Dedicated Clusters with Lobster, IEEE Conference on Cluster Computing, September, 2015.

Olivia Choudhury will present some work on modelling concurrent applications, trading off thread-level parallelism against task-level parallelism at scale:

Olivia Choudhury, Dinesh Rajan, Nicholas Hazekamp, Sandra Gesing, Douglas Thain, and Scott Emrich, Balancing Thread-level and Task-level Parallelism for Data-Intensive Workloads on Clusters and Clouds, IEEE Conference on Cluster Computing, September, 2015.