IT Summit

High-Throughput Computing on a PC Cluster in the Physics Department

For the last 20 years, Prof. Stan Zygmunt has been involved in computational catalysis research with VU students and colleagues at Argonne National Laboratory.  In this interdisciplinary research, Zygmunt generates computational results that can help interpret the experiments carried out by chemists, materials scientists, physicists, and chemical engineers. Zygmunt’s student research assistants at VU reflect a similar diversity: they have majored in physics, chemistry, and engineering.  Zygmunt and his students use computers to model chemical reactions that occur on the surfaces of catalysts. While the computational approach definitely has its limits, there is a great deal of information that can be calculated but simply cannot be determined experimentally.  In this way the computational and experimental approaches are complementary tools, and together they provide greater insight than either one could on its own.

In order to map out a catalytic reaction involving molecules and solid surfaces, the structures of reactant molecules and catalytic surface sites must be determined, along with the energies of each of the intermediate structures along the reaction pathway.  These tasks require high-level quantum mechanics to be implemented through computationally efficient algorithms.  For the last five years, Zygmunt and his students have been studying the oxidative dehydrogenation of propane by vanadium oxide catalysts.  This reaction converts inexpensive propane to propylene, a valuable molecule used to create a wide variety of consumer plastics.  However, before it can be used industrially, further research is needed to better understand this process.  And using a computer to model this kind of reaction is very time consuming.  On a single processor, a research calculation can take up to several weeks to run!  Zygmunt uses programs that take advantage of parallel processing to break a very complex calculation into smaller pieces that are independent of each other and can be performed by different computer processors at the same time.  The parallel strategy can greatly reduce the “real-time” required to finish a calculation, compared to a serial calculation in which each piece of the calculation is carried out sequentially.  Zygmunt has been doing his research using parallel computing on a 144-node PC cluster at Argonne since 2002. 

In 2006 Zygmunt received a small grant from the Indiana Space Grant Consortium (INSGC) to implement this “high-throughput computing” strategy on a smaller scale at VU.  The grant provided software and support for a student research assistant to run benchmark calculations on the propane-vanadium oxide system.  “Our introductory physics and astronomy lab has 12 Pentium-IV PCs that are used extensively for 3-4 days per week during the academic year, but are hardly used at all during the summer or on weekends.  Paul Nord (Physics and Astronomy Dept. Technical Specialist and IT staff member) and I thought we could use these machines to set up a computing cluster that would make creative use of existing PCs for productive research.” 

In spring 2006, Nord set up a dual-boot configuration for these machines so they could run both Windows (for academic laboratory use) and Scientific Linux (for research use).  He also installed the computational and visualization software and upgraded the network connections in the lab to minimize bottleneck problems when the “master node” communicates with the “slave nodes” on a computing job.  Zygmunt hired Jared Friedhoff ’07, a dual physics and chemistry major, to work on this project for 11 weeks in summer 2006.  Zygmunt says “This cluster represented a significant leap forward in student summer research productivity.  In previous years students ran calculations on a single PC and had to wait much longer for them to finish.  This restricted the scope of their work during a single summer to small-scale projects.  With the cluster, Jared found that at any given time he could make optimal use of the machines by running three different jobs on four nodes each.  For a four-node calculation, he found that the real-time speedup factor was about 3.1 (rather than the ideal value of 4) due to the overhead involved in communication between nodes, while for an 8-node calculation the speedup factor was, surprisingly, only 4.4.  Thus, it was clearly better to have multiple jobs running on fewer nodes.”  After the summer ended, Friedhoff continued to use the cluster for subsequent calculations over weekends and vacations.  As a result of the work done by Friedhoff at VU and Zygmunt at Argonne, the two attended the 2007 American Chemical Society National Meeting in Chicago, where Zygmunt gave a talk and Friedhoff presented a poster describing different aspects of the project.  Friedhoff is currently in a PhD program in chemistry at Vanderbilt University.

Another VU physics major, Dan Brandt ’10, continued work on this project in summer 2008, and Zygmunt has hired mechanical engineering and physics major Megan Kania ’11 for summer 2009.  He is also hoping to upgrade and expand the cluster through the purchase of faster PCs and the addition of a larger number of older machines that are being retired elsewhere at VU.  In the meantime, this project has attracted the attention of other VU physics and astronomy faculty who would like their student researchers to have access to this kind of computing cluster in the summers.  In fact, Tim Olson ’11 is currently setting up a smaller Linux cluster under the supervision of Prof. Jason Webb.  Their goal is to model the behavior of a particle detector that is being used for a high-energy nuclear physics experiment at Brookhaven National Laboratory. “We have nice computing facilities in place for our introductory and advanced laboratories, and they are quite busy for nine months of the year”, said Zygmunt.  “This just seemed like a natural way to make existing resources more productive and make summer research experiences more fruitful and satisfying for our students.”