Peta-Scale I/O With the Lustre File System

Overview

This paper describes low-level infrastructure in the Lustre file system that addresses scalability in very large clusters. The features described deal with I/O and networking, lock management, recovery after failure, and other scalability-related issues. The Lustre file system first went into production in Spring 2003 on the Multiprogrammatic Capability Resource (MCR) cluster at Lawrence Livermore National Laboratory (LLNL). The MCR cluster was one of the largest clusters at that time with 1100 Linux compute nodes as Lustre clients. Since then, the Lustre file system has been deployed on larger systems, notably the Sandia Red Storm deployment, with approximately 25,000 liblustre clients, and the Oak Ridge National Laboratory (ORNL) Jaguar system, which is of similar scale and runs Lustre technology both with Client-Node Linux (CNL) and with Catamount.