TY - GEN
T1 - Improving I/O performance for High Performance Computing with Application Forwarding Layer
AU - Mingming, Hu
AU - Shen, Hong
AU - Benbo, Zha
AU - Hengwei, Liu
AU - Ruiqi, Zhuo
N1 - Publisher Copyright:
© 2020 ACM.
PY - 2020/7/3
Y1 - 2020/7/3
N2 - In the use of High Performance Computing platforms, I/O performance has become an important limiting factor. For regular users, after a supercomputer is put into use, I/O performance optimization based on the physical structure is usually difficult to achieve. In this paper, we show that this can be achieved through modifying the application layer. Considering that a supercomputer has many computing nodes, we present an approach that uses dedicated computing nodes to process the requests from different applications to optimize the I/O performance. The dedicated computing nodes perform aggregation and reorder the operations before sending the requests to the storage system. We call the role played by these computing nodes in this process the logical I/O forwarding layer which corresponds to the I/O forwarding layer of the physical structure. Our experiments show that this approach is feasible and provides room for further improvement.
AB - In the use of High Performance Computing platforms, I/O performance has become an important limiting factor. For regular users, after a supercomputer is put into use, I/O performance optimization based on the physical structure is usually difficult to achieve. In this paper, we show that this can be achieved through modifying the application layer. Considering that a supercomputer has many computing nodes, we present an approach that uses dedicated computing nodes to process the requests from different applications to optimize the I/O performance. The dedicated computing nodes perform aggregation and reorder the operations before sending the requests to the storage system. We call the role played by these computing nodes in this process the logical I/O forwarding layer which corresponds to the I/O forwarding layer of the physical structure. Our experiments show that this approach is feasible and provides room for further improvement.
KW - High Performance Computing
KW - I/O Scheduling
KW - Performance Optimization
KW - Tianhe-2 Supercomputer
UR - http://www.scopus.com/inward/record.url?scp=85090885200&partnerID=8YFLogxK
U2 - 10.1145/3409501.3409511
DO - 10.1145/3409501.3409511
M3 - Conference contribution
AN - SCOPUS:85090885200
T3 - ACM International Conference Proceeding Series
SP - 1
EP - 5
BT - Proceedings of the 2020 4th High Performance Computing and Cluster Technologies Conference, HPCCT 2020 and 3rd International Conference on Big Data and Artificial Intelligence, BDAI 2020
PB - Association for Computing Machinery
T2 - 4th High Performance Computing and Cluster Technologies Conference, HPCCT 2020 and the 3rd International Conference on Big Data and Artificial Intelligence, BDAI 2020
Y2 - 3 July 2020 through 6 July 2020
ER -