Improving I/O performance for High Performance Computing with Application Forwarding Layer

Hu Mingming, Hong Shen, Zha Benbo, Liu Hengwei, Zhuo Ruiqi

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Citation (Scopus)

Abstract

In the use of High Performance Computing platforms, I/O performance has become an important limiting factor. For regular users, after a supercomputer is put into use, I/O performance optimization based on the physical structure is usually difficult to achieve. In this paper, we show that this can be achieved through modifying the application layer. Considering that a supercomputer has many computing nodes, we present an approach that uses dedicated computing nodes to process the requests from different applications to optimize the I/O performance. The dedicated computing nodes perform aggregation and reorder the operations before sending the requests to the storage system. We call the role played by these computing nodes in this process the logical I/O forwarding layer which corresponds to the I/O forwarding layer of the physical structure. Our experiments show that this approach is feasible and provides room for further improvement.

Original languageEnglish
Title of host publicationProceedings of the 2020 4th High Performance Computing and Cluster Technologies Conference, HPCCT 2020 and 3rd International Conference on Big Data and Artificial Intelligence, BDAI 2020
PublisherAssociation for Computing Machinery
Pages1-5
Number of pages5
ISBN (Electronic)9781450375603
DOIs
Publication statusPublished - 3 Jul 2020
Externally publishedYes
Event4th High Performance Computing and Cluster Technologies Conference, HPCCT 2020 and the 3rd International Conference on Big Data and Artificial Intelligence, BDAI 2020 - Qingdao, Online, China
Duration: 3 Jul 20206 Jul 2020

Publication series

NameACM International Conference Proceeding Series

Conference

Conference4th High Performance Computing and Cluster Technologies Conference, HPCCT 2020 and the 3rd International Conference on Big Data and Artificial Intelligence, BDAI 2020
Country/TerritoryChina
CityQingdao, Online
Period3/07/206/07/20

Keywords

  • High Performance Computing
  • I/O Scheduling
  • Performance Optimization
  • Tianhe-2 Supercomputer

Fingerprint

Dive into the research topics of 'Improving I/O performance for High Performance Computing with Application Forwarding Layer'. Together they form a unique fingerprint.

Cite this