A universal parameter-efficient fine-tuning approach for stereo image super-resolution

Despite advances in the use of the strategy of pre-training then fine-tuning in low-level vision tasks, the increasing size of models presents significant challenges for this paradigm, particularly in terms of training time and memory consumption. In addition, unsatisfactory results may occur when pre-trained single-image models are directly applied to a multi-image domain. In this paper, we propose an efficient method for transferring a pre-trained single-image super-resolution transformer network to the domain of stereo image super-resolution (SteISR) using a parameter-efficient fine-tuning approach. Specifically, the concept of stereo adapters and spatial adapters are introduced, which are incorporated into the pre-trained single-image super-resolution transformer network. Subsequently, only the inserted adapters are trained on stereo datasets. Compared with the classical full fine-tuning paradigm, our method can effectively reduce training time and memory consumption by 57% and 15%, respectively. Moreover, this method allows us to train only 4.8% of the original model parameters, achieving state-of-the-art performance on four commonly used SteISR benchmarks. This technology is expected to improve stereo image resolution in various fields such as medical imaging and autonomous driving, thereby indirectly enhancing the accuracy of depth estimation and object recognition tasks.