In the past few years, underwater image enhancement has attracted an increasing amount of research work because it plays an important role in computer vision related underwater tasks, such as aquatic robotics and marine engineering. However, wavelength-dependent light absorption and scattering introduces unpleasant color distortion and reduces the visibility of images in underwater scenes. In this paper, we propose a two-branch multi-scale (MSN) and multi-patch (MPN) synergy network, called Multi-SPNet, which aims to improve the contrast, brightness, and eliminate color distortion of non-uniform degraded underwater images. Specifically, the features extracted from multi-scale and multi-patch branches are interweaved for progressive image enhancement, where the upper and the lower branches utilize efficient Transformer blocks for learning multi-scale representation from low-to-high resolution and aggregating features via multiple image patches from fine to coarse level, respectively. The complementary branches can construct a synergistic merge to employ their mutual benefits for local and non-local pixel interactions. Extensive experiments on synthetic and real-world underwater image datasets clearly prove the effectiveness and superiority of the proposed Multi-SPNet against the state-of-the-art models both qualitatively and quantitatively.