PAMS Quantized Super-Resolution via.pdf

dong

130

16页

0次

2023-09-22

100墨值下载

PAMS: Quantized Super-Resolution via

Parameterized Max Scale

Huixia Li

1†

, Chenqian Yan

1†

, Shaohui Lin

, Xiawu Zheng

Baochang Zhang

, Fan Yang

, Rongrong Ji

15!

Media Analytics and Compu ti n g Lab, Department of Artiﬁcial Intelligence,

School of Informatics, Xiam en University, China.

National U n iversity of Singapore, Singa pore.

Beihang Un i versity, China.

Huawei Technologies Co., Ltd.

Peng Cheng Laboratory.

{hxlee,zhengxiawu}@stu.xmu.edu.cn, {im.cqyan,shaohuilin007}@gmail.com,

bczhang@buaa.edu.cn, yangfan74@huawei.com, rrji@xmu.edu.cn

Abstract. Deep convolutional neural networks (DCNNs) have shown

dominant performance in the task of super-resolution (SR). However,

their heavy memory cost and computation overhead s ig n i ﬁ ca ntly rest ri ct

their practical deployments on resource-limited devices, which mainly

arise from the ﬂoating-point storage a n d operations between weights a n d

activations. Although previous endeavors mainly resort to ﬁxed-point

operations, quantizing both weights and activations with ﬁxed coding

lengths may cause signiﬁc a nt performance drop, especially on low bits.

Speciﬁcally, most state -o f- th e -a rt SR models wit h ou t batch normaliz a -

tion have a large dynamic quantization range, which also serves as an-

other cause of performance drop. To address these two issues, we propose

a new quantization scheme termed PArameterized Max Scale (PAMS),

which applies the trainable truncated parameter to explore the upper

bound of the quantization rang e adaptively. Fin a ll y, a structured knowl-

edge transfer (SKT) loss is introdu c ed to ﬁne-t u n e the quantized network.

Extensive experiments demonstrate that the proposed PAMS scheme can

well compress and accelerate the existing SR models such as EDSR and

RDN. Notably, 8-bit PAMS-EDSR improves PSNR on Set5 benchmark

from 32.095dB to 32.124dB with 2.42× compres sio n ratio, which achieves

a new state-of-the-art.

Keywords: Super Resolution · Network Quantization

1 Introduction

Single image super-resolution (SISR) aims to recover a high-resolution (HR) im-

age from the corresponding low-resolution (LR) one, which has been a research

†

Equal contribution.

Corresponding author.

2 Huixia Li et al.

quant

clamp

conv

PAMSBlock

conv

relu

ResBlock

convconv

relu

ResBlock

! !

conv

PAMSBlock

conv

upsampler

relu

conv

Low-level

Feature Extractor

High-level

Feature Extractor

Reconstruction

ℒ

#$%

Structured

Knowledge

ℒ

&'(

Pixel-wise

Feature

Fig. 1. The framework of our approach. The super-reso lu t io n opera ti o n is split into

three modules, i.e., low-level feature extractor, high-level fea t u re extractor and recon-

struction. We deploy PAMS with diﬀerent α on each activation layer in the high-level

feature extractor. quant den o t es the quantization operation and clamp represents the

clamp function of quantization. Dash lines denote t h e weights are quantized with the

maximum. Here, we illustrate EDSR as backbone.

hot spot in computer vision for decades. Coming with the advances of dee p learn-

ing, deep convolutional neural networks (DCNNs) [ 7, 18, 23] have dominated SR

in recent years. These networks commonly use an extraction module to extract

a series of featur e maps from the LR image, cascaded with the u p- samp l in g

module, which stepwisely increases the resolution to reconstruct the HR image.

As one of the pioneering works for deep learning based SR, Dong et al. [7]

introduce three convolution layers to achieve high vis ual perception. After that,

Kim et al. [18] design a deep network VDSR by stacking 20 convolutional layers.

Subsequent works mainly resort to increasin g the network depth to improve SR

performance. For instance, Lim et al. [23] propose the enhanced deep resi d ual

networks (e.g. EDSR and MDSR) and remove batch normalization (BN) [16] to

reduce the memory consumption, which however still requires at least 64 convo-

lution layers (more than 160 layers for MDSR). A channel atte ntion mechanism

equipped into the RCAN model [38] requires more than 400 layers with about

30B FLOPs and 13M parameters. Such si gni ﬁ cant computation and memory

overheads restrict their applications in scenarios where only limited memory

and computation resources are avai l abl e . Consequently, compressing deep SR

networks has attracted increasing attention recently [29].

Beyond SR, neural network compression and acceleration have been widely

studied in the literature. Representative works include parameter pruning [11–13,

27, 26, 22], low-rank approximation [6, 24, 25], compact networks [33, 28], knowl-

edge distillat ion (KD) [14, 32], neural architecture search (NAS) [43, 40] and

quantization [5, 17]. Considering the uniq ue structures such as EDSR [23] and

RDN [39] in SR, it is by nature t o leverage quantization schemes to acceler-

ate and compress SR networks, i.e., by converting full-precision weights [17],

activations [2], and gradients [41] to low bits.

of 16

100墨值下载

大模型-必读论文及复现代码大模型 pams quantized

关注

评论