CSGAN: Cyclic-Synthesized Generative Adversarial Networks for Image-to-Image Transformation

II. PROPOSED CSGAN ARCHITECTURE

数据集$X\in\left \{ (A_i), (B_i) \right \}_{i=1}^n$X∈{(Ai​),(Bi​)}i=1n​，包含$n$n个样本，每个样本包含来自domain $A$A和$B$B的2幅paired images

学习目标是2个生成器：$G_{AB}: A\rightarrow B$GAB​:A→B，$G_{BA}: B\rightarrow A$GBA​:B→A
$G_{AB}$GAB​利用real image $R_A$RA​生成synthesized image $Syn_B$SynB​
$G_{BA}$GBA​利用real image $R_B$RB​生成synthesized image $Syn_A$SynA​
$\begin{aligned} &Syn_B=G_{AB}(R_A) \qquad(1) \\ &Syn_A=G_{BA}(R_B) \qquad(2) \end{aligned}$​SynB​=GAB​(RA​)(1)SynA​=GBA​(RB​)(2)​

两个判别器$D_A, D_B$DA​,DB​
$D_A$DA​负责判别$R_A$RA​和$Syn_A$SynA​
$D_B$DB​负责判别$R_B$RB​和$Syn_B$SynB​

生成图像$Syn_A$SynA​和$Syn_B$SynB​被用于二次生成，得到$Cyc_A$CycA​和$Cyc_B$CycB​
$\begin{aligned} &Cyc_A=G_{BA}(Syn_B)=G_{BA}(G_{AB}(R_A)) \qquad(3) \\ &Cyc_B=G_{AB}(Syn_A)=G_{AB}(G_{BA}(R_B)) \qquad(4) \end{aligned}$​CycA​=GBA​(SynB​)=GBA​(GAB​(RA​))(3)CycB​=GAB​(SynA​)=GAB​(GBA​(RB​))(4)​

模型框架图如Fig.2所示

A. Proposed Cyclic-Synthesized Loss

Cyclic-Synthesized Loss的思想是，使用同一个生成器生成的Synthesized Image和Cycled Image必须近似，具体定义如下
$\begin{aligned} &\mathcal{L}_{CS_A}=\left \| Syn_A-Cyc_A \right \|_1 \qquad(5) \\ &\mathcal{L}_{CS_B}=\left \| Syn_B-Cyc_B \right \|_1 \qquad(6) \end{aligned}$​LCSA​​=∥SynA​−CycA​∥1​(5)LCSB​​=∥SynB​−CycB​∥1​(6)​

B. CSGAN Objective Function

CSGAN的objective function定义如下
$\begin{aligned} \mathcal{L}&\left ( G_{AB}, G_{BA}, D_A, D_B \right )=\mathcal{L}_{LSGAN_A}+\mathcal{L}_{LSGAN_B}\\ &+\lambda_A\mathcal{L}_{cyc_A}+\lambda_B\mathcal{L}_{cyc_B}+\mu_A\mathcal{L}_{CS_A}+\mu_B\mathcal{L}_{CS_B} \qquad(7) \end{aligned}$L​(GAB​,GBA​,DA​,DB​)=LLSGANA​​+LLSGANB​​+λA​LcycA​​+λB​LcycB​​+μA​LCSA​​+μB​LCSB​​(7)​
其中$\mathcal{L}_{LSGAN_A}, \mathcal{L}_{LSGAN_B}$LLSGANA​​,LLSGANB​​是least square adversarial loss，$\mathcal{L}_{cyc_A}, \mathcal{L}_{cyc_B}$LcycA​​,LcycB​​是Cycle-consistency Loss，$\mathcal{L}_{CS_A}, \mathcal{L}_{CS_B}$LCSA​​,LCSB​​是提出的Cyclic-Synthesized Loss

【局限性】
必须要样本是paired，才能使用Cyclic-Synthesized Loss