We present an application of conditional generative adversarial network (cGAN) to produce photo-realistic portraits based on human face sketches.
Our basic U-Net and PatchGAN model architecture is from pix2pix GAN. U-Net is agenerator that skips connections between each layer i and layer n-i of neural networks to preserve lower-layer information between inputs and outputs, and PatchGAN is a discriminator modeling on small patches of images to force high-frequency correctness.
Based on U-Net and PatchGAN, we tried different loss functions, including L1+cGAN, L2+cGAN, L1+GAN and L1+WGAN. By training the paired images of sketches and real photos, the results show that the L1+cGAN and L1+WGAN are able to produce pictures of acceptable quality. We even found that our L1+WGAN loss has better performance than the original pix2pix model.
The results of our application are promising: everyone can get any photo-realistic portraits by their own drafts!