diff --git a/README.md b/README.md
index 7cf412c..d46d788 100644
--- a/README.md
+++ b/README.md
@@ -1,72 +1,18 @@
# TP-GAN
-
-Official TP-GAN Tensorflow implementation for the ICCV17 paper "[Beyond Face Rotation: Global and Local Perception GAN for Photorealistic and Identity Preserving Frontal View Synthesis](http://openaccess.thecvf.com/content_ICCV_2017/papers/Huang_Beyond_Face_Rotation_ICCV_2017_paper.pdf)" by [Huang, Rui](http://www.andrew.cmu.edu/user/ruih2/) and Zhang, Shu and Li, Tianyu and [He, Ran](http://www.nlpr.ia.ac.cn/english/irds/People/rhe.html).
-
-The goal is to **recover a frontal face image of the same person from a single face image under any poses**.
-
-Here are some examples from the paper.
-
-### Testing images
-
-Synthesized testing images of all poses, corresponding illumination in Setting 2 (and its cropped input) in MultiPIE can be obtained here [Google Drive](https://drive.google.com/file/d/1Kx0sMjFTzLX3-rZ03TAVBAj-gcd9rJrd/view?usp=sharing).
-
-FAQ: **Synthesized**(not the original) images for other illumination condition and/or training set can be obtained upon request. Unfortunately, I cannot redistribute the original dataset due to copyright. If you would like to access the original MultiPIE dataset, please contact [MultiPIE](http://www.cs.cmu.edu/afs/cs/project/PIE/MultiPie/Multi-Pie/Home.html).
-
-### Random examples
-
-Here are random examples of 10 testing image pairs for each degree.
-
-15 and 30 degrees:
-
-
-
-
-45 and 60 degrees:
-
-
-
-
-75 and 90 degrees:
-
-
-
-
-### Note
-
-It was initially written in Tensorflow 0.12. If you have implemented another version, I'll be happy to reference it here.
-
-This is an initial release of code, which may not be fully tested. Refinement, input data example, pre-trained models, and precomputed testing image features will come later.
-
-The input is cropped with the Matlab script `face_db_align_single_custom.m`, which accepts 5 keypoints and outputs a cropped image and transformed keypoints.
-
-Some example cropping outputs is shown in folder `data-example`.
-
-Our 90-degree model only used 45-90 degree images for training. Other models we trained didn't use 90 degree images. 90 degree images' left and right eye patches coincide.
-
-The 5 keypoints can be extracted from off-the-shelf landmark detectors, e.g. 'Zhang et al. Combining Data-driven and Model-driven Methods for Robust Facial Landmark Detection, 2016'. The synthesis performance is similar to using manually labelled keypoints. See released keypoint files below.
-
-We thank Xiang Wu for providing the [face feature network](https://github.com/AlfredXiangWu/face_verification_experiment). We load it as `DeepFace` in the code, the weights are from a custom Light-CNN cafeemodel file. Our implementation borrowed code from the [dcgan repo](https://github.com/carpedm20/DCGAN-tensorflow).
-
-Update 0.11:
-
-- Releasing *5 keypoint locations* for MultiPIE Session 1-4 dataset. Please download from [here](https://drive.google.com/open?id=1bbMH71SNnX09r7kZj7ExqRLTEUHZVkno). Most of the 60-90 degrees images are labelled manually, others come from MTCNNv2 detector. If you like it, please consider citing our paper.
-- Adding `DeepFace168.pickle` weights file for Light-CNN. Please note this is an improved version than the one originally used in the experiment.
-
-### Citation and Contact
-
-If you like our work or find our code useful, welcome to cite our paper!
-
-Any suggestion and/or comment would be valuable. Please send an email to Rui at huangrui@cmu.edu or other authors.
-
- @InProceedings{Huang_2017_ICCV,
- author = {Huang, Rui and Zhang, Shu and Li, Tianyu and He, Ran},
- title = {Beyond Face Rotation: Global and Local Perception GAN for Photorealistic and Identity Preserving Frontal View Synthesis},
- booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
- month = {Oct},
- year = {2017}
- }
-
-### License
-
-This code is freely available for free non-commercial use, and may be redistributed under the conditions set by the license. Please, see the [license](https://github.com/HRLTY/TP-GAN/blob/master/LICENSE) for further details. For commercial queries, please contact [Rui Huang](http://www.andrew.cmu.edu/user/ruih2/) and [Ran He](http://www.nlpr.ia.ac.cn/english/irds/People/rhe.html).
-
+data中5pt文件为人脸关键点部位,用mtcnn模型识别,五行分别是左右眼嘴鼻的x和y坐标,代码能在tensorflow0.12版本下跑,不想修改兼容其他版本了
+mtcnn的模型不能在tf0.12下跑,用5pt文件做下中转
+没用到face_db_align_single_custom.m
+tensorflow0.12要求CUDA版本Cuda 8.0 and CudNN 5.1?
+# 使用镜像运行
+sudo nvidia-docker pull nvidia/cuda:8.0-cudnn5-devel-ubuntu16.04能跑通
+拉取镜像后需要:
+1.apt-get update后apt-get install python3-dev python3-pip vim
+2.再pip3 install tensorflow-gpu==0.12 scipy pillow
+3.下载代码,放数据集后修改下路径
+4.修改TP_GAN-Mar6FS_0.12.py里面的参数后运行此文件
+### 测试单张图片
+1.放好图片,修改mtcnn.py里的文件参数后运行,生成.5pt文件
+2.修改test_pt.py后运行,输出结果到本文件夹
+## 数据集相关信息见https://github.com/HRLTY/TP-GAN
+记得修改下identical output的数目
+uitils.py里的scipy.misc的imread和imsave用imageio的imread和imwrite代替(较新版本的scipy.misc去除了这两个功能)
diff --git a/TP_GAN-Mar6FS.py b/TP_GAN-Mar6FS_0.12.py
similarity index 98%
rename from TP_GAN-Mar6FS.py
rename to TP_GAN-Mar6FS_0.12.py
index e6316fa..c8972eb 100644
--- a/TP_GAN-Mar6FS.py
+++ b/TP_GAN-Mar6FS_0.12.py
@@ -21,7 +21,7 @@
from utils import pp, visualize, to_json
import tensorflow as tf
-
+import logging#pt
#These parameters should provide a good initialization, but if for specific refinement, you can adjust them during training.
ALPHA_ADVER = 2e1
@@ -58,8 +58,8 @@
flags.DEFINE_integer("epoch", 250, "Epoch to train [25]")
flags.DEFINE_float("learning_rate", 1e-4, "Learning rate of for adam [0.0002]")
flags.DEFINE_float("beta1", 0.9, "Momentum term of adam [0.5]")
-flags.DEFINE_integer("train_size", np.inf, "The size of train images [np.inf]")
-flags.DEFINE_integer("batch_size", 20, "The size of batch images [64]")
+flags.DEFINE_integer("train_size", 1156, "The size of train images [np.inf]")
+flags.DEFINE_integer("batch_size", 4, "The size of batch images [64]")
flags.DEFINE_integer("image_size", 128, "The size of image to use (will be center cropped) [108]")
flags.DEFINE_integer("output_size", 128, "The size of the output images to produce [64]")
flags.DEFINE_integer("c_dim", 3, "Dimension of image color. [3]")
@@ -67,8 +67,8 @@
flags.DEFINE_string("checkpoint_dir", "checkpoint60", "Directory name to save the checkpoints [checkpoint]")
flags.DEFINE_string("sample_dir", "samples", "Directory name to save the image samples [samples]")
flags.DEFINE_boolean("is_train", True, "True for training, False for testing [False]")
-flags.DEFINE_boolean("is_crop", False, "True for training, False for testing [False]")
-flags.DEFINE_boolean("visualize", False, "True for visualizing, False for nothing [False]")
+flags.DEFINE_boolean("is_crop", True, "True for training, False for testing [False]")
+flags.DEFINE_boolean("visualize", True, "True for visualizing, False for nothing [False]")
FLAGS = flags.FLAGS
class DCGAN(object):
@@ -91,11 +91,11 @@ def __init__(self, sess, image_size=128, is_crop=True,
c_dim: (optional) Dimension of image color. For grayscale input, set to 1. [3]
"""
self.test_batch_size = batch_size
- self.save_interval = 300
+ self.save_interval = 500
self.sample_interval = 150
self.sess = sess
self.is_grayscale = (c_dim == 1)
- self.batch_size = 10
+ self.batch_size = FLAGS.batch_size
self.sample_run_num = 15
self.testing = False
self.testingphase = 'FS'
@@ -120,11 +120,13 @@ def __init__(self, sess, image_size=128, is_crop=True,
# batch normalization : deals with poor initialization helps gradient flow
random.seed()
- self.DeepFacePath = '/home/shu.zhang/ruihuang/data/DeepFace.pickle'
+ self.DeepFacePath = './DeepFace168.pickle'
self.dataset_name = dataset_name
self.checkpoint_dir = checkpoint_dir
- self.loadDeepFace(self.DeepFacePath)
- self.build_model()
+ self.loadDeepFace()#pt
+ self.build_model()#pt
+
+
def build_model(self):
@@ -222,6 +224,7 @@ def build_model(self):
_,_,_,_, self.G_pool5, self.Gvector = self.FeatureExtractDeepFace(tf.reduce_mean(self.G, axis=3, keep_dims=True))
_,_,_,_, self.label_pool5, self.labelvector = self.FeatureExtractDeepFace(tf.reduce_mean(self.g_labels, axis=3, keep_dims=True), reuse=True)
_,_,_,_, _, self.samplevector = self.FeatureExtractDeepFace(tf.reduce_mean(self.sample_images_nocode, axis=3, keep_dims=True), reuse=True)
+
#self.Dv, self.Dv_logits = self.discriminatorClassify(self.Gvector)
#self.dv_loss = tf.reduce_sum(tf.nn.sparse_softmax_cross_entropy_with_logits(self.Dv_logits, self.verify_labels))
self.dv_loss = tf.reduce_mean(tf.abs(self.Gvector-self.labelvector))
@@ -266,7 +269,6 @@ def build_model(self):
errcheck32 = tf.abs(self.check_sel32 - self.g32_labels) #* mask32
errcheck64 = tf.abs(self.check_sel64 - self.g64_labels) #* mask64
errcheck128 = tf.abs(self.check_sel128 - self.g_labels) #* mask128
-
self.weightedErrL1 = tf.reduce_mean(tf.reduce_sum(tf.reduce_sum(errL1, 1), 1))
self.symErrL1 = tf.reduce_mean(tf.reduce_sum(tf.reduce_sum(symL1(#self.processor(self.G)
tf.nn.avg_pool(self.G, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')
@@ -275,7 +277,6 @@ def build_model(self):
self.symErrL2 = tf.reduce_mean(tf.reduce_sum(tf.reduce_sum(symL1(self.processor(self.G2)), 1), 1))
self.weightedErrL3 = tf.reduce_mean(tf.reduce_sum(tf.reduce_sum(errL1_3, 1), 1))
self.symErrL3 = tf.reduce_mean(tf.reduce_sum(tf.reduce_sum(symL1(self.processor(self.G3, reuse=True)), 1), 1))
-
cond_L12 = tf.abs(tf.image.resize_bilinear(self.G, [64,64]) - tf.stop_gradient(self.G2))
#self.condErrL12 = tf.reduce_mean(tf.reduce_sum(tf.reduce_sum(cond_L12, 1), 1))
#cond_L23 = tf.abs(tf.image.resize_bilinear(self.G2, [32,32]) - tf.stop_gradient(self.G3))
@@ -293,14 +294,15 @@ def build_model(self):
self.g_loss = L1_1_W * (self.weightedErrL1 + SYM_W * self.symErrL1) + L1_2_W * (self.weightedErrL2 + SYM_W * self.symErrL2) \
+ L1_3_W * (self.weightedErrL3 + SYM_W * self.symErrL3)
self.g_loss += BELTA_FEATURE * self.dv_loss + ALPHA_ADVER * self.g_loss_adver + IDEN_W * self.idenloss + self.tv_loss * TV_WEIGHT
-
self.rot_loss = PART_W * (self.eyel_loss + self.eyer_loss + self.nose_loss + self.mouth_loss)
#self.sel_loss = self.weightedErr_check32 + self.weightedErr_check64 + self.weightedErr_check128
#self.g_loss += self.sel_loss
-
self.var_file = open('var_log.txt', mode='a')
t_vars = [var for var in tf.trainable_variables() if 'FeatureExtractDeepFace' not in var.name \
and 'processor' not in var.name]
+ print('done!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!')
+
+
def isTargetVar(name, tokens):
for token in tokens:
if token in name:
@@ -329,13 +331,14 @@ def isTargetVar(name, tokens):
self.saver = tf.train.Saver(t_vars, max_to_keep=2)
+
def train(self, config):
"""Train DCGAN"""
#data = glob(os.path.join("./data", config.dataset, "*.jpg"))
data = MultiPIE(LOAD_60_LABEL=LOAD_60_LABEL, GENERATE_MASK=USE_MASK, RANDOM_VERIFY=RANDOM_VERIFY, MIRROR_TO_ONE_SIDE = True, source = self.testingphase)
#np.random.shuffle(data)
config.sample_dir += '{:05d}'.format(random.randint(1,100000))
-
+ #print(self.__dict__,66666666666)
d_optim = tf.train.AdamOptimizer(config.learning_rate, beta1=config.beta1) \
.minimize(self.d_loss, var_list=self.d_vars)
#clip_D = [p.assign(tf.clip_by_value(p, -CLIP_D, CLIP_D)) for p in self.d_vars]
@@ -361,9 +364,11 @@ def train(self, config):
print(" [*] Load SUCCESS")
else:
print(" [!] Load failed...")
-
+ print(555555555555555555555,self.test_batch_size,self.sample_run_num)
+ #self.test_batch_size * self.sample_run_num分别是10,15
sample_images, filenames ,sample_eyel, sample_eyer, sample_nose, sample_mouth, \
sample_labels, sample_leyel, sample_leyer, sample_lnose, sample_lmouth, sample_iden = data.test_batch(self.test_batch_size * self.sample_run_num)
+
if not self.testing:
sample_imagesT, filenamesT ,sample_eyelT, sample_eyerT, sample_noseT, sample_mouthT, \
sample_labelsT, sample_leyelT, sample_leyerT, sample_lnoseT, sample_lmouthT, sample_idenT = data.test_batch(self.test_batch_size * self.sample_run_num * RANK_MUL, Pose=RANGE)
@@ -775,20 +780,27 @@ def evaluate(self,epoch, idx, batch_idxs, start_time, mode,
self.f.flush()
print(tobePrint)
#DEEPFACE MODEL BEGINS---
- def loadDeepFace(self, DeepFacePath):
+ def loadDeepFace(self, DeepFacePath= None):
if DeepFacePath is None:
path = sys.modules[self.__class__.__module__].__file__
# print path
path = os.path.abspath(os.path.join(path, os.pardir))
# print path
- path = os.path.join(path, "DeepFace.pickle")
+ path = os.path.join(path, "DeepFace168.pickle")
DeepFacePath = path
logging.info("Load npy file from '%s'.", DeepFacePath)
if not os.path.isfile(DeepFacePath):
logging.error(("File '%s' not found. "), DeepFacePath)
sys.exit(1)
- with open(DeepFacePath,'r') as file:
- self.data_dict = pickle.load(file)
+
+ #with open(DeepFacePath,'r') as file:
+ # self.data_dict = pickle.load(file)
+ with open(DeepFacePath,'rb') as file: # Binary read
+ u = pickle._Unpickler(file)
+ u.encoding = 'latin1'
+ p = u.load()
+ self.data_dict = p
+
print("Deep Face pickle data file loaded")
def FeatureExtractDeepFace(self, images, name = "FeatureExtractDeepFace", reuse=False):
@@ -798,7 +810,7 @@ def FeatureExtractDeepFace(self, images, name = "FeatureExtractDeepFace", reuse=
scope.reuse_variables()
conv1 = self._conv_layer(images, name='conv1')
- print(3, type(3))
+ #print(3, type(3))
slice1_1, slice1_2 = tf.split(3, 2, conv1)
eltwise1 = tf.maximum(slice1_1, slice1_2)
pool1 = tf.nn.max_pool(eltwise1, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1],
diff --git a/data/read_csv.py b/data/read_csv.py
new file mode 100644
index 0000000..2045c29
--- /dev/null
+++ b/data/read_csv.py
@@ -0,0 +1,12 @@
+import csv
+import os
+import numpy as np
+PATH = '/home/ubuntu3000/pt/TP-GAN/data/45_5pt'
+trans_points = np.empty([5,2],dtype=np.int32)
+for filename in os.listdir(PATH):
+ with open(os.path.join(PATH,filename), 'r') as csvfile:
+ reader = csv.reader(csvfile, delimiter=' ')
+ for ind,row in enumerate(reader):
+ trans_points[ind,:] = row
+ print(trans_points)
+ break
diff --git a/data/write_csv.py b/data/write_csv.py
new file mode 100644
index 0000000..cc32d49
--- /dev/null
+++ b/data/write_csv.py
@@ -0,0 +1,19 @@
+import csv
+import os
+
+def create_csv(path_):
+ PATH = '/home/ubuntu3000/pt/TP-GAN/data/45'
+ data=[]
+ data_test=[]
+ path=os.path.join(path_,'train.csv')
+ with open(path, "w") as file:
+ csv_file = csv.writer(file)
+ for filename in os.listdir(PATH):
+ head = [filename]
+ if not 'test' in filename:
+ data.append(head)
+ csv_file.writerows(data)
+
+if __name__== '__main__':
+
+ create_csv('/home/ubuntu3000/pt/TP-GAN/data/')
diff --git a/mtcnn.pb b/mtcnn.pb
new file mode 100644
index 0000000..4cc80c6
Binary files /dev/null and b/mtcnn.pb differ
diff --git a/mtcnn.py b/mtcnn.py
new file mode 100644
index 0000000..f23a20b
--- /dev/null
+++ b/mtcnn.py
@@ -0,0 +1,88 @@
+import argparse,cv2
+#from mtcnn import MTCNN
+import tensorflow as tf
+import os
+import csv
+class MTCNN:
+
+ def __init__(self, model_path, min_size=40, factor=0.709, thresholds=[0.6, 0.7, 0.7]):
+ self.min_size = min_size
+ self.factor = factor
+ self.thresholds = thresholds
+
+ graph = tf.Graph()
+ with graph.as_default():
+ with open(model_path, 'rb') as f:
+ graph_def = tf.GraphDef.FromString(f.read())
+ tf.import_graph_def(graph_def, name='')
+ self.graph = graph
+ config = tf.ConfigProto(
+ allow_soft_placement=True,
+ intra_op_parallelism_threads=4,
+ inter_op_parallelism_threads=4)
+ config.gpu_options.allow_growth = True
+ self.sess = tf.Session(graph=graph, config=config)
+
+ def detect(self, img):
+ feeds = {
+ self.graph.get_operation_by_name('input').outputs[0]: img,
+ self.graph.get_operation_by_name('min_size').outputs[0]: self.min_size,
+ self.graph.get_operation_by_name('thresholds').outputs[0]: self.thresholds,
+ self.graph.get_operation_by_name('factor').outputs[0]: self.factor
+ }
+ fetches = [self.graph.get_operation_by_name('prob').outputs[0],
+ self.graph.get_operation_by_name('landmarks').outputs[0],
+ self.graph.get_operation_by_name('box').outputs[0]]
+ prob, landmarks, box = self.sess.run(fetches, feeds)
+ return box, prob, landmarks
+
+def test_image(PATH):
+ mtcnn = MTCNN('./mtcnn.pb')
+ save_PATH='/home/ubuntu3000/pt/TP-GAN/data/45_5pt'
+
+ for imgpath in os.listdir(PATH):
+ path = os.path.join(save_PATH,imgpath.replace('.png','.5pt'))
+ csvfile = open(path, "w")
+ img = cv2.imread(os.path.join(PATH,imgpath))
+ data=[]
+ bbox, scores, landmarks = mtcnn.detect(img)
+ for box, pts in zip(bbox, landmarks):
+ pts = pts.astype('int32')
+ for i in range(5):
+ row = str(pts[i+5]) + ' '+ str(pts[i])+'\n'
+ csvfile.write(row)
+ csvfile.close()
+
+def create_img(PATH):
+ mtcnn = MTCNN('./mtcnn.pb')
+ csvfile = open('test_tem.5pt', "w")
+ img_rec = cv2.imread(PATH)
+ data=[]
+ bbox, scores, landmarks = mtcnn.detect(img_rec)
+ for box, pts in zip(bbox, landmarks):
+ bbox=bbox.astype('int32')
+ pts = pts.astype('int32')
+ print([int(box[0]),int(box[2]),int(box[1]),int(box[3])])
+ for i in range(5):
+ row = str(pts[i+5]) + ' '+ str(pts[i])+'\n'
+ #img=cv2.circle(img,(pts[i+5],pts[i]),1,(0,255,0),2)
+ img_rec=img_rec[int(box[0])-20:int(box[2])+20,int(box[1])-50:int(box[3])+50]
+ img_rec=cv2.resize(img_rec,(128,128))
+ cv2.imwrite('test_tem.png',img_rec)
+ bbox, scores, landmarks = mtcnn.detect(img_rec)
+ for box, pts in zip(bbox, landmarks):
+ pts = pts.astype('int32')
+ for i in range(5):
+ row = str(pts[i+5]) + ' '+ str(pts[i])+'\n'
+ img_vis=cv2.circle(img_rec,(pts[i+5],pts[i]),1,(0,255,0),2)
+ csvfile.write(row)
+ cv2.imwrite('test_tem_vis.png',img_vis)
+ csvfile.close()
+
+def cut(PATH):
+ img = cv2.imread(PATH)
+ #img=img[]
+ cv2.imwrite('test_tem.png',img)
+
+if __name__ == '__main__':
+ create_img('index.jpeg')
diff --git a/net_input_everything_featparts.py b/net_input_everything_featparts.py
index 2edf907..7acd4a2 100644
--- a/net_input_everything_featparts.py
+++ b/net_input_everything_featparts.py
@@ -28,7 +28,6 @@
import random
import re
import csv
-
# Process images of this size. Note that this differs from the original CIFAR
# image size of 32 x 32. If one alters this number, then the entire model
# architecture will change and any model would need to be retrained.
@@ -39,18 +38,16 @@
MOUTH_H = 32; MOUTH_W = 48;
re_pose = re.compile('_\d{3}_')
re_poseIllum = re.compile('_\d{3}_\d{2}_')
+#眼嘴鼻的高宽都是固定的,如上
class MultiPIE():
"""Reads and parses examples from MultiPIE data filelist
"""
def __init__(self, datasplit='train', Random=True, LOAD_60_LABEL=False, MIRROR_TO_ONE_SIDE=True, RANDOM_VERIFY=False,
GENERATE_MASK=False, source='without90', testing = False):
- self.dir = '/home/ruihuang/data/FS_aligned/'
- self.csvpath = '/home/ruihuang/data/{}/{}.csv'
- self.feat5ptDir = '/home/shu.zhang/ruihuang/data/FS_t5pt/'
- self.test_dir = '/home/shu.zhang/ruihuang/data/testlist/FS/{}.csv'
+ self.dir = '/home/TP-GAN/data/45/'#图片文件夹
self.testing = testing
-
+
self.split = datasplit
self.random = Random
self.seed = None
@@ -58,13 +55,10 @@ def __init__(self, datasplit='train', Random=True, LOAD_60_LABEL=False, MIRROR_T
self.MIRROR_TO_ONE_SIDE = MIRROR_TO_ONE_SIDE
self.RANDOM_VERIFY = RANDOM_VERIFY
self.GENERATE_MASK = GENERATE_MASK
- self.cameraPositions = {'24_0': (+90, '10'),'01_0' : (+75, '08'), '20_0' : (+60, '08'), '19_0' : (+45, '09'), '04_1' : (+30, '07'), '05_0' : (+15, '06'), #left
- '05_1' : (0,'06'), #center
- '14_0' : (-15,'06'), '13_0' : (-30, '05'), '08_0' : (-45, '15'),'09_0' : (-60, '15'),'12_0' : (-75, '15'),'11_0' : (-90, '15')} #right
-
if not testing:
- split_f = self.csvpath.format(source, self.split)
- split_f_test = self.csvpath.format(source, 'test')
+ split_f = '/home/TP-GAN/data/train.csv'
+ split_f_test = '/home/TP-GAN/data/test.csv'
+ #self.indices是图片的名称大全,原本是csv文件里读取的
self.indices = open(split_f, 'r').read().splitlines()
self.indices_test = open(split_f_test, 'r').read().splitlines()
self.size = len(self.indices)
@@ -77,7 +71,7 @@ def __init__(self, datasplit='train', Random=True, LOAD_60_LABEL=False, MIRROR_T
random.seed(self.seed)
self.idx = random.randint(0, len(self.indices)-1)
else:#only load test images for a separate list file.
- split_f_test = self.test_dir.format(source)
+ split_f_test = '/home/TP-GAN/data/test.csv'
self.indices_test = open(split_f_test, 'r').read().splitlines()
self.size = 0
self.test_size = len(self.indices_test)
@@ -104,20 +98,9 @@ def test_batch(self, test_batch_size=100,Random = True, Pose = -1):
random.shuffle(self.indices_test)
#resume randomeness for training
random.seed(self.seed)
+ #这个是什么?LOAD_60_LABEL,TODO:如果LOAD_60_LABEL，则跳过小于45度的图片；Pose为指定角度
j = 0
for i in range(test_batch_size):
- if self.LOAD_60_LABEL:
- while True:
- pose = abs(self.findPose(self.indices_test[j % len(self.indices_test)]))
- if pose >= 45:
- break
- j += 1
- if Pose != -1:
- while True:
- pose = abs(self.findPose(self.indices_test[j % len(self.indices_test)]))
- if pose == Pose:
- break
- j += 1
print(j, end=' ')
images[i, ...], feats = self.load_image(self.indices_test[j % len(self.indices_test)])
eyel[i,...] = feats[1]
@@ -133,6 +116,7 @@ def test_batch(self, test_batch_size=100,Random = True, Pose = -1):
j += 1
print('\n')
if not self.testing:
+ #labels应该是正面的图片,leyel是label_eye_left
return images, filenames, eyel, eyer, nose, mouth,\
labels, leyel, leyer, lnose, lmouth, idenlabels
else:
@@ -168,27 +152,17 @@ def next_image_and_label_mask_batch(self, batch_size, imageRange=-1,imageRangeLo
lnose = np.empty([batch_size, NOSE_H, NOSE_W, 3], dtype=np.float32)
lmouth = np.empty([batch_size, MOUTH_H, MOUTH_W, 3],dtype=np.float32)
- if self.GENERATE_MASK:
- masks = np.empty([batch_size, IMAGE_SIZE, IMAGE_SIZE, 3])
- else:
- masks = None
+
+ masks = None
if self.RANDOM_VERIFY:
verifyImages = np.empty([batch_size, IMAGE_SIZE, IMAGE_SIZE, 3])
- #verifyLabels = np.empty([batch_size, 1], dtype=np.float32)
verifyLabels = np.empty([batch_size], dtype=np.int32)
else:
verifyImages = None; verifyLabels = None
for i in range(batch_size):
if imageRange != -1:
- while True:
- pose = abs(self.findPose(self.indices[self.idx]))
- if not self.LOAD_60_LABEL:
- if pose <= imageRange and pose >= imageRangeLow:
- break
- else:
- if pose <= imageRange and pose >= 45:
- break
+ if True:
self.updateidx()
images[i, ...], feats = self.load_image(self.indices[self.idx])
filename = self.indices[self.idx]
@@ -204,6 +178,7 @@ def next_image_and_label_mask_batch(self, batch_size, imageRange=-1,imageRangeLo
nose[i,...] = feats[3]
mouth[i, ...] = feats[4]
self.updateidx()
+ #labels是什么,masks是什么,verifyImages和images区别,poselabels是位置角度吗
return images, labels, masks, verifyImages, verifyLabels, poselabels, idenlabels, landmarklabels,\
eyel, eyer, nose, mouth, leyel, leyer, lnose, lmouth
@@ -215,6 +190,7 @@ def updateidx(self):
if self.idx == len(self.indices):
self.idx = 0
def load_image(self, filename):
+ #读取一个图片
"""
Load input image & codemap and preprocess:
- cast to float
@@ -225,37 +201,20 @@ def load_image(self, filename):
in_ = np.array(im, dtype=np.float32)
in_ /= 256
features = self.GetFeatureParts(in_, filename)
- if self.MIRROR_TO_ONE_SIDE and self.findPose(filename) < 0:
- in_ = in_[:,::-1,:]
return in_, features
+ #训练使用,输入图片的路径,返回侧面的照片,并用GetFeatureParts得到它的各个部位并返回
def load_label_mask(self, filename, labelnum=-1):
-
_, labelname = self.findSameIllumCodeLabelpath(filename)
+ #返回图片路径
im = Image.open(self.dir + labelname)
- if self.MIRROR_TO_ONE_SIDE and self.findPose(labelname) < 0:
+ if self.MIRROR_TO_ONE_SIDE:
im = im.transpose(Image.FLIP_LEFT_RIGHT)
- if self.GENERATE_MASK:
- codeImg = codeImg.transpose(Image.FLIP_LEFT_RIGHT)
-
- if self.GENERATE_MASK:
- code = np.array(codeImg, dtype=np.float32)
- mask = (code != np.zeros(code.shape, code.dtype))
- mask = mask.astype(np.float32, copy=False)
- RATIO = 0.1
- mask *= 1-RATIO
- mask += RATIO
- else:
- mask = None
-
+ mask = None #pt
label = np.array(im, dtype=np.float32)
- #label -= label.mean()
- #label /= label.std()
label /= 256
- #label -= 1
feats = self.GetFeatureParts(label, labelname, label=True)
if not self.LOAD_60_LABEL:
- if self.findPose(filename) < 0:#random.random() < 0.5:
#print("fipping!")
label = label[:,::-1,:]
feats[1][...] = feats[1][:,::-1,:]
@@ -270,7 +229,6 @@ def load_label_mask(self, filename, labelnum=-1):
return label, mask, feats
- def load_label_mask_with_veri(self, filename, labelnum=-1):
label, mask = self.load_label_mask(filename, labelnum)
#if(random.random() > 0.5):#positive
@@ -287,22 +245,22 @@ def load_label_mask_with_veri(self, filename, labelnum=-1):
return label, mask, veryImArray, int(randomSubjectPath[0:3])
else:
return label, mask, None, None
-
- def findBestIllumCodeImagepath(self, fullpath):
+ #输入图片路径，寻找那个角度的最好的光照条件的图片编号
span = re_pose.search(fullpath).span()
camPos = list(fullpath[span[0]+1:span[1]-1])
camPos.insert(2,'_')
camPos = ''.join(camPos)
#get 01_0 like string
bestIllum = self.cameraPositions[camPos][1]
-
+ #bestIllum什么鬼。。。日
labelpath = list(fullpath)
+ #bestIllum是一个字符串。。加:索引干嘛
labelpath[span[1]:span[1]+2] = bestIllum[:]
labelpath = ''.join(labelpath)
codepath = str(labelpath).replace('cropped', 'code')
return (codepath, labelpath)
- def findCodeLabelpath(self, fullpath, labelnum):
+ #两个参数，第一个地址，什么的地址？第二个是标签的序号？？
span = re_poseIllum.search(fullpath).span()
#print span
#camPosIllu =fullpath[span[0]+1:span[1]-1]
@@ -331,101 +289,53 @@ def findCodeLabelpath(self, fullpath, labelnum):
if replace == None:
print('damn labelnum bug!')
return (codepath, labelpath)
-
+
+ #input 图片路径,return 监督图片,TODO:这个要改
def findSameIllumCodeLabelpath(self, fullpath):
- span = re_poseIllum.search(fullpath).span()
- tempath = list(fullpath)
- if self.LOAD_60_LABEL:
- if self.findPose(fullpath) >= 0:
- tempath[span[0]+1:span[0]+4] = '190' #+45
- else:
- tempath[span[0]+1:span[0]+4] = '080' #-45
- else:
- tempath[span[0]+1:span[0]+4] = '051'
- labelpath = ''.join(tempath)
- codepath = str(labelpath).replace('cropped', 'code')
+ #输入路径，提取路径中某个东西
+ labelpath = fullpath.replace('cropped','cropped_test')
+ codepath='_'
return (codepath, labelpath)
-
+
+ #这个函数意思是
def findPose(self, fullpath):
- span = re_pose.search(fullpath).span()
- camPos = list(fullpath[span[0]+1:span[1]-1])
- camPos.insert(2,'_')
- camPos = ''.join(camPos)
- #get 01_0 like string
- return self.cameraPositions[camPos][0]
-
+ return +60
+
+ ##输入img_resize是128x128的Image.open,filename是用来寻找标注文件;返回trans_points和图片各个部位的裁剪
def GetFeatureParts(self, img_resize, filename, label=False):
#crop four parts
trans_points = np.empty([5,2],dtype=np.int32)
- if True:#not label:
- featpath = self.feat5ptDir + filename[0:-15] + '_trans.5pt'
- #print(filename)
+ if True:
+ featpath = os.path.join('/home/TP-GAN/data/45_5pt',filename.replace('.png','.5pt'))
with open(featpath, 'r') as csvfile:
reader = csv.reader(csvfile, delimiter=' ')
for ind,row in enumerate(reader):
- trans_points[ind,:] = row
- #print(trans_points)
+ if not ind >=5:
+ trans_points[ind,:] = row
+
eyel_crop = np.zeros([EYE_H,EYE_W,3], dtype=np.float32);
crop_y = int(trans_points[0,1] - EYE_H / 2);
crop_y_end = crop_y + EYE_H;
crop_x = int(trans_points[0,0] - EYE_W / 2);
crop_x_end = crop_x + EYE_W;
+
eyel_crop[...] = img_resize[crop_y:crop_y_end,crop_x:crop_x_end,:];
-
+ ##########################################################
eyer_crop = np.zeros([EYE_H,EYE_W,3], dtype=np.float32);
crop_y = int(trans_points[1,1] - EYE_H / 2)
crop_y_end = crop_y + EYE_H;
crop_x = int(trans_points[1,0] - EYE_W / 2);
crop_x_end = crop_x + EYE_W;
eyer_crop[...] = img_resize[crop_y:crop_y_end,crop_x:crop_x_end,:];
-
-
+ #####################################################
month_crop = np.zeros([MOUTH_H,MOUTH_W,3], dtype=np.float32);
crop_y = int((trans_points[3,1] + trans_points[4,1]) // 2 - MOUTH_H / 2);
crop_y_end = crop_y + MOUTH_H;
crop_x = int((trans_points[3,0] + trans_points[4,0]) // 2 - MOUTH_W / 2);
crop_x_end = crop_x + MOUTH_W;
month_crop[...] = img_resize[crop_y:crop_y_end,crop_x:crop_x_end,:];
-
-
+ ##########################################################
nose_crop = np.zeros([NOSE_H,NOSE_W,3], dtype=np.float32);
-
- #pose specific crop for MultiPIE. But it doesn't affect much as long as you train and test with a consistant crop configuration.
- # A general crop for nose is provicded below as comment.
- pos_s = filename[-18:-15]
- if label or pos_s == '051':#frontal
- crop_y_end = trans_points[2,1] + 10 + 1;
- crop_x = trans_points[2,0] - 20 + 1;
- elif pos_s == '050':#+15 degrees
- crop_y_end = trans_points[2,1] + 10 + 1;
- crop_x = trans_points[2,0] - 25 + 1;
- elif pos_s == '140':#-15 degrees
- crop_y_end = trans_points[2,1] + 10 + 1;
- crop_x = trans_points[2,0] - (NOSE_W-25) + 1;
- elif pos_s == '041' or pos_s == '190':#+30/45 degrees
- crop_y_end = trans_points[2,1] + 10 + 1;
- crop_x = trans_points[2,0] - 30 + 1;
- elif pos_s == '130' or pos_s == '080':#-30/45 degrees
- crop_y_end = trans_points[2,1] + 10 + 1;
- crop_x = trans_points[2,0] - (NOSE_W-30) + 1;
- elif pos_s == '010' or pos_s =='200' or pos_s == '240':#+60/75/90 degrees
- crop_y_end = trans_points[2,1] + 10 + 1;
- crop_x = trans_points[2,0] - 32 + 1;
- elif pos_s == '120' or pos_s == '090' or pos_s == '110': #-60/75/90 degrees
- crop_y_end = trans_points[2,1] + 10 + 1;
- crop_x = trans_points[2,0] - (NOSE_W-32) + 1;
- else:
- print("BUG from nose selection!")
-
- #Or you can just use this general position for nose crop, that works for most poses.
- # crop_y_end = trans_points[2,1] + 10 + 1
- # crop_x = trans_points[2,0] - 20 + 1
- # crop_y_end = int(crop_y_end)
- # crop_x = int(crop_x)
- # crop_y = crop_y_end - NOSE_H;
- # crop_x_end = crop_x + NOSE_W;
-
-
crop_y_end = int(crop_y_end)
crop_x = int(crop_x)
crop_y = crop_y_end - NOSE_H;
@@ -433,7 +343,7 @@ def GetFeatureParts(self, img_resize, filename, label=False):
#import pdb; pdb.set_trace()
nose_crop[...] = img_resize[crop_y:crop_y_end,crop_x:crop_x_end,:];
- if not label and self.MIRROR_TO_ONE_SIDE and self.findPose(filename) < 0:
+ if not label and self.MIRROR_TO_ONE_SIDE:
teml = eyel_crop[:,::-1,:]
eyel_crop = eyer_crop[:,::-1,:]
eyer_crop = teml
@@ -447,5 +357,5 @@ def GetFeatureParts(self, img_resize, filename, label=False):
teml = trans_points[3,:].copy()
trans_points[3, :] = trans_points[4, :]
trans_points[4, :] = teml
-
+
return trans_points, eyel_crop, eyer_crop, nose_crop, month_crop
diff --git a/net_input_everything_featparts_0_12.py b/net_input_everything_featparts_0_12.py
new file mode 100644
index 0000000..a6004b4
--- /dev/null
+++ b/net_input_everything_featparts_0_12.py
@@ -0,0 +1,361 @@
+# Copyright 2015 The TensorFlow Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ==============================================================================
+
+"""Routine for decoding the CIFAR-10 binary file format."""
+
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+
+import os
+
+from six.moves import xrange # pylint: disable=redefined-builtin
+import tensorflow as tf
+import numpy as np
+from PIL import Image
+import random
+import re
+import csv
+# Process images of this size. Note that this differs from the original CIFAR
+# image size of 32 x 32. If one alters this number, then the entire model
+# architecture will change and any model would need to be retrained.
+IMAGE_SIZE = 128
+# Global constants describing the CIFAR-10 data set.
+EYE_H = 40; EYE_W = 40;
+NOSE_H = 32; NOSE_W = 40;
+MOUTH_H = 32; MOUTH_W = 48;
+re_pose = re.compile('_\d{3}_')
+re_poseIllum = re.compile('_\d{3}_\d{2}_')
+#眼嘴鼻的高宽都是固定的,如上
+
+class MultiPIE():
+ """Reads and parses examples from MultiPIE data filelist
+ """
+ def __init__(self, datasplit='train', Random=True, LOAD_60_LABEL=False, MIRROR_TO_ONE_SIDE=True, RANDOM_VERIFY=False,
+ GENERATE_MASK=False, source='without90', testing = False):
+ self.dir = '/home/ubuntu3000/pt/TP-GAN/data/45/'#图片文件夹
+ self.testing = testing
+
+ self.split = datasplit
+ self.random = Random
+ self.seed = None
+ self.LOAD_60_LABEL = LOAD_60_LABEL
+ self.MIRROR_TO_ONE_SIDE = MIRROR_TO_ONE_SIDE
+ self.RANDOM_VERIFY = RANDOM_VERIFY
+ self.GENERATE_MASK = GENERATE_MASK
+ if not testing:
+ split_f = '/home/ubuntu3000/pt/TP-GAN/data/train.csv'
+ split_f_test = '/home/ubuntu3000/pt/TP-GAN/data/test.csv'
+ #self.indices是图片的名称大全,原本是csv文件里读取的
+ self.indices = open(split_f, 'r').read().splitlines()
+ self.indices_test = open(split_f_test, 'r').read().splitlines()
+ self.size = len(self.indices)
+ self.test_size = len(self.indices_test)
+ # make eval deterministic
+ if 'train' not in self.split:
+ self.random = False
+ # randomization: seed and pick
+ if self.random:
+ random.seed(self.seed)
+ self.idx = random.randint(0, len(self.indices)-1)
+ else:#only load test images for a separate list file.
+ split_f_test = '/home/ubuntu3000/pt/TP-GAN/data/test.csv'
+ self.indices_test = open(split_f_test, 'r').read().splitlines()
+ self.size = 0
+ self.test_size = len(self.indices_test)
+ self.idx = 0
+
+
+ def test_batch(self, test_batch_size=100,Random = True, Pose = -1):
+ test_batch_size = min(test_batch_size, len(self.indices_test))
+ images = np.empty([test_batch_size, IMAGE_SIZE, IMAGE_SIZE, 3])
+ eyel = np.empty([test_batch_size, EYE_H, EYE_W, 3], dtype=np.float32)
+ eyer = np.empty([test_batch_size, EYE_H, EYE_W, 3], dtype=np.float32)
+ nose = np.empty([test_batch_size, NOSE_H, NOSE_W, 3], dtype=np.float32)
+ mouth = np.empty([test_batch_size, MOUTH_H, MOUTH_W, 3],dtype=np.float32)
+ if not self.testing:
+ idenlabels = np.empty([test_batch_size], dtype=np.int32)
+ leyel = np.empty([test_batch_size, EYE_H, EYE_W, 3], dtype=np.float32)
+ leyer = np.empty([test_batch_size, EYE_H, EYE_W, 3], dtype=np.float32)
+ lnose = np.empty([test_batch_size, NOSE_H, NOSE_W, 3], dtype=np.float32)
+ lmouth = np.empty([test_batch_size, MOUTH_H, MOUTH_W, 3],dtype=np.float32)
+ labels = np.empty([test_batch_size, IMAGE_SIZE, IMAGE_SIZE, 3])
+ filenames = list()
+ if Random:
+ random.seed(2017)#make testing batch deterministic
+ random.shuffle(self.indices_test)
+ #resume randomeness for training
+ random.seed(self.seed)
+ #这个是什么?LOAD_60_LABEL,TODO:如果LOAD_60_LABEL，则跳过小于45度的图片；Pose为指定角度
+ j = 0
+ for i in range(test_batch_size):
+ print(j, end=' ')
+ images[i, ...], feats = self.load_image(self.indices_test[j % len(self.indices_test)])
+ eyel[i,...] = feats[1]
+ eyer[i,...] = feats[2]
+ nose[i,...] = feats[3]
+ mouth[i, ...] = feats[4]
+ filename = self.indices_test[j % len(self.indices_test)]
+ filenames.append(filename)
+ if not self.testing:
+ labels[i,...], _, leyel[i,...], leyer[i,...], lnose[i,...], lmouth[i, ...] = self.load_label_mask(filename)
+ identity = int(filename[0:3])
+ idenlabels[i] = identity
+ j += 1
+ print('\n')
+ if not self.testing:
+ #labels应该是正面的图片,leyel是label_eye_left
+ return images, filenames, eyel, eyer, nose, mouth,\
+ labels, leyel, leyer, lnose, lmouth, idenlabels
+ else:
+ return images, filenames, eyel, eyer, nose, mouth, None, None, None, None, None, None
+
+ def next_image_and_label_mask_batch(self, batch_size, imageRange=-1,imageRangeLow = 0, labelnum=None):
+ """Construct a batch of images and labels masks.
+
+ Args:
+ batch_size: Number of images per batch.
+ shuffle: boolean indicating whether to use a shuffling queue.
+ Returns:
+ ndarray feed.
+ images: Images. 4D of [batch_size, height, width, 6] size.
+ labels: Labels. 4D of [batch_size, height, width, 3] size.
+ masks: masks. 4D of [batch_size, height, width, 3] size.
+ verifyImages: Images. 4D of [batch_size, height, width, 3] size.
+ verifyLabels: 1D of [batch_size] 0 / 1 classification label
+ """
+ assert batch_size >= 1
+ images = np.empty([batch_size, IMAGE_SIZE, IMAGE_SIZE, 3])
+ labels = np.empty([batch_size, IMAGE_SIZE, IMAGE_SIZE, 3])
+
+ poselabels = np.empty([batch_size],dtype=np.int32)
+ idenlabels = np.empty([batch_size],dtype=np.int32)
+ landmarklabels = np.empty([batch_size, 5*2],dtype=np.float32)
+ eyel = np.empty([batch_size, EYE_H, EYE_W, 3], dtype=np.float32)
+ eyer = np.empty([batch_size, EYE_H, EYE_W, 3], dtype=np.float32)
+ nose = np.empty([batch_size, NOSE_H, NOSE_W, 3], dtype=np.float32)
+ mouth = np.empty([batch_size, MOUTH_H, MOUTH_W, 3],dtype=np.float32)
+ leyel = np.empty([batch_size, EYE_H, EYE_W, 3], dtype=np.float32)
+ leyer = np.empty([batch_size, EYE_H, EYE_W, 3], dtype=np.float32)
+ lnose = np.empty([batch_size, NOSE_H, NOSE_W, 3], dtype=np.float32)
+ lmouth = np.empty([batch_size, MOUTH_H, MOUTH_W, 3],dtype=np.float32)
+
+
+ masks = None
+ if self.RANDOM_VERIFY:
+ verifyImages = np.empty([batch_size, IMAGE_SIZE, IMAGE_SIZE, 3])
+ verifyLabels = np.empty([batch_size], dtype=np.int32)
+ else:
+ verifyImages = None; verifyLabels = None
+
+ for i in range(batch_size):
+ if imageRange != -1:
+ if True:
+ self.updateidx()
+ images[i, ...], feats = self.load_image(self.indices[self.idx])
+ filename = self.indices[self.idx]
+ labels[i,...], _, leyel[i,...], leyer[i,...], lnose[i,...], lmouth[i, ...] = self.load_label_mask(filename)
+
+ pose = abs(self.findPose(filename))
+ poselabels[i] = int(pose/15)
+ identity = int(filename[0:3])
+ idenlabels[i] = identity
+ landmarklabels[i,:] = feats[0].flatten()
+ eyel[i,...] = feats[1]
+ eyer[i,...] = feats[2]
+ nose[i,...] = feats[3]
+ mouth[i, ...] = feats[4]
+ self.updateidx()
+ #labels是什么,masks是什么,verifyImages和images区别,poselabels是位置角度吗
+ return images, labels, masks, verifyImages, verifyLabels, poselabels, idenlabels, landmarklabels,\
+ eyel, eyer, nose, mouth, leyel, leyer, lnose, lmouth
+
+ def updateidx(self):
+ if self.random:
+ self.idx = random.randint(0, len(self.indices)-1)
+ else:
+ self.idx += 1
+ if self.idx == len(self.indices):
+ self.idx = 0
+ def load_image(self, filename):
+ #读取一个图片
+ """
+ Load input image & codemap and preprocess:
+ - cast to float
+ - subtract mean divide stdadv
+ - concatenate together
+ """
+ im = Image.open(self.dir + filename)
+ in_ = np.array(im, dtype=np.float32)
+ in_ /= 256
+ features = self.GetFeatureParts(in_, filename)
+ return in_, features
+
+ #训练使用,输入图片的路径,返回侧面的照片,并用GetFeatureParts得到它的各个部位并返回
+ def load_label_mask(self, filename, labelnum=-1):
+ _, labelname = self.findSameIllumCodeLabelpath(filename)
+ #返回图片路径
+ im = Image.open(self.dir + labelname)
+ if self.MIRROR_TO_ONE_SIDE:
+ im = im.transpose(Image.FLIP_LEFT_RIGHT)
+ mask = None #pt
+ label = np.array(im, dtype=np.float32)
+ label /= 256
+ feats = self.GetFeatureParts(label, labelname, label=True)
+ if not self.LOAD_60_LABEL:
+ #print("fipping!")
+ label = label[:,::-1,:]
+ feats[1][...] = feats[1][:,::-1,:]
+ feats[2][...] = feats[2][:,::-1,:]
+ feats[3][...] = feats[3][:,::-1,:]
+ feats[4][...] = feats[4][:,::-1,:]
+ return label, mask, feats[2], feats[1], feats[3], feats[4]
+ #print("not flipping!")
+ return label, mask, feats[1], feats[2], feats[3], feats[4]
+ #use coler code to generate mask
+ #background area weights 0.2, face area weights 1.0
+
+ return label, mask, feats
+
+ label, mask = self.load_label_mask(filename, labelnum)
+
+ #if(random.random() > 0.5):#positive
+ if self.RANDOM_VERIFY:
+ if True:
+ return label, mask, label, int(filename[0:3])
+ else:
+ randomSubjectPath = self.indices[random.randint(0, len(self.indices)-1)]
+ _, veryPath = self.findCodeLabelpath(randomSubjectPath)
+ veryIm = Image.open(self.codeLabelDir + veryPath)
+ veryImArray = np.array(veryIm, dtype=np.float32)
+ veryImArray /= 256
+ #veryImArray -= 1
+ return label, mask, veryImArray, int(randomSubjectPath[0:3])
+ else:
+ return label, mask, None, None
+ #输入图片路径，寻找那个角度的最好的光照条件的图片编号
+ span = re_pose.search(fullpath).span()
+ camPos = list(fullpath[span[0]+1:span[1]-1])
+ camPos.insert(2,'_')
+ camPos = ''.join(camPos)
+ #get 01_0 like string
+ bestIllum = self.cameraPositions[camPos][1]
+ #bestIllum什么鬼。。。日
+ labelpath = list(fullpath)
+ #bestIllum是一个字符串。。加:索引干嘛
+ labelpath[span[1]:span[1]+2] = bestIllum[:]
+ labelpath = ''.join(labelpath)
+ codepath = str(labelpath).replace('cropped', 'code')
+ return (codepath, labelpath)
+
+ #两个参数，第一个地址，什么的地址？第二个是标签的序号？？
+ span = re_poseIllum.search(fullpath).span()
+ #print span
+ #camPosIllu =fullpath[span[0]+1:span[1]-1]
+ #print camPosIllu
+ #labelpath = fullpath.replace(camPosIllu, '051_06')
+ tempath = list(fullpath)
+ if self.LOAD_60_LABEL:
+ camPos = fullpath[span[0]+1:span[0]+4]
+ if(camPos == '240' or camPos == '010'): #+90/75
+ tempath[span[0]+1:span[1]-1] = '200_08' #+60
+ elif (camPos == '120' or camPos == '110'): #-90/75
+ tempath[span[0]+1:span[1]-1] = '090_15' #-60
+ else:
+ tempath[span[0]+1:span[1]-1] = '051_06'
+ else:
+ tempath[span[0]+1:span[1]-1] = '051_06'
+ labelpath = ''.join(tempath)
+ codepath = str(labelpath).replace('cropped', 'code')
+ if labelnum != -1:
+ replace = None
+ for i in self.cameraPositions.items():
+ if i[1][0] == labelnum:
+ replace = ''.join([i[0][0:2],i[0][3],'_',i[1][1]])
+ tempath[span[0]+1:span[1]-1] = replace
+ labelpath = ''.join(tempath)
+ if replace == None:
+ print('damn labelnum bug!')
+ return (codepath, labelpath)
+
+ #input 图片路径,return 监督图片,TODO:这个要改
+ def findSameIllumCodeLabelpath(self, fullpath):
+ #输入路径，提取路径中某个东西
+ labelpath = fullpath.replace('cropped','cropped_test')
+ codepath='_'
+ return (codepath, labelpath)
+
+ #这个函数意思是
+ def findPose(self, fullpath):
+ return +60
+
+ ##输入img_resize是128x128的Image.open,filename是用来寻找标注文件;返回trans_points和图片各个部位的裁剪
+ def GetFeatureParts(self, img_resize, filename, label=False):
+ #crop four parts
+ trans_points = np.empty([5,2],dtype=np.int32)
+ if True:
+ featpath = os.path.join('/home/ubuntu3000/pt/TP-GAN/data/45_5pt',filename.replace('.png','.5pt'))
+ with open(featpath, 'r') as csvfile:
+ reader = csv.reader(csvfile, delimiter=' ')
+ for ind,row in enumerate(reader):
+ if not ind >=5:
+ trans_points[ind,:] = row
+
+ eyel_crop = np.zeros([EYE_H,EYE_W,3], dtype=np.float32);
+ crop_y = int(trans_points[0,1] - EYE_H / 2);
+ crop_y_end = crop_y + EYE_H;
+ crop_x = int(trans_points[0,0] - EYE_W / 2);
+ crop_x_end = crop_x + EYE_W;
+
+ eyel_crop[...] = img_resize[crop_y:crop_y_end,crop_x:crop_x_end,:];
+ ##########################################################
+ eyer_crop = np.zeros([EYE_H,EYE_W,3], dtype=np.float32);
+ crop_y = int(trans_points[1,1] - EYE_H / 2)
+ crop_y_end = crop_y + EYE_H;
+ crop_x = int(trans_points[1,0] - EYE_W / 2);
+ crop_x_end = crop_x + EYE_W;
+ eyer_crop[...] = img_resize[crop_y:crop_y_end,crop_x:crop_x_end,:];
+ #####################################################
+ month_crop = np.zeros([MOUTH_H,MOUTH_W,3], dtype=np.float32);
+ crop_y = int((trans_points[3,1] + trans_points[4,1]) // 2 - MOUTH_H / 2);
+ crop_y_end = crop_y + MOUTH_H;
+ crop_x = int((trans_points[3,0] + trans_points[4,0]) // 2 - MOUTH_W / 2);
+ crop_x_end = crop_x + MOUTH_W;
+ month_crop[...] = img_resize[crop_y:crop_y_end,crop_x:crop_x_end,:];
+ ##########################################################
+ nose_crop = np.zeros([NOSE_H,NOSE_W,3], dtype=np.float32);
+ crop_y_end = int(crop_y_end)
+ crop_x = int(crop_x)
+ crop_y = crop_y_end - NOSE_H;
+ crop_x_end = crop_x + NOSE_W;
+ #import pdb; pdb.set_trace()
+ nose_crop[...] = img_resize[crop_y:crop_y_end,crop_x:crop_x_end,:];
+
+ if not label and self.MIRROR_TO_ONE_SIDE:
+ teml = eyel_crop[:,::-1,:]
+ eyel_crop = eyer_crop[:,::-1,:]
+ eyer_crop = teml
+ month_crop = month_crop[:,::-1,:]
+ nose_crop = nose_crop[:,::-1,:]
+ trans_points[:,0] = IMAGE_SIZE - trans_points[:,0]
+ #exchange eyes and months
+ teml = trans_points[0,:].copy()
+ trans_points[0, :] = trans_points[1, :]
+ trans_points[1, :] = teml
+ teml = trans_points[3,:].copy()
+ trans_points[3, :] = trans_points[4, :]
+ trans_points[4, :] = teml
+
+ return trans_points, eyel_crop, eyer_crop, nose_crop, month_crop
diff --git a/pt_test_net_input_everything_featparts_.py b/pt_test_net_input_everything_featparts_.py
new file mode 100644
index 0000000..2c46443
--- /dev/null
+++ b/pt_test_net_input_everything_featparts_.py
@@ -0,0 +1,207 @@
+# Copyright 2015 The TensorFlow Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ==============================================================================
+
+"""Routine for decoding the CIFAR-10 binary file format."""
+
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+
+import os
+
+from six.moves import xrange # pylint: disable=redefined-builtin
+import tensorflow as tf
+import numpy as np
+from PIL import Image
+import random
+import re
+import csv
+# Process images of this size. Note that this differs from the original CIFAR
+# image size of 32 x 32. If one alters this number, then the entire model
+# architecture will change and any model would need to be retrained.
+IMAGE_SIZE = 128
+# Global constants describing the CIFAR-10 data set.
+EYE_H = 40; EYE_W = 40;
+NOSE_H = 32; NOSE_W = 40;
+MOUTH_H = 32; MOUTH_W = 48;
+re_pose = re.compile('_\d{3}_')
+re_poseIllum = re.compile('_\d{3}_\d{2}_')
+#眼嘴鼻的高宽都是固定的,如上
+
+class MultiPIE():
+ """Reads and parses examples from MultiPIE data filelist
+ """
+ def __init__(self, datasplit='train', Random=True, LOAD_60_LABEL=False, MIRROR_TO_ONE_SIDE=True, RANDOM_VERIFY=False,
+ GENERATE_MASK=False, source='without90', testing = False):
+ self.testing = testing
+ self.split = datasplit
+ self.random = Random
+ self.seed = None
+ self.LOAD_60_LABEL = LOAD_60_LABEL
+ self.MIRROR_TO_ONE_SIDE = MIRROR_TO_ONE_SIDE
+ self.RANDOM_VERIFY = RANDOM_VERIFY
+ self.GENERATE_MASK = GENERATE_MASK
+ if not testing:
+ #split_f = '/home/TP-GAN/data/train.csv'
+ #split_f_test = '/home/TP-GAN/data/test.csv'
+ #self.indices是图片的名称大全,原本是csv文件里读取的
+ #self.indices = open(split_f, 'r').read().splitlines()
+ #self.indices_test = open(split_f_test, 'r').read().splitlines()
+ #self.size = len(self.indices)
+ #self.test_size = len(self.indices_test)
+ # make eval deterministic
+ if 'train' not in self.split:
+ self.random = False
+
+ else:#only load test images for a separate list file.
+ split_f_test = '/home/TP-GAN/data/test.csv'
+ self.indices_test = open(split_f_test, 'r').read().splitlines()
+ self.size = 0
+ self.test_size = len(self.indices_test)
+ self.idx = 0
+
+
+ def test_batch(self, test_batch_size=100,Random = True, Pose = -1):
+ test_batch_size = min(test_batch_size, len(self.indices_test))
+ images = np.empty([test_batch_size, IMAGE_SIZE, IMAGE_SIZE, 3])
+ eyel = np.empty([test_batch_size, EYE_H, EYE_W, 3], dtype=np.float32)
+ eyer = np.empty([test_batch_size, EYE_H, EYE_W, 3], dtype=np.float32)
+ nose = np.empty([test_batch_size, NOSE_H, NOSE_W, 3], dtype=np.float32)
+ mouth = np.empty([test_batch_size, MOUTH_H, MOUTH_W, 3],dtype=np.float32)
+ if not self.testing:
+ idenlabels = np.empty([test_batch_size], dtype=np.int32)
+ leyel = np.empty([test_batch_size, EYE_H, EYE_W, 3], dtype=np.float32)
+ leyer = np.empty([test_batch_size, EYE_H, EYE_W, 3], dtype=np.float32)
+ lnose = np.empty([test_batch_size, NOSE_H, NOSE_W, 3], dtype=np.float32)
+ lmouth = np.empty([test_batch_size, MOUTH_H, MOUTH_W, 3],dtype=np.float32)
+ labels = np.empty([test_batch_size, IMAGE_SIZE, IMAGE_SIZE, 3])
+ filenames = list()
+ if Random:
+ random.seed(2017)#make testing batch deterministic
+ random.shuffle(self.indices_test)
+ #resume randomeness for training
+ random.seed(self.seed)
+ #这个是什么?LOAD_60_LABEL,TODO:如果LOAD_60_LABEL，则跳过小于45度的图片；Pose为指定角度
+ j = 0
+ for i in range(test_batch_size):
+ print(j, end=' ')
+ images[i, ...], feats = self.load_image(self.indices_test[j % len(self.indices_test)])
+ eyel[i,...] = feats[1]
+ eyer[i,...] = feats[2]
+ nose[i,...] = feats[3]
+ mouth[i, ...] = feats[4]
+ filename = self.indices_test[j % len(self.indices_test)]
+ filenames.append(filename)
+ if not self.testing:
+ labels[i,...], _, leyel[i,...], leyer[i,...], lnose[i,...], lmouth[i, ...] = self.load_label_mask(filename)
+ identity = int(filename[0:3])
+ idenlabels[i] = identity
+ j += 1
+ print('\n')
+ if not self.testing:
+ #labels应该是正面的图片,leyel是label_eye_left
+ return images, filenames, eyel, eyer, nose, mouth,\
+ labels, leyel, leyer, lnose, lmouth, idenlabels
+ else:
+ return images, filenames, eyel, eyer, nose, mouth, None, None, None, None, None, None
+
+
+
+ def load_image(self, filename):
+ #读取一个图片
+ """
+ Load input image & codemap and preprocess:
+ - cast to float
+ - subtract mean divide stdadv
+ - concatenate together
+ """
+ im = Image.open(filename)
+ in_ = np.array(im, dtype=np.float32)
+ in_ /= 256
+ features = self.GetFeatureParts(in_, filename)
+ return in_, features
+
+
+ #input 图片路径,return 监督图片,TODO:这个要改
+ def findSameIllumCodeLabelpath(self, fullpath):
+ #输入路径，提取路径中某个东西
+ labelpath = fullpath.replace('cropped','cropped_test')
+ codepath='_'
+ return (codepath, labelpath)
+
+ #这个函数意思是
+ def findPose(self, fullpath):
+ return +60
+
+ ##输入img_resize是128x128的Image.open,filename是用来寻找标注文件;返回trans_points和图片各个部位的裁剪
+ def GetFeatureParts(self, img_resize, filename, label=False):
+ #crop four parts
+ trans_points = np.empty([5,2],dtype=np.int32)
+ #把这里改成mtcnn
+ if True:
+ path_5pt='./test_tem.5pt'
+ featpath = path_5pt
+ with open(featpath, 'r') as csvfile:
+ reader = csv.reader(csvfile, delimiter=' ')
+ for ind,row in enumerate(reader):
+ if not ind >=5:
+ trans_points[ind,:] = row
+ #把这里改成mtcnn
+ eyel_crop = np.zeros([EYE_H,EYE_W,3], dtype=np.float32);
+ crop_y = int(trans_points[0,1] - EYE_H / 2);
+ crop_y_end = crop_y + EYE_H;
+ crop_x = int(trans_points[0,0] - EYE_W / 2);
+ crop_x_end = crop_x + EYE_W;
+
+ eyel_crop[...] = img_resize[crop_y:crop_y_end,crop_x:crop_x_end,:];
+ ##########################################################
+ eyer_crop = np.zeros([EYE_H,EYE_W,3], dtype=np.float32);
+ crop_y = int(trans_points[1,1] - EYE_H / 2)
+ crop_y_end = crop_y + EYE_H;
+ crop_x = int(trans_points[1,0] - EYE_W / 2);
+ crop_x_end = crop_x + EYE_W;
+ eyer_crop[...] = img_resize[crop_y:crop_y_end,crop_x:crop_x_end,:];
+ #####################################################
+ month_crop = np.zeros([MOUTH_H,MOUTH_W,3], dtype=np.float32);
+ crop_y = int((trans_points[3,1] + trans_points[4,1]) // 2 - MOUTH_H / 2);
+ crop_y_end = crop_y + MOUTH_H;
+ crop_x = int((trans_points[3,0] + trans_points[4,0]) // 2 - MOUTH_W / 2);
+ crop_x_end = crop_x + MOUTH_W;
+ month_crop[...] = img_resize[crop_y:crop_y_end,crop_x:crop_x_end,:];
+ ##########################################################
+ nose_crop = np.zeros([NOSE_H,NOSE_W,3], dtype=np.float32);
+ crop_y_end = int(crop_y_end)
+ crop_x = int(crop_x)
+ crop_y = crop_y_end - NOSE_H;
+ crop_x_end = crop_x + NOSE_W;
+ #import pdb; pdb.set_trace()
+ nose_crop[...] = img_resize[crop_y:crop_y_end,crop_x:crop_x_end,:];
+
+ if not label and self.MIRROR_TO_ONE_SIDE:
+ teml = eyel_crop[:,::-1,:]
+ eyel_crop = eyer_crop[:,::-1,:]
+ eyer_crop = teml
+ month_crop = month_crop[:,::-1,:]
+ nose_crop = nose_crop[:,::-1,:]
+ trans_points[:,0] = IMAGE_SIZE - trans_points[:,0]
+ #exchange eyes and months
+ teml = trans_points[0,:].copy()
+ trans_points[0, :] = trans_points[1, :]
+ trans_points[1, :] = teml
+ teml = trans_points[3,:].copy()
+ trans_points[3, :] = trans_points[4, :]
+ trans_points[4, :] = teml
+
+ return trans_points, eyel_crop, eyer_crop, nose_crop, month_crop
diff --git a/test_pt.py b/test_pt.py
new file mode 100644
index 0000000..abdab35
--- /dev/null
+++ b/test_pt.py
@@ -0,0 +1,912 @@
+from __future__ import division
+from __future__ import print_function
+import os,sys
+import time
+from glob import glob
+import tensorflow as tf
+import numpy as np
+from six.moves import xrange
+
+from ops import *
+from utils import *
+
+from pt_test_net_input_everything_featparts_ import *
+from time import localtime, strftime
+import random
+import pickle
+import subprocess
+relu = tf.nn.relu
+import scipy.misc
+import shutil
+from utils import pp, visualize, to_json
+
+import tensorflow as tf
+import logging#pt
+import imageio
+#These parameters should provide a good initialization, but if for specific refinement, you can adjust them during training.
+
+ALPHA_ADVER = 2e1
+BELTA_FEATURE = 4e1 #3800 inital loss
+UPDATE_G = 2 #optimize D once and UPDATE_G times G
+UPDATE_D = 1
+PART_W = 3
+IDEN_W = 1e1
+TV_WEIGHT = 1e-3
+COND_WEIGHT = 0.3
+L1_1_W = 1#0.5
+L1_2_W = 1
+L1_3_W = 1.5
+RANGE= 60
+RANGE_LOW = 0
+SYM_W = 3e-1
+CLIP_D = 0.1
+L1 = True
+MODE = 'fs60' #'f' feature loss enabled. 'v' -verification enanbled. 'o' original, 'm' masked is mandatory and no need to specify
+UPDATE_DV = 1 #optimize DV net
+DF = True #local discriminator 4x4
+LOAD_60_LABEL = False #otherwise load frontal label
+WITHOUT_CODEMAP = True
+USE_MASK = False
+ENABLE_SELECTION = True
+RANDOM_VERIFY = False
+RANK_MUL = 6
+if WITHOUT_CODEMAP:
+ CHANNEL = 3
+else:
+ CHANNEL = 6
+
+flags = tf.app.flags
+flags.DEFINE_integer("epoch", 250, "Epoch to train [25]")
+flags.DEFINE_float("learning_rate", 1e-4, "Learning rate of for adam [0.0002]")
+flags.DEFINE_float("beta1", 0.9, "Momentum term of adam [0.5]")
+flags.DEFINE_integer("train_size", 1156, "The size of train images [np.inf]")
+flags.DEFINE_integer("batch_size", 4, "The size of batch images [64]")
+flags.DEFINE_integer("image_size", 128, "The size of image to use (will be center cropped) [108]")
+flags.DEFINE_integer("output_size", 128, "The size of the output images to produce [64]")
+flags.DEFINE_integer("c_dim", 3, "Dimension of image color. [3]")
+flags.DEFINE_string("dataset", "MultiPIE", "The name of dataset [celebA, mnist, lsun]")
+flags.DEFINE_string("checkpoint_dir", "/home/ubuntu3000/pt/TP-GAN/checkpoint60/MultiPIE_2_128", "Directory name to save the checkpoints [checkpoint]")
+flags.DEFINE_string("sample_dir", "samples", "Directory name to save the image samples [samples]")
+flags.DEFINE_boolean("is_train", True, "True for training, False for testing [False]")
+flags.DEFINE_boolean("is_crop", True, "True for training, False for testing [False]")
+flags.DEFINE_boolean("visualize", True, "True for visualizing, False for nothing [False]")
+flags.DEFINE_integer("test_batch_size", 1, "test batch size. [1]")
+FLAGS = flags.FLAGS
+
+class DCGAN(object):
+ def __init__(self, sess, image_size=128, is_crop=True,
+ batch_size=10, sample_size = 100, output_size=128,
+ y_dim=None, z_dim=100, gf_dim=64, df_dim=64,
+ gfc_dim=1024, dfc_dim=1024, c_dim=3, dataset_name='MultiPIE',
+ checkpoint_dir=None, sample_dir=None):
+ """
+ Args:
+ sess: TensorFlow session
+ batch_size: The size of batch. Should be specified before training.
+ output_size: (optional) The resolution in pixels of the images. [64]
+ y_dim: (optional) Dimension of dim for y. [None]
+ z_dim: (optional) Dimension of dim for Z. [100]
+ gf_dim: (optional) Dimension of gen filters in first conv layer. [64]
+ df_dim: (optional) Dimension of discrim filters in first conv layer. [64]
+ gfc_dim: (optional) Dimension of gen units for for fully connected layer. [1024]
+ dfc_dim: (optional) Dimension of discrim units for fully connected layer. [1024]
+ c_dim: (optional) Dimension of image color. For grayscale input, set to 1. [3]
+ """
+ self.save_interval = 500
+ self.sample_interval = 150
+ self.sess = sess
+ self.is_grayscale = (c_dim == 1)
+ self.batch_size = FLAGS.batch_size
+ self.sample_run_num = 15
+ self.testing = False
+ self.testingphase = 'FS'
+ self.testimg = True
+ if self.testing:
+ self.testingphase = '60'#'gt50'
+ self.sample_run_num = 99999999
+
+ self.test_batch_size = 1
+ self.image_size = image_size
+ self.sample_size = sample_size
+ self.output_size = output_size
+
+ self.gf_dim = gf_dim
+ self.df_dim = df_dim
+
+ self.gfc_dim = gfc_dim
+ self.dfc_dim = dfc_dim
+ self.z_dim = 100
+ self.c_dim = c_dim
+
+ # batch normalization : deals with poor initialization helps gradient flow
+ random.seed()
+ self.DeepFacePath = './DeepFace168.pickle'
+ self.dataset_name = dataset_name
+ self.checkpoint_dir = checkpoint_dir
+ self.loadDeepFace()#pt
+ self.build_model()#pt
+
+
+
+ def build_model(self):
+
+ #hold all four
+ #Note: not true, if WITHOUT_CODEMAP is true, then here is pure images without codemap and 3 channels
+ #mirror concatenate
+ mc = lambda left : tf.concat_v2([left, left[:,:,::-1,:]], 3)
+ self.images_with_code = tf.placeholder(tf.float32, [self.test_batch_size] + [self.output_size, self.output_size, CHANNEL], name='images_with_code')
+ self.sample_images = tf.placeholder(tf.float32, [self.test_batch_size] + [self.output_size, self.output_size, CHANNEL], name='sample_images')
+
+ if WITHOUT_CODEMAP:
+ self.images = self.images_with_code
+ self.sample_images_nocode = self.sample_images
+ else:
+ self.images = tf.split(3, 2, self.images_with_code)[0]
+ self.sample_images_nocode = tf.split(3, 2, self.sample_images)[0]
+
+ self.g_images = self.images #tf.reduce_mean(self.images, axis=3, keep_dims=True)
+ self.g_samples = self.sample_images_nocode #tf.reduce_mean(self.sample_images_nocode, axis=3, keep_dims=True)
+
+ self.g32_images_with_code = tf.image.resize_bilinear(self.images_with_code, [32, 32])
+ self.g64_images_with_code = tf.image.resize_bilinear(self.images_with_code, [64, 64])
+
+ self.g32_sampleimages_with_code = tf.image.resize_bilinear(self.sample_images, [32, 32])
+ self.g64_sampleimages_with_code = tf.image.resize_bilinear(self.sample_images, [64, 64])
+
+ self.labels = tf.placeholder(tf.float32, [self.test_batch_size] + [self.output_size, self.output_size, 3], name='label_images')
+ self.poselabels = tf.placeholder(tf.int32, [self.test_batch_size])
+ self.idenlabels = tf.placeholder(tf.int32, [self.test_batch_size])
+ self.landmarklabels = tf.placeholder(tf.float32, [self.test_batch_size, 5*2])
+ self.g_labels = self.labels #tf.reduce_mean(self.labels, 3, keep_dims=True)
+ self.g8_labels = tf.image.resize_bilinear(self.g_labels, [8, 8])
+ self.g16_labels = tf.image.resize_bilinear(self.g_labels, [16, 16])
+ self.g32_labels = tf.image.resize_bilinear(self.g_labels, [32, 32])
+ self.g64_labels = tf.image.resize_bilinear(self.g_labels, [64, 64])
+
+ self.eyel = tf.placeholder(tf.float32, [self.test_batch_size, EYE_H, EYE_W, 3])
+ self.eyer = tf.placeholder(tf.float32, [self.test_batch_size, EYE_H, EYE_W, 3])
+ self.nose = tf.placeholder(tf.float32, [self.test_batch_size, NOSE_H, NOSE_W, 3])
+ self.mouth = tf.placeholder(tf.float32, [self.test_batch_size, MOUTH_H, MOUTH_W, 3])
+
+ self.eyel_label = tf.placeholder(tf.float32, [self.test_batch_size, EYE_H, EYE_W, 3])
+ self.eyer_label = tf.placeholder(tf.float32, [self.test_batch_size, EYE_H, EYE_W, 3])
+ self.nose_label = tf.placeholder(tf.float32, [self.test_batch_size, NOSE_H, NOSE_W, 3])
+ self.mouth_label = tf.placeholder(tf.float32, [self.test_batch_size, MOUTH_H, MOUTH_W, 3])
+
+ self.eyel_sam = tf.placeholder(tf.float32, [self.test_batch_size, EYE_H, EYE_W, 3])
+ self.eyer_sam = tf.placeholder(tf.float32, [self.test_batch_size, EYE_H, EYE_W, 3])
+ self.nose_sam = tf.placeholder(tf.float32, [self.test_batch_size, NOSE_H, NOSE_W, 3])
+ self.mouth_sam = tf.placeholder(tf.float32, [self.test_batch_size, MOUTH_H, MOUTH_W, 3])
+
+
+ #feats contains: self.feat128, self.feat64, self.feat32, self.feat16, self.feat8, self.feat
+ self.G_eyel,self.c_eyel = self.partRotator(self.eyel, "PartRotator_eyel")
+ self.G_eyer,self.c_eyer = self.partRotator(tf.concat_v2([self.eyer, self.eyel], axis=3), "PartRotator_eyer")
+ self.G_nose,self.c_nose = self.partRotator(self.nose, "PartRotator_nose")
+ self.G_mouth,self.c_mouth = self.partRotator(self.mouth, "PartRotator_mouth")
+
+ self.G_eyel_sam, self.c_eyel_sam = self.partRotator(self.eyel_sam, "PartRotator_eyel", reuse=True)
+ self.G_eyer_sam, self.c_eyer_sam = self.partRotator(tf.concat_v2([self.eyer_sam, self.eyel_sam],axis=3), "PartRotator_eyer", reuse=True)
+ self.G_nose_sam, self.c_nose_sam = self.partRotator(self.nose_sam, "PartRotator_nose", reuse=True)
+ self.G_mouth_sam, self.c_mouth_sam = self.partRotator(self.mouth_sam, "PartRotator_mouth", reuse=True)
+
+ self.z = tf.random_normal([self.test_batch_size, self.z_dim], mean=0.0, stddev=0.02, seed=2017)
+
+
+
+ self.feats = self.generator(mc(self.images_with_code), self.test_batch_size, name="encoder")
+ self.feats += (mc(self.images_with_code), mc(self.g64_images_with_code), mc(self.g32_images_with_code),
+ self.G_eyel, self.G_eyer, self.G_nose, self.G_mouth,
+ self.c_eyel, self.c_eyer, self.c_nose, self.c_mouth,)
+ self.check_sel128, self.check_sel64, self.check_sel32, self.check_sel16, self.check_sel8, self.G, self.G2, self.G3 = \
+ self.decoder(*self.feats, batch_size=self.test_batch_size)
+ self.poselogits, self.identitylogits, self.Glandmark = self.FeaturePredict(self.feats[5])
+
+ sample_feats = self.generator(mc(self.sample_images),self.test_batch_size, name="encoder", reuse=True)
+ self.sample512 = sample_feats[-1]
+ sample_feats += (mc(self.sample_images), mc(self.g64_sampleimages_with_code), mc(self.g32_sampleimages_with_code),
+ self.G_eyel_sam, self.G_eyer_sam, self.G_nose_sam, self.G_mouth_sam,
+ self.c_eyel_sam, self.c_eyer_sam, self.c_nose_sam, self.c_mouth_sam,)
+ self.sample_generator = self.decoder(*sample_feats, batch_size=self.test_batch_size, reuse=True)
+ if not DF:
+ self.D, self.D_logits = self.discriminator(self.g_labels)
+ self.D_, self.D_logits_ = self.discriminator(self.G, reuse=True)
+ else:
+ print("Using local discriminator!")
+ self.D, self.D_logits = self.discriminatorLocal(self.g_labels)
+ self.D_, self.D_logits_ = self.discriminatorLocal(self.G, reuse=True)
+ self.logfile = 'loss.txt'
+
+ if 'f' in MODE:
+ #self.verify_images_masked = tf.mul(self.verify_images, self.masks_binary)
+ #can not apply mask !!!
+ # self.Dv, self.Dv_logits = self.discriminatorVerify(self.labels, self.verify_images)
+ _,_,_,_, self.G_pool5, self.Gvector = self.FeatureExtractDeepFace(tf.reduce_mean(self.G, axis=3, keep_dims=True))
+ _,_,_,_, self.label_pool5, self.labelvector = self.FeatureExtractDeepFace(tf.reduce_mean(self.g_labels, axis=3, keep_dims=True), reuse=True)
+ _,_,_,_, _, self.samplevector = self.FeatureExtractDeepFace(tf.reduce_mean(self.sample_images_nocode, axis=3, keep_dims=True), reuse=True)
+
+ #self.Dv, self.Dv_logits = self.discriminatorClassify(self.Gvector)
+ #self.dv_loss = tf.reduce_sum(tf.nn.sparse_softmax_cross_entropy_with_logits(self.Dv_logits, self.verify_labels))
+ self.dv_loss = tf.reduce_mean(tf.abs(self.Gvector-self.labelvector))
+ self.dv_loss += tf.reduce_mean(tf.reduce_sum(tf.reduce_sum(tf.abs(self.G_pool5-self.label_pool5),1),1))
+ self.logfile = 'loss_verify.txt'
+ #self.dv_sum = histogram_summary("dv_", self.Dv)
+
+
+ self.d_loss_real = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(self.D_logits, tf.ones_like(self.D) * 0.9))
+ self.d_loss_fake = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(self.D_logits_, tf.zeros_like(self.D_)))
+ self.g_loss_adver = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(self.D_logits_, tf.ones_like(self.D_) * 0.9))
+
+ #self.mark_regression_loss = tf.reduce_mean(tf.square(tf.abs(self.landmarklabels-self.Glandmark)))
+ #self.poseloss = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(self.poselogits, self.poselabels))
+ self.idenloss = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(self.identitylogits, self.idenlabels))
+
+ self.eyel_loss = tf.reduce_mean(tf.reduce_sum(tf.reduce_sum(tf.abs(self.c_eyel - self.eyel_label), 1), 1))
+ self.eyer_loss = tf.reduce_mean(tf.reduce_sum(tf.reduce_sum(tf.abs(self.c_eyer - self.eyer_label), 1), 1))
+ self.nose_loss = tf.reduce_mean(tf.reduce_sum(tf.reduce_sum(tf.abs(self.c_nose - self.nose_label), 1), 1))
+ self.mouth_loss = tf.reduce_mean(tf.reduce_sum(tf.reduce_sum(tf.abs(self.c_mouth - self.mouth_label), 1), 1))
+ #rotation L1 / L2 loss in g_loss
+
+ #use L2 for 128, L1 for others. mask emphasize left side.
+ errL1 = tf.abs(self.G - self.g_labels) #* mask128
+ errL1_2 = tf.abs(self.G2 - self.g64_labels) #* mask64
+ errL1_3 = tf.abs(self.G3 - self.g32_labels) #* mask32
+ #errcheck8 = tf.abs(self.check_sel8 - self.g8_labels) #* mask8
+ #errcheck16 = tf.abs(self.check_sel16 - self.g16_labels) #* mask16
+ errcheck32 = tf.abs(self.check_sel32 - self.g32_labels) #* mask32
+ errcheck64 = tf.abs(self.check_sel64 - self.g64_labels) #* mask64
+ errcheck128 = tf.abs(self.check_sel128 - self.g_labels) #* mask128
+ self.weightedErrL1 = tf.reduce_mean(tf.reduce_sum(tf.reduce_sum(errL1, 1), 1))
+ self.symErrL1 = tf.reduce_mean(tf.reduce_sum(tf.reduce_sum(symL1(#self.processor(self.G)
+ tf.nn.avg_pool(self.G, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')
+ ), 1), 1))
+ self.weightedErrL2 = tf.reduce_mean(tf.reduce_sum(tf.reduce_sum(errL1_2, 1), 1))
+ self.symErrL2 = tf.reduce_mean(tf.reduce_sum(tf.reduce_sum(symL1(self.processor(self.G2)), 1), 1))
+ self.weightedErrL3 = tf.reduce_mean(tf.reduce_sum(tf.reduce_sum(errL1_3, 1), 1))
+ self.symErrL3 = tf.reduce_mean(tf.reduce_sum(tf.reduce_sum(symL1(self.processor(self.G3, reuse=True)), 1), 1))
+ cond_L12 = tf.abs(tf.image.resize_bilinear(self.G, [64,64]) - tf.stop_gradient(self.G2))
+ #self.condErrL12 = tf.reduce_mean(tf.reduce_sum(tf.reduce_sum(cond_L12, 1), 1))
+ #cond_L23 = tf.abs(tf.image.resize_bilinear(self.G2, [32,32]) - tf.stop_gradient(self.G3))
+ #self.condErrL23 = tf.reduce_mean(tf.reduce_sum(tf.reduce_sum(cond_L23, 1), 1))
+ self.tv_loss = tf.reduce_mean(total_variation(self.G))
+
+ self.d_loss = self.d_loss_real + self.d_loss_fake
+ self.g_loss = L1_1_W * (self.weightedErrL1 + SYM_W * self.symErrL1) + L1_2_W * (self.weightedErrL2 + SYM_W * self.symErrL2) \
+ + L1_3_W * (self.weightedErrL3 + SYM_W * self.symErrL3)
+ self.g_loss += BELTA_FEATURE * self.dv_loss + ALPHA_ADVER * self.g_loss_adver + IDEN_W * self.idenloss + self.tv_loss * TV_WEIGHT
+ self.rot_loss = PART_W * (self.eyel_loss + self.eyer_loss + self.nose_loss + self.mouth_loss)
+ self.var_file = open('var_log.txt', mode='a')
+ t_vars = [var for var in tf.trainable_variables() if 'FeatureExtractDeepFace' not in var.name \
+ and 'processor' not in var.name]
+ print('done!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!')
+
+
+ def isTargetVar(name, tokens):
+ for token in tokens:
+ if token in name:
+ return True
+ return False
+ dec128toks = ['dec128', 'recon128', 'check_img128']
+ self.d_vars = [var for var in t_vars if 'discriminatorLocal' in var.name]
+ self.all_g_vars = [var for var in t_vars if 'discriminatorLocal' not in var.name]
+ self.rot_vars = [var for var in t_vars if 'Rotator' in var.name]
+ self.sel_vars = [var for var in t_vars if 'select' in var.name]
+ self.dec_vars = [var for var in t_vars if 'decoder' in var.name and 'select' not in var.name]
+ self.enc_vars = [var for var in t_vars if 'encoder' in var.name]
+ self.pre_vars = [var for var in t_vars if 'FeaturePredict' in var.name]
+ #
+ self.se_vars = list(self.enc_vars); self.se_vars.extend(self.sel_vars)
+
+ self.ed_vars = list(self.dec_vars); self.ed_vars.extend(self.enc_vars);
+ self.ed_vars.extend(self.pre_vars); self.ed_vars.extend(self.rot_vars);
+ self.ed_vars.extend(self.sel_vars)
+ self.saver = tf.train.Saver(t_vars, max_to_keep=2)
+
+
+ def train(self, config):
+ """Train DCGAN"""
+ #data = glob(os.path.join("./data", config.dataset, "*.jpg"))
+ data = MultiPIE(LOAD_60_LABEL=LOAD_60_LABEL, GENERATE_MASK=USE_MASK, RANDOM_VERIFY=RANDOM_VERIFY, MIRROR_TO_ONE_SIDE = True, source = self.testingphase)
+
+ config.sample_dir += '{:05d}'.format(random.randint(1,100000))
+ init_op = tf.global_variables_initializer()
+ self.sess.run(init_op)
+
+ counter = random.randint(1,30)
+ start_time = time.time()
+
+ if self.load(self.checkpoint_dir):
+ print(" [*] Load SUCCESS")
+ else:
+ print(" [!] Load failed...")
+
+ ##这里调整下
+ sample_images = np.empty([self.test_batch_size, IMAGE_SIZE, IMAGE_SIZE, 3])
+ sample_images[0, ...], feats = data.load_image(self.test_filename)
+
+ sample_eyel = np.empty([self.test_batch_size, EYE_H, EYE_W, 3], dtype=np.float32)
+ sample_eyer = np.empty([self.test_batch_size, EYE_H, EYE_W, 3], dtype=np.float32)
+ sample_nose = np.empty([self.test_batch_size, NOSE_H, NOSE_W, 3], dtype=np.float32)
+ sample_mouth = np.empty([self.test_batch_size, MOUTH_H, MOUTH_W, 3],dtype=np.float32)
+
+ sample_eyel[0,...] = feats[1]
+ sample_eyer[0,...] = feats[2]
+ sample_nose[0,...] = feats[3]
+ sample_mouth[0, ...] = feats[4]
+ #这里调整下
+ if True:
+ if WITHOUT_CODEMAP:
+ sample_images = sample_images[..., 0:3]
+ #append loss log to file
+ counter += 1
+ print('.',end='');sys.stdout.flush()
+ if True:
+ currentBatchSamples = sample_images[ :]
+ currentBatchEyel = sample_eyel[ :]
+ currentBatchEyer = sample_eyer[ :]
+ currentBatchNose = sample_nose[ :]
+ currentBatchMouth = sample_mouth[:]
+ samples = self.sess.run(
+ self.sample_generator,
+ feed_dict={ self.sample_images: currentBatchSamples,
+ self.eyel_sam : currentBatchEyel,
+ self.eyer_sam : currentBatchEyer,
+ self.nose_sam : currentBatchNose,
+ self.mouth_sam : currentBatchMouth
+ })
+ print(samples[5][0,:,:,:].shape)#5,6,7 stand for 32 64 128
+ #原来的保存的是 samples[5或者6或者7]，然后imageio.imwrite(filename,images[i,:,:,0])，合起来就是 samples[5][0,:,:,0]或者samples[5][0,:,:,:]
+
+ print('ok')
+ imageio.imwrite('./train_pt.png',samples[5][0,:,:,:])
+
+ print("[ completed and saved ]")
+
+ else:
+
+ print('test samples reading complete')
+ #batchnum = sample_images.shape[0] // self.test_batch_size #current test batch size
+ savedtest = 0
+ savedoutput = 0
+ sample_dir = 'testall'
+ for i in range(batchnum):
+ print('generating test result batch{}'.format(i))
+ ind = (i*self.test_batch_size, (i+1)*self.test_batch_size)
+ if self.testimg:#Save images
+ samples = self.sess.run(
+ self.sample_generator,
+ feed_dict={ self.sample_images: sample_images[ind[0]:ind[1],:,:,:],
+ self.eyel_sam : sample_eyel[ind[0]:ind[1],...],
+ self.eyer_sam : sample_eyer[ind[0]:ind[1],...],
+ self.nose_sam : sample_nose[ind[0]:ind[1],...],
+ self.mouth_sam : sample_mouth[ind[0]:ind[1],...]}
+ )
+ colorgt = sample_images[ind[0]:ind[1],:,:,0:3]
+ savedtest += save_images(colorgt, [128, 128],
+ './{}/'.format(sample_dir),isOutput=False, filelist=filenames[ind[0]:ind[1]])
+ savedoutput += save_images(samples[5], [128, 128],
+ './{}/'.format(sample_dir),isOutput=True, filelist=filenames[ind[0]:ind[1]])
+ print("[{} completed{} and saved {}.]".format(sample_dir, savedtest, savedoutput))
+ else:#save features
+ savedir = 'testall_f'#'gt50_f'
+ samples = self.sess.run(
+ self.sample512,
+ feed_dict={ self.sample_images: sample_images[ind[0]:ind[1],:,:,:],})
+ listfid = open('probef.txt','a')
+ for j in range(self.test_batch_size):
+ filename = filenames[ind[0]+j]
+ savefilename = filename.replace('.png','.feat')
+ label = filename[0:3]
+ listfid.write(savedir + '/' + filename +' '+ label + '\n')
+ result = samples[j,0:448]
+ if not os.path.exists(savedir):
+ os.mkdir(savedir)
+ f = open(savedir +'/'+ savefilename,'wb')
+ f.write(result)
+ f.close()
+ print("saved %d files!" % (self.test_batch_size * (i+1)))
+ listfid.close()
+
+
+ def processor(self, images, reuse=False):
+ #accept 3 channel images, output orginal 3 channels and 3 x 4 gradient map-> 15 channels
+ with tf.variable_scope("processor") as scope:
+ if reuse:
+ scope.reuse_variables()
+ input_dim = images.get_shape()[-1]
+ gradientKernel = gradientweight()
+ output_dim = gradientKernel.shape[-1]
+ print("processor:", output_dim)
+ k_hw = gradientKernel.shape[0]
+ init = tf.constant_initializer(value=gradientKernel, dtype=tf.float32)
+ w = tf.get_variable('w', [k_hw, k_hw, input_dim, output_dim],
+ initializer=init)
+ conv = tf.nn.conv2d(images, w, strides=[1, 1, 1, 1], padding='SAME')
+ #conv = conv * 2
+ return tf.concat_v2([images, conv], 3)
+
+ def FeaturePredict(self, featvec, reuse=False):
+ with tf.variable_scope("FeaturePredict") as scope:
+ if reuse:
+ scope.reuse_variables()
+ identitylogits = linear(Dropout(featvec, keep_prob=0.3, is_training= not self.testing), output_size=340, scope='idenLinear', bias_start=0.1, with_w=True)[0]
+ return None, identitylogits, None
+
+ def discriminatorLocal(self, images, reuse=False):
+ with tf.variable_scope("discriminatorLocal") as scope:
+ if reuse:
+ scope.reuse_variables()
+
+ h0 = lrelu(conv2d(images, self.df_dim, name='d_h0_conv'))
+ #64
+ h1 = lrelu(batch_norm(conv2d(h0, self.df_dim*2, name='d_h1_conv'), name='d_bn1'))
+ #32
+ h2 = lrelu(batch_norm(conv2d(h1, self.df_dim*4, name='d_h2_conv'), name='d_bn2'))
+ #16
+ h3 = lrelu(batch_norm(conv2d(h2, self.df_dim*8, name='d_h3_conv'), name='d_bn3'))
+ # #8x8
+ h3r1 = resblock(h3, name = "d_h3_conv_res1")
+ h4 = lrelu(batch_norm(conv2d(h3r1, self.df_dim*8, name='d_h4_conv'), name='d_bn4'))
+ h4r1 = resblock(h4, name = "d_h4_conv_res1")
+ h5 = conv2d(h4r1, 1, k_h=1, k_w=1, d_h=1, d_w=1, name='d_h5_conv')
+ h6 = tf.reshape(h5, [self.test_batch_size, -1])
+ #fusing 512 feature map to one layer prediction.
+ return h6, h6 #tf.nn.sigmoid(h6), h6
+
+
+ def decoder(self, feat128, feat64, feat32, feat16, feat8, featvec,
+ g128_images_with_code, g64_images_with_code, g32_images_with_code,
+ eyel, eyer, nose, mouth, c_eyel, c_eyer, c_nose, c_mouth, batch_size = 10, name="decoder", reuse = False):
+ sel_feat_capacity = self.gf_dim
+ with tf.variable_scope(name) as scope:
+ if reuse:
+ scope.reuse_variables()
+ initial_all = tf.concat_v2([featvec, self.z], 1)
+ initial_8 = relu(tf.reshape(linear(initial_all, output_size=8*8*self.gf_dim,scope='initial8', bias_start=0.1, with_w=True)[0],
+ [batch_size, 8, 8, self.gf_dim]))
+ initial_32 = relu(deconv2d(initial_8, [batch_size, 32, 32, self.gf_dim // 2], d_h=4, d_w=4, name="initial32"))
+ initial_64 = relu(deconv2d(initial_32, [batch_size, 64, 64, self.gf_dim // 4], name="initial64"))
+ initial_128 = relu(deconv2d(initial_64, [batch_size, 128, 128, self.gf_dim // 8], name="initial128"))
+
+ before_select8 = resblock(tf.concat_v2([initial_8, feat8], 3), k_h=2, k_w=2, name = "select8_res_1")
+ #selection T module
+ reconstruct8 = resblock(resblock(before_select8, k_h=2, k_w=2, name="dec8_res1"), k_h=2, k_w=2, name="dec8_res2")
+
+ #selection F module
+ reconstruct16_deconv = relu(batch_norm(deconv2d(reconstruct8, [batch_size, 16, 16, self.gf_dim*8], name="g_deconv16"), name="g_bnd1"))
+ before_select16 = resblock(feat16, name = "select16_res_1")
+ reconstruct16 = resblock(resblock(tf.concat_v2([reconstruct16_deconv, before_select16], 3), name="dec16_res1"), name="dec16_res2")
+
+ reconstruct32_deconv = relu(batch_norm(deconv2d(reconstruct16, [batch_size, 32, 32, self.gf_dim*4], name="g_deconv32"), name="g_bnd2"))
+ before_select32 = resblock(tf.concat_v2([feat32, g32_images_with_code, initial_32], 3), name = "select32_res_1")
+ reconstruct32 = resblock(resblock(tf.concat_v2([reconstruct32_deconv, before_select32], 3), name="dec32_res1"), name="dec32_res2")
+ img32 = tf.nn.tanh(conv2d(reconstruct32, 3, d_h=1, d_w=1, name="check_img32"))
+
+ reconstruct64_deconv = relu(batch_norm(deconv2d(reconstruct32, [batch_size, 64, 64, self.gf_dim*2], name="g_deconv64"), name="g_bnd3"))
+ before_select64 = resblock(tf.concat_v2([feat64, g64_images_with_code, initial_64], 3), k_h=5, k_w=5, name = "select64_res_1")
+ reconstruct64 = resblock(resblock(tf.concat_v2([reconstruct64_deconv, before_select64,
+ tf.image.resize_bilinear(img32, [64,64])], 3), name="dec64_res1"), name="dec64_res2")
+ img64 = tf.nn.tanh(conv2d(reconstruct64, 3, d_h=1, d_w=1, name="check_img64"))
+
+ reconstruct128_deconv = relu(batch_norm(deconv2d(reconstruct64, [batch_size, 128, 128, self.gf_dim], name="g_deconv128"), name="g_bnd4"))
+ before_select128 = resblock(tf.concat_v2([feat128, initial_128, g128_images_with_code],3), k_h = 7, k_w = 7, name = "select128_res_1")
+ reconstruct128 = resblock(tf.concat_v2([reconstruct128_deconv, before_select128,
+ self.partCombiner(eyel, eyer, nose, mouth),
+ self.partCombiner(c_eyel, c_eyer, c_nose, c_mouth),
+ tf.image.resize_bilinear(img64, [128,128])], 3), k_h=5, k_w=5, name="dec128_res1")
+ reconstruct128_1 = lrelu(batch_norm(conv2d(reconstruct128, self.gf_dim, k_h=5, k_w=5, d_h=1, d_w=1, name="recon128_conv"), name="recon128_bnc"))
+ reconstruct128_1_r = resblock(reconstruct128_1, name="dec128_res2")
+ reconstruct128_2 = lrelu(batch_norm(conv2d(reconstruct128_1_r, self.gf_dim/2, d_h=1, d_w=1, name="recon128_conv2"),name="recon128_bnc2"))
+ img128 = tf.nn.tanh(conv2d(reconstruct128_2, 3, d_h=1, d_w=1, name="check_img128"))
+
+ return img128, img64, img32, img32, img32, img128, img64, img32
+
+ def generator(self, images, batch_size, name = "generator", reuse = False):
+ with tf.variable_scope(name) as scope:
+ if reuse:
+ scope.reuse_variables()
+ # imgs: input: IMAGE_SIZE x IMAGE_SIZE x CHANNEL
+ # return labels: IMAGE_SIZE x IMAGE_SIZE x 3
+ # U-Net structure, slightly different from the original on the location of relu/lrelu
+ #128x128
+ c0 = lrelu(conv2d(images, self.gf_dim, k_h=7, k_w=7, d_h=1, d_w=1, name="g_conv0"))
+ c0r = resblock(c0, k_h=7, k_w=7, name="g_conv0_res")
+ c1 = lrelu(batch_norm(conv2d(c0r, self.gf_dim, k_h=5, k_w=5, name="g_conv1"),name="g_bnc1"))
+ #64x64
+ c1r = resblock(c1, k_h=5, k_w=5, name="g_conv1_res")
+ c2 = lrelu(batch_norm(conv2d(c1r, self.gf_dim*2, name='g_conv2'),name="g_bnc2"))
+ #32x32
+ c2r = resblock(c2, name="g_conv2_res")
+ c3 = lrelu(batch_norm(conv2d(c2r, self.gf_dim*4, name='g_conv3'),name="g_bnc3"))
+ #16x16
+ c3r = resblock(c3, name="g_conv3_res")
+ c4 = lrelu(batch_norm(conv2d(c3r, self.gf_dim*8, name='g_conv4'),name="g_bnc4"))
+ #8x8
+ c4r = resblock(c4, name="g_conv4_res")
+ # c5 = lrelu(batch_norm(conv2d(c4r, self.gf_dim*8, name='g_conv5'),name="g_bnc5"))
+ # #4x4
+ # #2x2
+ # c6r = resblock(c6,k_h=2, k_w=2, name="g_conv6_res")
+ c4r2 = resblock(c4r, name="g_conv4_res2")
+ c4r3 = resblock(c4r2, name="g_conv4_res3")
+ c4r4 = resblock(c4r3, name="g_conv4_res4")
+ c4r4_l = tf.reshape(c4r4,[batch_size, -1])
+ c7_l = linear(c4r4_l, output_size=512,scope='feature', bias_start=0.1, with_w=True)[0]
+ c7_l_m = tf.maximum(c7_l[:, 0:256], c7_l[:, 256:])
+
+ return c0r, c1r, c2r, c3r, c4r4, c7_l_m
+
+ def partRotator(self, images, name, batch_size=10, reuse=False):
+ #HW 40x40, 32x40, 32x48
+ with tf.variable_scope(name) as scope:
+ if reuse:
+ scope.reuse_variables()
+ c0 = lrelu(conv2d(images, self.gf_dim, d_h=1, d_w=1, name="p_conv0"))
+ c0r = resblock(c0, name="p_conv0_res")
+ c1 = lrelu(batch_norm(conv2d(c0r, self.gf_dim*2, name="p_conv1"),name="p_bnc1"))
+ #down1
+ c1r = resblock(c1, name="p_conv1_res")
+ c2 = lrelu(batch_norm(conv2d(c1r, self.gf_dim*4, name='p_conv2'),name="p_bnc2"))
+ #down2
+ c2r = resblock(c2, name="p_conv2_res")
+ c3 = lrelu(batch_norm(conv2d(c2r, self.gf_dim*8, name='p_conv3'),name="p_bnc3"))
+ #down3 5x5, 4x5, 4x6
+ c3r = resblock(c3, name="p_conv3_res")
+ c3r2 = resblock(c3r, name="p_conv3_res2")
+
+ shape = c3r2.get_shape().as_list()
+ d1 = lrelu(batch_norm(deconv2d(c3r2, [shape[0], shape[1] * 2, shape[2] * 2, self.gf_dim*4], name="p_deconv1"), name="p_bnd1"))
+ #up1
+ after_select_d1 = lrelu(batch_norm(conv2d(tf.concat_v2([d1, c2r], axis=3), self.gf_dim*4, d_h=1, d_w=1, name="p_deconv1_s"),name="p_bnd1_s"))
+ d1_r = resblock(after_select_d1, name="p_deconv1_res")
+ d2 = lrelu(batch_norm(deconv2d(d1_r, [shape[0], shape[1] * 4, shape[2] * 4, self.gf_dim*2], name="p_deconv2"), name="p_bnd2"))
+ #up2
+ after_select_d2 = lrelu(batch_norm(conv2d(tf.concat_v2([d2, c1r], axis=3), self.gf_dim*2, d_h=1, d_w=1, name="p_deconv2_s"),name="p_bnd2_s"))
+ d2_r = resblock(after_select_d2, name="p_deconv2_res")
+ d3 = lrelu(batch_norm(deconv2d(d2_r, [shape[0], shape[1] * 8, shape[2] * 8, self.gf_dim], name="p_deconv3"), name="p_bnd3"))
+ #up3
+ after_select_d3 = lrelu(batch_norm(conv2d(tf.concat_v2([d3, c0r], axis=3), self.gf_dim, d_h=1, d_w=1, name="p_deconv3_s"),name="p_bnd3_s"))
+ d3_r = resblock(after_select_d3, name="p_deconv3_res")
+
+ check_part = tf.nn.tanh(conv2d(d3_r, 3, d_h=1, d_w=1, name="p_check"))
+
+ return d3_r, check_part
+
+ def partCombiner(self, eyel, eyer, nose, mouth):
+ '''
+ x y
+ 43.5823 41.0000
+ 86.4177 41.0000
+ 64.1165 64.7510
+ 47.5863 88.8635
+ 82.5904 89.1124
+ this is the mean locaiton of 5 landmarks
+ '''
+ eyel_p = tf.pad(eyel, [[0,0], [int(41 - EYE_H / 2 - 1), int(IMAGE_SIZE - (41+EYE_H/2 - 1))], [int(44 - EYE_W / 2 - 1), int(IMAGE_SIZE - (44+EYE_W/2-1))], [0,0]])
+ eyer_p = tf.pad(eyer, [[0,0], [int(41 - EYE_H / 2 - 1), int(IMAGE_SIZE - (41+EYE_H/2 - 1))], [int(86 - EYE_W / 2 - 1), int(IMAGE_SIZE - (86+EYE_W/2-1))], [0,0]])
+ nose_p = tf.pad(nose, [[0,0], [int(65 - NOSE_H / 2 - 1), int(IMAGE_SIZE - (65+NOSE_H/2 - 1))], [int(64 - NOSE_W / 2 - 1), int(IMAGE_SIZE - (64+NOSE_W/2-1))], [0,0]])
+ month_p = tf.pad(mouth, [[0,0], [int(89 - MOUTH_H / 2 - 1), int(IMAGE_SIZE - (89+MOUTH_H/2 - 1))], [int(65 - MOUTH_W / 2 - 1), int(IMAGE_SIZE - (65+MOUTH_W/2-1))], [0,0]])
+ eyes = tf.maximum(eyel_p, eyer_p)
+ eye_nose = tf.maximum(eyes, nose_p)
+ return tf.maximum(eye_nose, month_p)
+ def evaluate(self,epoch, idx, batch_idxs, start_time, mode,
+ batch_images_with_code, batch_eyel, batch_eyer, batch_nose, batch_mouth,
+ batch_labels, batch_eyel_label, batch_eyer_label, batch_nose_label, batch_mouth_label, batch_iden):
+ errD = self.d_loss.eval({self.images_with_code: batch_images_with_code, self.labels: batch_labels,
+ self.eyel : batch_eyel, self.eyer : batch_eyer, self.nose: batch_nose, self.mouth: batch_mouth,})
+
+ errG_L = self.weightedErrL1.eval({self.images_with_code: batch_images_with_code, self.labels : batch_labels,
+ self.eyel : batch_eyel, self.eyer : batch_eyer, self.nose: batch_nose, self.mouth: batch_mouth,})
+ errG_L2 = self.weightedErrL2.eval({self.images_with_code: batch_images_with_code, self.labels : batch_labels,
+ self.eyel : batch_eyel, self.eyer : batch_eyer, self.nose: batch_nose, self.mouth: batch_mouth,})
+ errG_L3 = self.weightedErrL3.eval({self.images_with_code: batch_images_with_code, self.labels : batch_labels,
+ self.eyel : batch_eyel, self.eyer : batch_eyer, self.nose: batch_nose, self.mouth: batch_mouth,})
+ errG_adver = self.g_loss_adver.eval({self.images_with_code: batch_images_with_code,
+ self.eyel : batch_eyel, self.eyer : batch_eyer, self.nose: batch_nose, self.mouth: batch_mouth,})
+ errtv = self.tv_loss.eval({self.images_with_code: batch_images_with_code, self.labels : batch_labels,
+ self.eyel : batch_eyel, self.eyer : batch_eyer, self.nose: batch_nose, self.mouth: batch_mouth,})
+
+ errG_sym = self.symErrL1.eval({self.images_with_code: batch_images_with_code,
+ self.eyel : batch_eyel, self.eyer : batch_eyer, self.nose: batch_nose, self.mouth: batch_mouth,})
+ errG2_sym = self.symErrL2.eval({self.images_with_code: batch_images_with_code,
+ self.eyel : batch_eyel, self.eyer : batch_eyer, self.nose: batch_nose, self.mouth: batch_mouth,})
+ errG3_sym = self.symErrL3.eval({self.images_with_code: batch_images_with_code,
+ self.eyel : batch_eyel, self.eyer : batch_eyer, self.nose: batch_nose, self.mouth: batch_mouth,})
+
+ errcheck32 = 0#self.weightedErr_check32.eval({self.images_with_code: batch_images_with_code, self.labels : batch_labels, })
+ errcheck64 = 0#self.weightedErr_check64.eval({self.images_with_code: batch_images_with_code, self.labels : batch_labels, })
+ errcheck128 = 0#self.weightedErr_check128.eval({self.images_with_code: batch_images_with_code, self.labels : batch_labels, })
+
+ erreyel = self.eyel_loss.eval({self.eyel: batch_eyel, self.eyel_label: batch_eyel_label })
+ erreyer = self.eyer_loss.eval({self.eyel: batch_eyel, self.eyer: batch_eyer, self.eyer_label: batch_eyer_label})
+ errnose = self.nose_loss.eval({self.nose: batch_nose, self.nose_label: batch_nose_label})
+ errmouth = self.mouth_loss.eval({self.mouth : batch_mouth, self.mouth_label: batch_mouth_label})
+ erriden = self.idenloss.eval({self.images_with_code: batch_images_with_code, self.idenlabels : batch_iden,})
+
+
+ if 'f' in MODE:
+ errDv = self.dv_loss.eval({self.images_with_code: batch_images_with_code, self.labels : batch_labels,
+ self.eyel : batch_eyel, self.eyer : batch_eyer, self.nose: batch_nose, self.mouth: batch_mouth,})
+ else:
+ errDv = 0
+ err_total_G = L1_1_W * (errG_L + errG_sym * SYM_W) + L1_2_W * (errG_L2 + errG2_sym * SYM_W) + L1_3_W * (errG_L3 + errG3_sym * SYM_W) \
+ + ALPHA_ADVER * errG_adver + errDv * BELTA_FEATURE + IDEN_W*erriden #+ COND_WEIGHT*(errCondL12 + errCondL23) + errtv * TV_WEIGHT
+ errfeat_total = errcheck32 + errcheck64+errcheck128 + PART_W * (erreyel + erreyer + errnose + errmouth)
+ tobePrint = "%s Epo[%2d][%4d/%4d][t%4.2f] d_l:%.4f" % (MODE + 'T' if mode == 'test' else '', epoch, idx, batch_idxs, time.time() - start_time, errD)
+ tobePrint += " fstol:%.0f cg32:%.0f cg64:%.0f cg128:%.0f el:%.0f er:%.0f no:%.0f mo:%.0f\n" % \
+ (errfeat_total, errcheck32, errcheck64,errcheck128, erreyel, erreyer, errnose, errmouth)
+ tobePrint += "g_l:%.0f gL1:%.0f(sym:%.0f) gL2:%.0f(sym:%.0f) gL3:%.0f(sym:%.0f) gadv:%.4f dv_l:%.2f iden:%.4f, tv:%.0f " \
+ % (err_total_G, errG_L, errG_sym, errG_L2, errG2_sym, errG_L3, errG3_sym, errG_adver, errDv, erriden, errtv)
+ if 'f' in MODE:
+ tobePrint += 'L:{} G:{}'.format(ALPHA_ADVER, BELTA_FEATURE)
+ self.f.write(tobePrint+'\n')
+ self.f.flush()
+ print(tobePrint)
+ #DEEPFACE MODEL BEGINS---
+ def loadDeepFace(self, DeepFacePath= None):
+ if DeepFacePath is None:
+ path = sys.modules[self.__class__.__module__].__file__
+ # print path
+ path = os.path.abspath(os.path.join(path, os.pardir))
+ # print path
+ path = os.path.join(path, "DeepFace168.pickle")
+ DeepFacePath = path
+ logging.info("Load npy file from '%s'.", DeepFacePath)
+ if not os.path.isfile(DeepFacePath):
+ logging.error(("File '%s' not found. "), DeepFacePath)
+ sys.exit(1)
+
+ #with open(DeepFacePath,'r') as file:
+ # self.data_dict = pickle.load(file)
+ with open(DeepFacePath,'rb') as file: # Binary read
+ u = pickle._Unpickler(file)
+ u.encoding = 'latin1'
+ p = u.load()
+ self.data_dict = p
+
+ print("Deep Face pickle data file loaded")
+
+ def FeatureExtractDeepFace(self, images, name = "FeatureExtractDeepFace", reuse=False):
+ #Preprocessing: from color to gray(reduce_mean)
+ with tf.variable_scope(name) as scope:
+ if reuse:
+ scope.reuse_variables()
+
+ conv1 = self._conv_layer(images, name='conv1')
+ #print(3, type(3))
+ slice1_1, slice1_2 = tf.split(3, 2, conv1)
+ eltwise1 = tf.maximum(slice1_1, slice1_2)
+ pool1 = tf.nn.max_pool(eltwise1, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1],
+ padding='SAME')
+ conv2_1 = self._conv_layer(pool1, name='conv2_1')
+ slice2_1_1, slice2_1_2 = tf.split(3, 2, conv2_1)
+ eltwise2_1 = tf.maximum(slice2_1_1, slice2_1_2)
+
+ conv2_2 = self._conv_layer(eltwise2_1, name='conv2_2')
+ slice2_2_1, slice2_2_2 = tf.split(3, 2, conv2_2)
+ eltwise2_2 = tf.maximum(slice2_2_1, slice2_2_2)
+
+ res2_1 = pool1 + eltwise2_2
+
+ conv2a = self._conv_layer(res2_1, name='conv2a')
+ slice2a_1, slice2a_2 = tf.split(3, 2, conv2a)
+ eltwise2a = tf.maximum(slice2a_1, slice2a_2)
+
+ conv2 = self._conv_layer(eltwise2a, name='conv2')
+ slice2_1, slice2_2 = tf.split(3, 2, conv2)
+ eltwise2 = tf.maximum(slice2_1, slice2_2)
+
+ pool2 = tf.nn.max_pool(eltwise2, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1],
+ padding='SAME')
+
+ conv3_1 = self._conv_layer(pool2, name='conv3_1')
+ slice3_1_1, slice3_1_2 = tf.split(3, 2, conv3_1)
+ eltwise3_1 = tf.maximum(slice3_1_1, slice3_1_2)
+
+ conv3_2 = self._conv_layer(eltwise3_1, name='conv3_2')
+ slice3_2_1, slice3_2_2 = tf.split(3, 2, conv3_2)
+ eltwise3_2 = tf.maximum(slice3_2_1, slice3_2_2)
+
+ res3_1 = pool2 + eltwise3_2
+
+ conv3_3 = self._conv_layer(res3_1, name='conv3_3')
+ slice3_3_1, slice3_3_2 = tf.split(3, 2, conv3_3)
+ eltwise3_3 = tf.maximum(slice3_3_1, slice3_3_2)
+
+ conv3_4 = self._conv_layer(eltwise3_3, name='conv3_4')
+ slice3_4_1, slice3_4_2 = tf.split(3, 2, conv3_4)
+ eltwise3_4 = tf.maximum(slice3_4_1, slice3_4_2)
+
+ res3_2 = res3_1 + eltwise3_4
+
+ conv3a = self._conv_layer(res3_2, name='conv3a')
+ slice3a_1, slice3a_2 = tf.split(3, 2, conv3a)
+ eltwise3a = tf.maximum(slice3a_1, slice3a_2)
+
+ conv3 = self._conv_layer(eltwise3a, name='conv3')
+ slice3_1, slice3_2 = tf.split(3, 2, conv3)
+ eltwise3 = tf.maximum(slice3_1, slice3_2)
+
+ pool3 = tf.nn.max_pool(eltwise3, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1],
+ padding='SAME')
+
+ conv4_1 = self._conv_layer(pool3, name='conv4_1')
+ slice4_1_1, slice4_1_2 = tf.split(3, 2, conv4_1)
+ eltwise4_1 = tf.maximum(slice4_1_1, slice4_1_2)
+
+ conv4_2 = self._conv_layer(eltwise4_1, name='conv4_2')
+ slice4_2_1, slice4_2_2 = tf.split(3, 2, conv4_2)
+ eltwise4_2 = tf.maximum(slice4_2_1, slice4_2_2)
+
+ res4_1 = pool3 + eltwise4_2
+
+ conv4_3 = self._conv_layer(res4_1, name='conv4_3')
+ slice4_3_1, slice4_3_2 = tf.split(3, 2, conv4_3)
+ eltwise4_3 = tf.maximum(slice4_3_1, slice4_3_2)
+
+ conv4_4 = self._conv_layer(eltwise4_3, name='conv4_4')
+ slice4_4_1, slice4_4_2 = tf.split(3, 2, conv4_4)
+ eltwise4_4 = tf.maximum(slice4_4_1, slice4_4_2)
+
+ res4_2 = res4_1 + eltwise4_4
+
+ conv4_5 = self._conv_layer(res4_2, name='conv4_5')
+ slice4_5_1, slice4_5_2 = tf.split(3, 2, conv4_5)
+ eltwise4_5 = tf.maximum(slice4_5_1, slice4_5_2)
+
+ conv4_6 = self._conv_layer(eltwise4_5, name='conv4_6')
+ slice4_6_1, slice4_6_2 = tf.split(3, 2, conv4_6)
+ eltwise4_6 = tf.maximum(slice4_6_1, slice4_6_2)
+
+ res4_3 = res4_2 + eltwise4_6
+
+ conv4a = self._conv_layer(res4_3, name='conv4a')
+ slice4a_1, slice4a_2 = tf.split(3, 2, conv4a)
+ eltwise4a = tf.maximum(slice4a_1, slice4a_2)
+
+ conv4 = self._conv_layer(eltwise4a, name='conv4')
+ slice4_1, slice4_2 = tf.split(3, 2, conv4)
+ eltwise4 = tf.maximum(slice4_1, slice4_2)
+
+ conv5_1 = self._conv_layer(eltwise4, name='conv5_1')
+ slice5_1_1, slice5_1_2 = tf.split(3, 2, conv5_1)
+ eltwise5_1 = tf.maximum(slice5_1_1, slice5_1_2)
+
+ conv5_2 = self._conv_layer(eltwise5_1, name='conv5_2')
+ slice5_2_1, slice5_2_2 = tf.split(3, 2, conv5_2)
+ eltwise5_2 = tf.maximum(slice5_2_1, slice5_2_2)
+
+ res5_1 = eltwise4 + eltwise5_2
+
+ conv5_3 = self._conv_layer(res5_1, name='conv5_3')
+ slice5_3_1, slice5_3_2 = tf.split(3, 2, conv5_3)
+ eltwise5_3 = tf.maximum(slice5_3_1, slice5_3_2)
+
+ conv5_4 = self._conv_layer(eltwise5_3, name='conv5_4')
+ slice5_4_1, slice5_4_2 = tf.split(3, 2, conv5_4)
+ eltwise5_4 = tf.maximum(slice5_4_1, slice5_4_2)
+
+ res5_2 = res5_1 + eltwise5_4
+
+ conv5_5 = self._conv_layer(res5_2, name='conv5_5')
+ slice5_5_1, slice5_5_2 = tf.split(3, 2, conv5_5)
+ eltwise5_5 = tf.maximum(slice5_5_1, slice5_5_2)
+
+ conv5_6 = self._conv_layer(eltwise5_5, name='conv5_6')
+ slice5_6_1, slice5_6_2 = tf.split(3, 2, conv5_6)
+ eltwise5_6 = tf.maximum(slice5_6_1, slice5_6_2)
+
+ res5_3 = res5_2 + eltwise5_6
+
+ conv5_7 = self._conv_layer(res5_3, name='conv5_7')
+ slice5_7_1, slice5_7_2 = tf.split(3, 2, conv5_7)
+ eltwise5_7 = tf.maximum(slice5_7_1, slice5_7_2)
+
+ conv5_8 = self._conv_layer(eltwise5_7, name='conv5_8')
+ slice5_8_1, slice5_8_2 = tf.split(3, 2, conv5_8)
+ eltwise5_8 = tf.maximum(slice5_8_1, slice5_8_2)
+
+ res5_4 = res5_3 + eltwise5_8
+
+ conv5a = self._conv_layer(res5_4, name='conv5a')
+ slice5a_1, slice5a_2 = tf.split(3, 2, conv5a)
+ eltwise5a = tf.maximum(slice5a_1, slice5a_2)
+
+ conv5 = self._conv_layer(eltwise5a, name='conv5')
+ slice5_1, slice5_2 = tf.split(3, 2, conv5)
+ eltwise5 = tf.maximum(slice5_1, slice5_2)
+ pool4 = tf.nn.max_pool(eltwise5, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1],
+ padding='SAME')
+ pool4_transposed = tf.transpose(pool4, perm=[0, 3, 1, 2])
+ # pool4_reshaped = tf.reshape(pool4_transposed, shape=[tf.shape(pool4)[0],-1])
+ fc1 = self._fc_layer(pool4_transposed, name="fc1")
+ slice_fc1_1, slice_fc1_2 = tf.split(1, 2, fc1)
+ eltwise_fc1 = tf.maximum(slice_fc1_1, slice_fc1_2)
+
+ return eltwise1, eltwise2, eltwise3, eltwise5, pool4, eltwise_fc1
+ #DEEPFACE NET ENDS---
+
+ #DEEPFACE OPS BEGINS---
+ def _conv_layer(self, input_, output_dim=96,
+ k_h=3, k_w=3, d_h=1, d_w=1, stddev=0.02,
+ name="conv2d"):
+ #Note: currently kernel size and input output channel num are decided by loaded filter weights.
+ #only strides are decided by calling param.
+ with tf.variable_scope(name) as scope:
+ filt = self.get_conv_filter(name)
+ conv = tf.nn.conv2d(input_, filt, strides=[1, d_h, d_w, 1], padding='SAME')
+ conv_biases = self.get_bias(name)
+ return tf.nn.bias_add(conv, conv_biases)
+ return conv
+
+ def _fc_layer(self, bottom, name="fc1", num_classes=None):
+ with tf.variable_scope(name) as scope:
+ #shape = bottom.get_shape().as_list()
+ if name == 'fc1':
+ filt = self.get_fc_weight(name)
+ bias = self.get_bias(name)
+ reshaped_bottom = tf.reshape(bottom,[tf.shape(bottom)[0],-1])
+ return tf.matmul(reshaped_bottom, filt) + bias
+
+ def get_conv_filter(self, name):
+ init = tf.constant_initializer(value=self.data_dict[name][0],
+ dtype=tf.float32)
+ shape = self.data_dict[name][0].shape
+ var = tf.get_variable(name="filter", initializer=init, shape=shape)
+ return var
+
+ def get_bias(self, name, num_classes=None):
+ bias_wights = self.data_dict[name][1]
+ shape = self.data_dict[name][1].shape
+ init = tf.constant_initializer(value=bias_wights,
+ dtype=tf.float32)
+ var = tf.get_variable(name="biases", initializer=init, shape=shape)
+ return var
+
+ def get_fc_weight(self, name):
+ init = tf.constant_initializer(value=self.data_dict[name][0],
+ dtype=tf.float32)
+ shape = self.data_dict[name][0].shape
+ var = tf.get_variable(name="weights", initializer=init, shape=shape)
+ return var
+
+ #DEEPFACE OPS ENDS---
+
+
+ def load(self, checkpoint_dir):
+ print(" [*] Reading checkpoints...")
+
+ #model_dir = "%s_%s_%s" % (self.dataset_name, self.test_batch_size, self.output_size)
+ #checkpoint_dir = os.path.join(checkpoint_dir, model_dir)
+ checkpoint_dir = '/home/ubuntu3000/pt/TP-GAN/checkpoint60/MultiPIE_2_128/'
+ ckpt = tf.train.get_checkpoint_state(checkpoint_dir)
+ if ckpt and ckpt.model_checkpoint_path:
+ ckpt_name = os.path.basename(ckpt.model_checkpoint_path)
+ self.saver.restore(self.sess, os.path.join(checkpoint_dir, ckpt_name))
+ print(" [*] Success to read {}".format(ckpt_name))
+ return True
+ else:
+ print(" [*] Failed to find a checkpoint")
+ return False
+
+def main(_):
+ pp.pprint(flags.FLAGS.__flags)
+
+ assert os.path.exists(FLAGS.checkpoint_dir)
+ assert os.path.exists(FLAGS.sample_dir)
+
+ with tf.Session() as sess:
+ dcgan = DCGAN(sess,
+ image_size=FLAGS.image_size,
+ batch_size=FLAGS.test_batch_size,
+ output_size=FLAGS.output_size,
+ c_dim=FLAGS.c_dim,
+ dataset_name=FLAGS.dataset,
+ is_crop=FLAGS.is_crop,
+ checkpoint_dir=FLAGS.checkpoint_dir,
+ sample_dir=FLAGS.sample_dir)
+ dcgan.test_filename='test_tem.png'
+ dcgan.train(FLAGS)
+
+if __name__ == '__main__':
+ tf.app.run()
diff --git a/test_result/2.jpeg b/test_result/2.jpeg
new file mode 100644
index 0000000..4c5a0e4
Binary files /dev/null and b/test_result/2.jpeg differ
diff --git a/test_result/21.png b/test_result/21.png
new file mode 100644
index 0000000..c79d1e9
Binary files /dev/null and b/test_result/21.png differ
diff --git a/test_result/22.png b/test_result/22.png
new file mode 100644
index 0000000..7451de0
Binary files /dev/null and b/test_result/22.png differ
diff --git a/test_result/2_vis.png b/test_result/2_vis.png
new file mode 100644
index 0000000..8aa9b06
Binary files /dev/null and b/test_result/2_vis.png differ
diff --git a/test_result/test.txt b/test_result/test.txt
new file mode 100644
index 0000000..3ab3e3f
--- /dev/null
+++ b/test_result/test.txt
@@ -0,0 +1 @@
+测试结果