Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

dataset problem #11

Open
LingJiangXie opened this issue Dec 26, 2017 · 22 comments
Open

dataset problem #11

LingJiangXie opened this issue Dec 26, 2017 · 22 comments

Comments

@LingJiangXie
Copy link

Hi ! Thank you for this pytorch version sphereface code ,I have found this for a long time .But when I run train.py ,some problems occured.
VisibleDeprecationWarning: using a non-integer number instead of an integer will result in an error in the future nparrimage = np.frombuffer(arrimage.get_obj(),np.float32).reshape(10,len(arrimage)/10) .I think my numpy version(1.11.0) is not fine.
'There is no item named %r in the archive' % name) KeyError: "There is no item named '1212722/038.jpg' in the archive" . But I find this image in the fold 1212722 .
Last problem may I get the mtcnn landmarks for casia? or you can tell me mtcnn codes ,I run it.Thank you very much!
@LingJiangXie
Copy link
Author

When I changed my python from 3.6 to 2.7 .Problem "VisibleDeprecationWarning: using a non-integer number instead of an integer will result in an error in the future nparrimage = np.frombuffer(arrimage.get_obj(),np.float32).reshape(10,len(arrimage)/10) ." disappeared. And I find the landmarks in the fold (data).But problem some image not found still exists.I thinlk multiprocessing module to read images cause the problem.Can you tell me how to fix it ?

@LingJiangXie
Copy link
Author

Process Process-1:
Traceback (most recent call last):
File "/home/dany/anaconda3/envs/snakes/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/home/dany/anaconda3/envs/snakes/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "/home/dany/Documents/workspace/sphereface_pytorch-master_2.7/dataset.py", line 38, in dataset_handle
data = callback(name,filename,pindex,cacheobj,zfile)
File "/home/dany/Documents/workspace/sphereface_pytorch-master_2.7/train.py", line 54, in dataset_load
data = np.frombuffer(zfile.read(nameinzip),np.uint8)
File "/home/dany/anaconda3/envs/snakes/lib/python2.7/zipfile.py", line 938, in read
return self.open(name, "r", pwd).read()
File "/home/dany/anaconda3/envs/snakes/lib/python2.7/zipfile.py", line 964, in open
zinfo = self.getinfo(name)
File "/home/dany/anaconda3/envs/snakes/lib/python2.7/zipfile.py", line 912, in getinfo
'There is no item named %r in the archive' % name)
KeyError: "There is no item named '3331486/011.jpg' in the archive"
Process Process-2:
Traceback (most recent call last):
File "/home/dany/anaconda3/envs/snakes/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/home/dany/anaconda3/envs/snakes/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "/home/dany/Documents/workspace/sphereface_pytorch-master_2.7/dataset.py", line 38, in dataset_handle
data = callback(name,filename,pindex,cacheobj,zfile)
File "/home/dany/Documents/workspace/sphereface_pytorch-master_2.7/train.py", line 54, in dataset_load
data = np.frombuffer(zfile.read(nameinzip),np.uint8)
File "/home/dany/anaconda3/envs/snakes/lib/python2.7/zipfile.py", line 938, in read
return self.open(name, "r", pwd).read()
File "/home/dany/anaconda3/envs/snakes/lib/python2.7/zipfile.py", line 964, in open
zinfo = self.getinfo(name)
File "/home/dany/anaconda3/envs/snakes/lib/python2.7/zipfile.py", line 912, in getinfo
'There is no item named %r in the archive' % name)
KeyError: "There is no item named '0235966/005.jpg' in the archive"

@lucaskyle
Copy link

plz check the path should be /..../.../lfw.zip/file_name/xxx.jpg

@LingJiangXie
Copy link
Author

Yes,I have checked ...

@hujuan940506
Copy link

I have the same problem as LingJiangXie, I want to ask you that your problem has been solved？

Traceback (most recent call last):
File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "/home/liuyh/IDcar/sphereface_pytorch-master/dataset.py", line 38, in dataset_handle
data = callback(name,filename,pindex,cacheobj,zfile)
File "train.py", line 59, in dataset_load
print(zfile.read('casia/' + nameinzip))
File "/usr/lib/python2.7/zipfile.py", line 935, in read
return self.open(name, "r", pwd).read()
File "/usr/lib/python2.7/zipfile.py", line 961, in open
zinfo = self.getinfo(name)
File "/usr/lib/python2.7/zipfile.py", line 909, in getinfo
'There is no item named %r in the archive' % name)
KeyError: "There is no item named 'casia/0001392/091.jpg' in the archive"

@LingJiangXie
Copy link
Author

Not solved.......

@clcarwin
Copy link
Owner

clcarwin commented Jan 3, 2018

Please try to read this issue #3 .

@LingJiangXie
Copy link
Author

Thank you !

@hujuan940506
Copy link

@clcarwin Thank you very much!

@hujuan940506
Copy link

Have you solved it now? @LingJiangXie

@LingJiangXie
Copy link
Author

Downloading datasets
https://pan.baidu.com/s/1i5gX7Dz
d31k

@dpcross
Copy link

dpcross commented Jan 25, 2018

modify zfile.read('nameinzip) to zfile.read('CASIA-WebFace/'+nameinzip) (train.py)
add try catch before data = callback(name,filename,pindex,cacheobj,zfile) ,continue except

@hujuan940506
Copy link

@dpcross Thank you very much! I have solved this route problem in your way, but I have encountered this problem:

Traceback (most recent call last):
File "train.py", line 162, in
train(epoch,args)
File "train.py", line 131, in train
outputs = net(inputs)
File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 206, in call
result = self.forward(*input, **kwargs)
File "/home/liuyh/IDcar/sphereface_pytorch-master/net_sphere.py", line 175, in forward
x = self.fc6(x)
File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 206, in call
result = self.forward(*input, **kwargs)
File "/home/liuyh/IDcar/sphereface_pytorch-master/net_sphere.py", line 40, in forward
cos_theta = cos_theta / xlen.view(-1,1) / wlen.view(1,-1)
File "/usr/local/lib/python2.7/dist-packages/torch/autograd/variable.py", line 780, in div
return self.div(other)
File "/usr/local/lib/python2.7/dist-packages/torch/autograd/variable.py", line 312, in div
return Div()(self, other)
File "/usr/local/lib/python2.7/dist-packages/torch/autograd/_functions/basic_ops.py", line 57, in forward
return a.div(b)
RuntimeError: sizes do not match at /b/wheel/pytorch-src/torch/lib/THC/generated/../generic/THCTensorMathPointwise.cu:344

@hujuan940506
Copy link

@LingJiangXie Have you encounter this problem in your trainning process

@hujuan940506
Copy link

@clcarwin Thank you very much! I have solved this problem in your way, update pytorch to version 0.2.x.

@sevenHsu
Copy link

sevenHsu commented Apr 1, 2018

I just have the same problem with you guys.And my pytorch version is 0.2.0_3,but it does'nt work.How can i solve this problem??
Traceback (most recent call last):
File "lfw_eval.py", line 102, in
img1 = alignment(cv2.imdecode(np.frombuffer(zfile.read(name1),np.uint8),1),landmark[name1])
File "/usr/lib/python3.6/zipfile.py", line 1314, in read
with self.open(name, "r", pwd) as fp:
File "/usr/lib/python3.6/zipfile.py", line 1352, in open
zinfo = self.getinfo(name)
File "/usr/lib/python3.6/zipfile.py", line 1281, in getinfo
'There is no item named %r in the archive' % name)
KeyError: "There is no item named 'Abel_Pacheco/Abel_Pacheco_0001.jpg' in the archive"

@lucaskyle
Copy link

lucaskyle commented Apr 2, 2018 via email

@sevenHsu
Copy link

sevenHsu commented Apr 2, 2018 via email

@yonger001
Copy link

hi，guys. when I run the train.py,I got a problem as fllows.
KeyError: "There is no item named '1478996/020.jpg' in the archive"
then i modified the np.frombuffer(zfile.read(nameinzip),np.uint8) to np.frombuffer(zfile.read(a+nameinzip),np.uint8), a+nameinzip is the absolute path of image,but it didn‘t work.
Do you have any idea?
thank you so much!waiting for your reply！

@gmsdelmundo gmsdelmundo mentioned this issue Aug 28, 2018
Open
@netpcvnn
Copy link

Hi @LingJiangXie, Can you upload the washed dataset to other sites? I cannot to download from baidu because we dont have Chinese mobile phone number to activate the account.
Thank you!

@chenmingjiong
Copy link

chenmingjiong commented Jan 21, 2019 •
edited
Loading

Hi @LingJiangXie, Can you upload the washed dataset to other sites? I cannot to download from baidu because we dont have Chinese mobile phone number to activate the account.
Thank you!

I have uploaded it to OneDrive:
https://1drv.ms/u/s!AjMP8wpvXdkfhMhnmNDEdj5oor7M_A

@helang818
Copy link

hi，guys. when I run the train.py,I got a problem as fllows. KeyError: "There is no item named '1478996/020.jpg' in the archive" then i modified the np.frombuffer(zfile.read(nameinzip),np.uint8) to np.frombuffer(zfile.read(a+nameinzip),np.uint8), a+nameinzip is the absolute path of image,but it didn‘t work. Do you have any idea? thank you so much!waiting for your reply！

have you soved this issue?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
10 participants
@clcarwin @netpcvnn @helang818 @yonger001 @LingJiangXie @sevenHsu @dpcross @lucaskyle @hujuan940506 @chenmingjiong