when i was training my data, the trainging suddenly stoped and the error shows as follows:

[Jnyle]

Traceback (most recent call last):
File "train.py", line 298, in
train()
File "train.py", line 275, in train
summary_str = sess.run(summary_op)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 789, in run
run_metadata_ptr)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 997, in _run
feed_dict_string, options, run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1132, in _do_run
target_list, options, run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1152, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: input must have at least k columns
[[Node: rpn_losses/TopKV2 = TopKV2[T=DT_FLOAT, sorted=true, _device="/job:localhost/replica:0/task:0/cpu:0"](rpn_losses/strided_slice_2/_4331, rpn_losses/TopKV2/k)]]
[[Node: fast_rcnn_predict/fast_rcnn_proposals/Where/_4155 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/gpu:0", send_device="/job:localhost/replica:0/task:0/cpu:0", send_device_incarnation=1, tensor_name="edge_6861_fast_rcnn_predict/fast_rcnn_proposals/Where", tensor_type=DT_INT64, _device="/job:localhost/replica:0/task:0/gpu:0"]]

Caused by op u'rpn_losses/TopKV2', defined at:
File "train.py", line 298, in
train()
File "train.py", line 93, in train
rpn_location_loss, rpn_classification_loss, rpn_predict_boxes, rpn_predict_scores = rpn.rpn_losses()
File "../libs/rpn/build_rpn.py", line 456, in rpn_losses
top_k_scores, top_k_indices = tf.nn.top_k(minibatch_boxes_softmax_scores[:, 1], k=20)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/nn_ops.py", line 1946, in top_k
return gen_nn_ops._top_kv2(input, k=k, sorted=sorted, name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_nn_ops.py", line 2572, in _top_kv2
name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 767, in apply_op
op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2506, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1269, in init
self._traceback = _extract_stack()

InvalidArgumentError (see above for traceback): input must have at least k columns
[[Node: rpn_losses/TopKV2 = TopKV2[T=DT_FLOAT, sorted=true, _device="/job:localhost/replica:0/task:0/cpu:0"](rpn_losses/strided_slice_2/_4331, rpn_losses/TopKV2/k)]]
[[Node: fast_rcnn_predict/fast_rcnn_proposals/Where/_4155 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/gpu:0", send_device="/job:localhost/replica:0/task:0/cpu:0", send_device_incarnation=1, tensor_name="edge_6861_fast_rcnn_predict/fast_rcnn_proposals/Where", tensor_type=DT_INT64, _device="/job:localhost/replica:0/task:0/gpu:0"]]

[GuoleiSun]

Same problem, could you please reply?

[Jin-Weihua]

你训练成功了么

[zsh1234567]

May I ask what data set do you use?

[wanghangege]

请问你的数据集制作工具是什么样子呢