Running DeepRL code issues

[VijiKK]

Hi,

I've gone through deepril it's very interesting and related to my research what I'm doing now.
So I tried to run the deepRL code in my local machine. I've installed Sumo and required python and tensorflow as mentioned.
But the code is throwing errors, i've used config_test_large.ini. I'm running directly in my windows machine, through pycharm.

Now i'm with the error in input and output shapes,
ValueError: Shape of a new variable (lr_0a_q/q/w) must be fully defined, but instead was (?, 5).

Could you please help me? I might be missing some of the configurations or may be the way I'm running the code is not right.

[cts198859]

Can you print the full stack of errors? Seems like an error inside tensorflow. Also I only tested the code on mac / ubuntu, so not sure if this is something specific to Windows.

[VijiKK]

Hi, thank you so much for the response.
Here are the error details, Initially I was facing issue due to the place holder, at the line self.S = tf.placeholder(tf.float32, shape=[None, n_s]) in LRQPolicy class. Then just to pass that error I changed to not passing the size, self.S = tf.placeholder(tf.float32).
Then now the error is at self.qvalues.

Error:
C:\Users\lry466.conda\envs\ML3dot5Vij\python.exe C:/Users/lry466/Documents/Viji/TafficLight_GNN/deeprl_signal_control-master/main.py train
Loading configuration ... done.
2021-12-15 18:43:44,864 [INFO] Env: init 25 node information:
nt8:
neigbor: ['nt13', 'nt9', 'nt3', 'nt7']
ilds_in: ['nt13_nt8_0', 'nt9_nt8_0', 'nt9_nt8_1', 'nt3_nt8_0', 'nt7_nt8_0', 'nt7_nt8_1']
nt25:
neigbor: ['nt20', 'nt24']
ilds_in: ['np11_nt25_0', 'np10_nt25_0', 'np10_nt25_1', 'nt20_nt25_0', 'nt24_nt25_0', 'nt24_nt25_1']
nt12:
neigbor: ['nt17', 'nt13', 'nt7', 'nt11']
ilds_in: ['nt17_nt12_0', 'nt13_nt12_0', 'nt13_nt12_1', 'nt7_nt12_0', 'nt11_nt12_0', 'nt11_nt12_1']
nt6:
neigbor: ['nt11', 'nt7', 'nt1']
ilds_in: ['nt11_nt6_0', 'nt7_nt6_0', 'nt7_nt6_1', 'nt1_nt6_0', 'np19_nt6_0', 'np19_nt6_1']
nt10:
neigbor: ['nt15', 'nt5', 'nt9']
ilds_in: ['nt15_nt10_0', 'np7_nt10_0', 'np7_nt10_1', 'nt5_nt10_0', 'nt9_nt10_0', 'nt9_nt10_1']
nt23:
neigbor: ['nt24', 'nt18', 'nt22']
ilds_in: ['np13_nt23_0', 'nt24_nt23_0', 'nt24_nt23_1', 'nt18_nt23_0', 'nt22_nt23_0', 'nt22_nt23_1']
nt15:
neigbor: ['nt20', 'nt10', 'nt14']
ilds_in: ['nt20_nt15_0', 'np8_nt15_0', 'np8_nt15_1', 'nt10_nt15_0', 'nt14_nt15_0', 'nt14_nt15_1']
nt19:
neigbor: ['nt24', 'nt20', 'nt14', 'nt18']
ilds_in: ['nt24_nt19_0', 'nt20_nt19_0', 'nt20_nt19_1', 'nt14_nt19_0', 'nt18_nt19_0', 'nt18_nt19_1']
nt1:
neigbor: ['nt6', 'nt2']
ilds_in: ['nt6_nt1_0', 'nt2_nt1_0', 'nt2_nt1_1', 'np1_nt1_0', 'np20_nt1_0', 'np20_nt1_1']
nt17:
neigbor: ['nt22', 'nt18', 'nt12', 'nt16']
ilds_in: ['nt22_nt17_0', 'nt18_nt17_0', 'nt18_nt17_1', 'nt12_nt17_0', 'nt16_nt17_0', 'nt16_nt17_1']
nt14:
neigbor: ['nt19', 'nt15', 'nt9', 'nt13']
ilds_in: ['nt19_nt14_0', 'nt15_nt14_0', 'nt15_nt14_1', 'nt9_nt14_0', 'nt13_nt14_0', 'nt13_nt14_1']
nt18:
neigbor: ['nt23', 'nt19', 'nt13', 'nt17']
ilds_in: ['nt23_nt18_0', 'nt19_nt18_0', 'nt19_nt18_1', 'nt13_nt18_0', 'nt17_nt18_0', 'nt17_nt18_1']
nt11:
neigbor: ['nt16', 'nt12', 'nt6']
ilds_in: ['nt16_nt11_0', 'nt12_nt11_0', 'nt12_nt11_1', 'nt6_nt11_0', 'np18_nt11_0', 'np18_nt11_1']
nt22:
neigbor: ['nt23', 'nt17', 'nt21']
ilds_in: ['np14_nt22_0', 'nt23_nt22_0', 'nt23_nt22_1', 'nt17_nt22_0', 'nt21_nt22_0', 'nt21_nt22_1']
nt7:
neigbor: ['nt12', 'nt8', 'nt2', 'nt6']
ilds_in: ['nt12_nt7_0', 'nt8_nt7_0', 'nt8_nt7_1', 'nt2_nt7_0', 'nt6_nt7_0', 'nt6_nt7_1']
nt2:
neigbor: ['nt7', 'nt3', 'nt1']
ilds_in: ['nt7_nt2_0', 'nt3_nt2_0', 'nt3_nt2_1', 'np2_nt2_0', 'nt1_nt2_0', 'nt1_nt2_1']
nt21:
neigbor: ['nt22', 'nt16']
ilds_in: ['np15_nt21_0', 'nt22_nt21_0', 'nt22_nt21_1', 'nt16_nt21_0', 'np16_nt21_0', 'np16_nt21_1']
nt3:
neigbor: ['nt8', 'nt4', 'nt2']
ilds_in: ['nt8_nt3_0', 'nt4_nt3_0', 'nt4_nt3_1', 'np3_nt3_0', 'nt2_nt3_0', 'nt2_nt3_1']
nt4:
neigbor: ['nt9', 'nt5', 'nt3']
ilds_in: ['nt9_nt4_0', 'nt5_nt4_0', 'nt5_nt4_1', 'np4_nt4_0', 'nt3_nt4_0', 'nt3_nt4_1']
nt16:
neigbor: ['nt21', 'nt17', 'nt11']
ilds_in: ['nt21_nt16_0', 'nt17_nt16_0', 'nt17_nt16_1', 'nt11_nt16_0', 'np17_nt16_0', 'np17_nt16_1']
nt9:
neigbor: ['nt14', 'nt10', 'nt4', 'nt8']
ilds_in: ['nt14_nt9_0', 'nt10_nt9_0', 'nt10_nt9_1', 'nt4_nt9_0', 'nt8_nt9_0', 'nt8_nt9_1']
nt5:
neigbor: ['nt10', 'nt4']
ilds_in: ['nt10_nt5_0', 'np6_nt5_0', 'np6_nt5_1', 'np5_nt5_0', 'nt4_nt5_0', 'nt4_nt5_1']
nt20:
neigbor: ['nt25', 'nt15', 'nt19']
ilds_in: ['nt25_nt20_0', 'np9_nt20_0', 'np9_nt20_1', 'nt15_nt20_0', 'nt19_nt20_0', 'nt19_nt20_1']
nt13:
neigbor: ['nt18', 'nt14', 'nt8', 'nt12']
ilds_in: ['nt18_nt13_0', 'nt14_nt13_0', 'nt14_nt13_1', 'nt8_nt13_0', 'nt12_nt13_0', 'nt12_nt13_1']
nt24:
neigbor: ['nt25', 'nt19', 'nt23']
ilds_in: ['np12_nt24_0', 'nt25_nt24_0', 'nt25_nt24_1', 'nt19_nt24_0', 'nt23_nt24_0', 'nt23_nt24_1']

2021-12-15 18:43:44,880 [INFO] Training: s dim: 300, a dim 167814181, s dim ls: [12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12], a dim ls: [5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5]
2021-12-15 18:43:44.890411: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2
Traceback (most recent call last):
File "C:/Users/lry466/Documents/Viji/TafficLight_GNN/deeprl_signal_control-master/main.py", line 236, in
train(args)
File "C:/Users/lry466/Documents/Viji/TafficLight_GNN/deeprl_signal_control-master/main.py", line 129, in train
seed=0, model_type='lr') #Viji - added env.n_f_ls, param
File "C:\Users\lry466\Documents\Viji\TafficLight_GNN\deeprl_signal_control-master\agents\models.py", line 288, in init
agent_name='{:d}a'.format(i))) #Viji added n_f parameter
File "C:\Users\lry466\Documents\Viji\TafficLight_GNN\deeprl_signal_control-master\agents\models.py", line 306, in _init_policy
policy = LRQPolicy(n_s, n_a, n_w, self.n_step, name=agent_name)
File "C:\Users\lry466\Documents\Viji\TafficLight_GNN\deeprl_signal_control-master\agents\policies.py", line 406, in init
self.qvalues = self._build_net(self.S)
File "C:\Users\lry466\Documents\Viji\TafficLight_GNN\deeprl_signal_control-master\agents\policies.py", line 409, in _build_net
return self._build_fc_net(S, [])
File "C:\Users\lry466\Documents\Viji\TafficLight_GNN\deeprl_signal_control-master\agents\policies.py", line 303, in _build_fc_net
q = fc(h, 'q', self.n_a, act=lambda x: x)
File "C:\Users\lry466\Documents\Viji\TafficLight_GNN\deeprl_signal_control-master\agents\utils.py", line 72, in fc
initializer=init_method(init_scale, init_mode))
File "C:\Users\lry466.conda\envs\ML3dot5Vij\lib\site-packages\tensorflow\python\ops\variable_scope.py", line 1467, in get_variable
aggregation=aggregation)
File "C:\Users\lry466.conda\envs\ML3dot5Vij\lib\site-packages\tensorflow\python\ops\variable_scope.py", line 1217, in get_variable
aggregation=aggregation)
File "C:\Users\lry466.conda\envs\ML3dot5Vij\lib\site-packages\tensorflow\python\ops\variable_scope.py", line 527, in get_variable
aggregation=aggregation)
File "C:\Users\lry466.conda\envs\ML3dot5Vij\lib\site-packages\tensorflow\python\ops\variable_scope.py", line 481, in _true_getter
aggregation=aggregation)
File "C:\Users\lry466.conda\envs\ML3dot5Vij\lib\site-packages\tensorflow\python\ops\variable_scope.py", line 869, in _get_single_variable
"but instead was %s." % (name, shape))
ValueError: Shape of a new variable (lr_0a_q/q/w) must be fully defined, but instead was (?, 5).

Process finished with exit code 1

[TrinhTuanHung2021]

I could run the model but it took me a lot of time. Your command had errors