Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Protect config_db.json from minigraph misconfig #1727

Merged
merged 23 commits into from
May 24, 2018
Merged

Protect config_db.json from minigraph misconfig #1727

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
b9f2080
Add noise config for PortChannel & EthernetInterface in simple-sample…
wendani May 16, 2018
696e791
Add noise config for PORTCHANNEL_INTERFACE in simple-sample-graph.xml
wendani May 17, 2018
b5e43d6
Add noice config for DEVICE_NEIGHBOR in t0-sample-graph.xml
wendani May 17, 2018
f3c97e0
DeviceInterfaceLink in minigraph.xml can contain port not existing in
wendani May 17, 2018
6518266
Protect PORTCHANNEL from ports not existing in port_config.ini
wendani May 17, 2018
097f1e9
Protect PORTCHANNEL_INTERFACE from portchannels containing ports not
wendani May 17, 2018
1f293e2
Protect DEVICE_NEIGHBOR from ports not existing in port_config.ini
wendani May 18, 2018
6db9a0a
Add noise config Ethernet1 in DeviceInterfaceLinks in simple-sample-g…
wendani May 18, 2018
987bc92
Add noise config Ethernet1 in DeviceInterfaceLinks in simple-sample-g…
wendani May 18, 2018
9021233
Protect PORTCHANNEL from ports not existing in port_config.ini
wendani May 17, 2018
d604685
Protect PORTCHANNEL_INTERFACE from portchannels containing ports not
wendani May 17, 2018
c9f22b3
Protect DEVICE_NEIGHBOR from ports not existing in port_config.ini
wendani May 18, 2018
955e4be
Correct space in minigraph.py
wendani May 18, 2018
1e79656
Does not allow non-port_config.ini port to get into the port list
wendani May 18, 2018
f545f41
Merge branch 'config_db_protect_master' into sample_graph_master
wendani May 18, 2018
6c3026a
Check PORTCHANNEL against PORT list only if port_config_file exists
wendani May 18, 2018
8caa011
Correct format
wendani May 18, 2018
37f1f81
print warning when a port coming from DeviceInterfaceLink is not in
wendani May 19, 2018
472c499
Change Ethernet1 and 2 to fortyGigE0/1 and 2,respectively
wendani May 22, 2018
8efa34d
Merge branch 'fortyGigE' into sample_graph_master
wendani May 22, 2018
c3c46ba
Change Ethernet1 and 2 to fortyGigE0/1 and 2,respectively
wendani May 22, 2018
75a35fa
Merge branch 'fortyGigE' into sample_graph_master
wendani May 23, 2018
293ac1a
print warning when ignoring ports, portchannels, portchannel interfac…
wendani May 23, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 32 additions & 5 deletions src/sonic-config-engine/minigraph.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -414,7 +414,7 @@ def parse_xml(filename, platform=None, port_config_file=None):
(port_speeds_default, port_descriptions) = parse_deviceinfo(child, hwsku)

current_device = [devices[key] for key in devices if key.lower() == hostname.lower()][0]
results = {}
results = {}
results['DEVICE_METADATA'] = {'localhost': {
'bgp_asn': bgp_asn,
'deployment_id': deployment_id,
Expand Down Expand Up @@ -447,7 +447,6 @@ def parse_xml(filename, platform=None, port_config_file=None):

results['INTERFACE'] = phyport_intfs
results['VLAN_INTERFACE'] = vlan_intfs
results['PORTCHANNEL_INTERFACE'] = pc_intfs

for port_name in port_speeds_default:
# ignore port not in port_config.ini
Expand All @@ -457,9 +456,11 @@ def parse_xml(filename, platform=None, port_config_file=None):
ports.setdefault(port_name, {})['speed'] = port_speeds_default[port_name]

for port_name in port_speed_png:
# if port_name is not in port_config.ini, still consider it.
# and later swss will pick up and behave on-demand port break-up.
# if on-deman port break-up is not supported on a specific platform, swss will return error.
# not consider port not in port_config.ini
if port_name not in ports:
Copy link
Collaborator

@qiluo-msft qiluo-msft May 18, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if port_name not in ports: [](start = 8, length = 26)

Should we treat this as a minigraph generator's bug instead of parser's bug? #Closed

Copy link
Contributor Author

@wendani wendani May 18, 2018 •
edited by qiluo-msft
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we can mitigate that in minigraph, it should be ideal #Closed

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed with Guohan, this is the good.

In reply to: 189410214 [](ancestors = 189410214)

print >> sys.stderr, "Warning: ignore interface '%s' as it is not in the port_config.ini" % port_name
continue

ports.setdefault(port_name, {})['speed'] = port_speed_png[port_name]

for port_name, port in ports.items():
Expand All @@ -474,10 +475,36 @@ def parse_xml(filename, platform=None, port_config_file=None):
ports.setdefault(port_name, {})['description'] = port_descriptions[port_name]

results['PORT'] = ports

if port_config_file:
port_set = set(ports.keys())
for (pc_name, mbr_map) in pcs.items():
# remove portchannels that contain ports not existing in port_config.ini
# when port_config.ini exists
if not set(mbr_map['members']).issubset(port_set):
print >> sys.stderr, "Warning: ignore '%s' as part of its member interfaces is not in the port_config.ini" % pc_name
del pcs[pc_name]

results['PORTCHANNEL'] = pcs


for pc_intf in pc_intfs.keys():
# remove portchannels not in PORTCHANNEL dictionary
if pc_intf[0] not in pcs:
print >> sys.stderr, "Warning: ignore '%s' interface '%s' as '%s' is not in the valid PortChannel list" % (pc_intf[0], pc_intf[1], pc_intf[0])
del pc_intfs[pc_intf]

results['PORTCHANNEL_INTERFACE'] = pc_intfs

results['VLAN'] = vlans
results['VLAN_MEMBER'] = vlan_members

for nghbr in neighbors.keys():
# remove port not in port_config.ini
if nghbr not in ports:
print >> sys.stderr, "Warning: ignore interface '%s' in DEVICE_NEIGHBOR as it is not in the port_config.ini" % nghbr
del neighbors[nghbr]

results['DEVICE_NEIGHBOR'] = neighbors
results['DEVICE_NEIGHBOR_METADATA'] = { key:devices[key] for key in devices if key.lower() != hostname.lower() }
results['SYSLOG_SERVER'] = dict((item, {}) for item in syslog_servers)
Expand Down
52 changes: 51 additions & 1 deletion src/sonic-config-engine/tests/simple-sample-graph.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,11 @@
<AttachTo>fortyGigE0/4</AttachTo>
<SubInterface/>
</PortChannel>
<PortChannel>
<Name>PortChannel1001</Name>
<AttachTo>fortyGigE0/1;fortyGigE0/2</AttachTo>
<SubInterface/>
</PortChannel>
</PortChannelInterfaces>
<VlanInterfaces>
<VlanInterface>
Expand All @@ -147,6 +152,16 @@
<AttachTo>PortChannel01</AttachTo>
<Prefix>FC00::71/126</Prefix>
</IPInterface>
<IPInterface>
<Name i:nil="true"/>
<AttachTo>PortChannel1001</AttachTo>
<Prefix>10.0.0.57/31</Prefix>
</IPInterface>
<IPInterface>
<Name i:Name="true"/>
<AttachTo>PortChannel1001</AttachTo>
<Prefix>FC00::72/126</Prefix>
</IPInterface>
<IPInterface>
<Name i:nil="true"/>
<AttachTo>fortyGigE0/0</AttachTo>
Expand Down Expand Up @@ -193,6 +208,28 @@
<StartPort>fortyGigE0/8</StartPort>
<Validate>true</Validate>
</DeviceLinkBase>
<DeviceLinkBase i:type="DeviceInterfaceLink">
<ElementType>DeviceInterfaceLink</ElementType>
<AutoNegotiation>true</AutoNegotiation>
<Bandwidth>10000</Bandwidth>
<EndDevice>switch-t0</EndDevice>
<EndPort>fortyGigE0/1</EndPort>
<FlowControl>true</FlowControl>
<StartDevice>ARISTA05T1</StartDevice>
<StartPort>Ethernet1/32</StartPort>
<Validate>true</Validate>
</DeviceLinkBase>
<DeviceLinkBase i:type="DeviceInterfaceLink">
<ElementType>DeviceInterfaceLink</ElementType>
<AutoNegotiation>true</AutoNegotiation>
<Bandwidth>10000</Bandwidth>
<EndDevice>switch-t0</EndDevice>
<EndPort>fortyGigE0/2</EndPort>
<FlowControl>true</FlowControl>
<StartDevice>ARISTA06T1</StartDevice>
<StartPort>Ethernet1/33</StartPort>
<Validate>true</Validate>
</DeviceLinkBase>
</DeviceInterfaceLinks>
<Devices>
<Device i:type="ToRRouter">
Expand Down Expand Up @@ -240,7 +277,20 @@
<EnableAutoNegotiation>true</EnableAutoNegotiation>
<EnableFlowControl>true</EnableFlowControl>
<Index>1</Index>
<InterfaceName>fortyGigE0/1</InterfaceName>
<InterfaceName>Ethernet1</InterfaceName>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be fortyGigE0/1

Copy link
Contributor Author

@wendani wendani May 19, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we use fortyGigE0/1 and fortyGigE0/2, even without your patch, it does not introduce extra port into PORT but only change the order listed in PORT. Because of this bad test case in the first place, it does not catch the extra port that can be introduced in later commit 0e7a7dfa.

I will confirm with other places. But here we should use Ethernet1 and Ethernet2.

<InterfaceType i:nil="true"/>
<MultiPortsInterface>false</MultiPortsInterface>
<PortName>0</PortName>
<Priority>0</Priority>
<Speed>10000</Speed>
</a:EthernetInterface>
<a:EthernetInterface>
<ElementType>DeviceInterface</ElementType>
<AlternateSpeeds i:nil="true"/>
<EnableAutoNegotiation>true</EnableAutoNegotiation>
<EnableFlowControl>true</EnableFlowControl>
<Index>1</Index>
<InterfaceName>Ethernet2</InterfaceName>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-> fortyGigE0/2

<InterfaceType i:nil="true"/>
<MultiPortsInterface>false</MultiPortsInterface>
<PortName>0</PortName>
Expand Down
11 changes: 11 additions & 0 deletions src/sonic-config-engine/tests/t0-sample-graph.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -338,6 +338,17 @@
<StartDevice>switch-t0</StartDevice>
<StartPort>fortyGigE0/124</StartPort>
</DeviceLinkBase>
<DeviceLinkBase>
<ElementType>DeviceInterfaceLink</ElementType>
<AutoNegotiation>true</AutoNegotiation>
<Bandwidth>10000</Bandwidth>
<EndDevice>switch-t0</EndDevice>
<EndPort>fortyGigE0/2</EndPort>
<FlowControl>true</FlowControl>
<StartDevice>ARISTA05T1</StartDevice>
<StartPort>Ethernet1/33</StartPort>
<Validate>true</Validate>
</DeviceLinkBase>
</DeviceInterfaceLinks>
<Devices>
<Device i:type="ToRRouter">
Expand Down
11 changes: 11 additions & 0 deletions src/sonic-config-engine/tests/test_cfggen.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,8 @@ def test_minigraph_acl(self):
argument = '-m "' + self.sample_graph_t0 + '" -p "' + self.port_config + '" -v ACL_TABLE'
output = self.run_script(argument, True)
self.assertEqual(output.strip(), "Warning: Ignoring Control Plane ACL NTP_ACL without type\n"
"Warning: ignore interface 'fortyGigE0/2' as it is not in the port_config.ini\n"
"Warning: ignore interface 'fortyGigE0/2' in DEVICE_NEIGHBOR as it is not in the port_config.ini\n"
"{'SSH_ACL': {'services': ['SSH'], 'type': 'CTRLPLANE', 'policy_desc': 'SSH_ACL'},"
" 'SNMP_ACL': {'services': ['SNMP'], 'type': 'CTRLPLANE', 'policy_desc': 'SNMP_ACL'},"
" 'DATAACL': {'type': 'L3', 'policy_desc': 'DATAACL', 'ports': ['Ethernet112', 'Ethernet116', 'Ethernet120', 'Ethernet124']},"
Expand Down Expand Up @@ -130,6 +132,15 @@ def test_minigraph_neighbors(self):
output = self.run_script(argument)
self.assertEqual(output.strip(), "{'name': 'ARISTA04T1', 'port': 'Ethernet1/1'}")

def test_minigraph_extra_neighbors(self):
argument = '-m "' + self.sample_graph_t0 + '" -p "' + self.port_config + '" -v DEVICE_NEIGHBOR'
output = self.run_script(argument)
self.assertEqual(output.strip(), \
"{'Ethernet116': {'name': 'ARISTA02T1', 'port': 'Ethernet1/1'}, "
"'Ethernet124': {'name': 'ARISTA04T1', 'port': 'Ethernet1/1'}, "
"'Ethernet112': {'name': 'ARISTA01T1', 'port': 'Ethernet1/1'}, "
"'Ethernet120': {'name': 'ARISTA03T1', 'port': 'Ethernet1/1'}}")

def test_minigraph_bgp(self):
argument = '-m "' + self.sample_graph_bgp_speaker + '" -p "' + self.port_config + '" -v "BGP_NEIGHBOR[\'10.0.0.59\']"'
output = self.run_script(argument)
Expand Down