Add OTBR snap test #40

MonicaisHer · 2024-02-02T12:36:56Z

This PR adds tests and enables GitHub action workflow for OTBR snap.

Testing locally

Run the following command:

$ cd tests/
$ INFRA_IF="eno1" go test -v -failfast -count 1
2024/02/14 15:12:38 [CLEAN]
2024/02/14 15:12:38 [exec] sudo snap remove --purge openthread-border-router
2024/02/14 15:12:38 [stderr] snap "openthread-border-router" is not installed
2024/02/14 15:12:38 [SETUP]
2024/02/14 15:12:38 [exec] sudo snap install openthread-border-router --channel=latest/edge
2024/02/14 15:12:41 [stdout] openthread-border-router (edge) thread-reference-20230119+snap from Canonical IoT Labs installed
2024/02/14 15:12:41 [exec] sudo snap connect openthread-border-router:avahi-control 
2024/02/14 15:12:42 [exec] sudo snap connect openthread-border-router:firewall-control 
2024/02/14 15:12:42 [exec] sudo snap connect openthread-border-router:raw-usb 
2024/02/14 15:12:42 [exec] sudo snap connect openthread-border-router:network-control 
2024/02/14 15:12:43 [exec] sudo snap connect openthread-border-router:bluetooth-control 
2024/02/14 15:12:44 [exec] sudo snap connect openthread-border-router:bluez 
2024/02/14 15:12:44 [exec] sudo cp ot-rcp-simulator-thread-reference-20230119-amd64 /var/snap/openthread-border-router/common/
2024/02/14 15:12:44 [exec] sudo snap set openthread-border-router radio-url=''spinel+hdlc+forkpty:///var/snap/openthread-border-router/common/ot-rcp-simulator-thread-reference-20230119-amd64?forkpty-arg=1''
2024/02/14 15:12:45 [exec] sudo snap set openthread-border-router infra-if='eno1'
=== RUN   TestSetup
    exec.go:32: [exec] sudo snap get openthread-border-router infra-if
    exec.go:32: [exec] sudo snap stop --disable openthread-border-router
    exec.go:118: [stdout] Stopped.
2024/02/14 15:12:45 [exec] sudo snap start --enable openthread-border-router
2024/02/14 15:12:46 [stdout] Started.
=== RUN   TestSetup/Firewall_rule
    exec.go:32: [exec] sudo iptables -S FORWARD | grep "comment OTBR"
=== RUN   TestSetup/IP_forwarding
    exec.go:32: [exec] sudo sysctl net.ipv6.conf.all.forwarding
    exec.go:32: [exec] sudo sysctl net.ipv4.ip_forward
=== RUN   TestSetup/RT_tables_for_backbone_router
    exec.go:32: [exec] sudo sysctl net.core.optmem_max
=== RUN   TestSetup/Random_fwmark_bits
    exec.go:32: [exec] sudo iptables -t mangle -L PREROUTING -n -v | grep OTBR
    exec.go:32: [exec] sudo iptables -t nat -L POSTROUTING -n -v | grep OTBR
=== RUN   TestSetup/Firewall_rule_setup_for_Infrastructure_interface
    exec.go:32: [exec] sudo iptables -t filter -L FORWARD -n -v | grep OTBR | grep eno1
=== RUN   TestSetup/Border_routing
    exec.go:32: [exec] sudo sysctl net.ipv6.conf.eno1.accept_ra
    exec.go:32: [exec] sudo sysctl net.ipv6.conf.eno1.accept_ra_rt_info_max_plen
=== NAME  TestSetup
    exec.go:32: [exec] sudo snap stop --disable openthread-border-router
    exec.go:118: [stdout] Stopped.
--- PASS: TestSetup (1.69s)
    --- PASS: TestSetup/Firewall_rule (0.01s)
    --- PASS: TestSetup/IP_forwarding (0.02s)
    --- PASS: TestSetup/RT_tables_for_backbone_router (0.01s)
    --- PASS: TestSetup/Random_fwmark_bits (0.02s)
    --- PASS: TestSetup/Firewall_rule_setup_for_Infrastructure_interface (0.02s)
    --- PASS: TestSetup/Border_routing (0.02s)
=== RUN   TestSocketFile
    exec.go:32: [exec] sudo snap stop --disable openthread-border-router
    exec.go:118: [stdout] Stopped.
2024/02/14 15:12:47 [exec] sudo snap start --enable openthread-border-router
2024/02/14 15:12:48 [stdout] Started.
    snap_test.go:102: Retry 1/10: Waiting for file creation: /run/snap.openthread-border-router/openthread-wpan0.sock
    snap_test.go:102: Socket File exists in: /run/snap.openthread-border-router/openthread-wpan0.sock
    exec.go:32: [exec] sudo snap stop --disable openthread-border-router
    exec.go:118: [stdout] Stopped.
--- PASS: TestSocketFile (2.54s)
=== RUN   TestSnapServicesStatus
    exec.go:32: [exec] sudo snap stop --disable openthread-border-router
    exec.go:118: [stdout] Stopped.
2024/02/14 15:12:49 [exec] sudo snap start --enable openthread-border-router
2024/02/14 15:12:50 [stdout] Started.
    exec.go:32: [exec] snap services openthread-border-router.otbr-setup | awk 'FNR == 2 {print $3}'
    exec.go:118: [stdout] inactive
    exec.go:32: [exec] snap services openthread-border-router.otbr-web | awk 'FNR == 2 {print $3}'
    exec.go:118: [stdout] active
    exec.go:32: [exec] snap services openthread-border-router.otbr-agent | awk 'FNR == 2 {print $3}'
    exec.go:118: [stdout] active
    exec.go:32: [exec] sudo snap stop --disable openthread-border-router
    exec.go:118: [stdout] Stopped.
--- PASS: TestSnapServicesStatus (1.67s)
=== RUN   TestConfig
    exec.go:32: [exec] sudo snap stop --disable openthread-border-router
    exec.go:118: [stdout] Stopped.
=== RUN   TestConfig/Set_infra-if
    exec.go:32: [exec] sudo snap stop --disable openthread-border-router
    exec.go:118: [stdout] Stopped.
    exec.go:32: [exec] sudo snap set openthread-border-router infra-if='wpan1'
    snap_test.go:138: [exec] sudo snap start openthread-border-router
    config.go:59: Retry 1/10: Waiting for expected content in logs: INFRA_IF=wpan1
    exec.go:32: [exec] sudo journalctl --since "2024-02-14 15:12:51" --no-pager | grep "openthread-border-router.otbr-setup"|| true
    config.go:63: Found expected content in logs: INFRA_IF=wpan1
    exec.go:32: [exec] sudo snap set openthread-border-router infra-if='eno1'
    exec.go:32: [exec] sudo snap stop --disable openthread-border-router
    exec.go:118: [stdout] Stopped.
=== RUN   TestConfig/Set_radio-url
    exec.go:32: [exec] sudo snap stop --disable openthread-border-router
    exec.go:118: [stdout] Stopped.
    exec.go:32: [exec] sudo snap set openthread-border-router radio-url='spinel+hdlc+uart:///dev/ttyACM1'
    snap_test.go:145: [exec] sudo snap start openthread-border-router
    config.go:59: Retry 1/10: Waiting for expected content in logs: RADIO_URL=spinel+hdlc+uart:///dev/ttyACM1
    exec.go:32: [exec] sudo journalctl --since "2024-02-14 15:12:54" --no-pager | grep "openthread-border-router.otbr-agent"|| true
    config.go:63: Found expected content in logs: RADIO_URL=spinel+hdlc+uart:///dev/ttyACM1
    exec.go:32: [exec] sudo snap set openthread-border-router radio-url='spinel+hdlc+uart:///dev/ttyACM0'
    exec.go:32: [exec] sudo snap stop --disable openthread-border-router
    exec.go:118: [stdout] Stopped.
=== RUN   TestConfig/Set_invalid_thread_interface
    snap_test.go:160: [exec] sudo snap set openthread-border-router thread-if=wpan1
    exec.go:32: [exec] sudo snap set openthread-border-router thread-if='wpan0'
    exec.go:32: [exec] sudo snap stop --disable openthread-border-router
    exec.go:118: [stdout] Stopped.
=== RUN   TestConfig/Set_webgui-listen-address
    exec.go:32: [exec] sudo snap set openthread-border-router webgui-listen-address='192.168.178.1'
    exec.go:32: [exec] sudo snap start --enable openthread-border-router
    exec.go:118: [stdout] Started.
    exec.go:32: [exec] curl 192.168.178.1:80
    exec.go:32: [exec] sudo snap set openthread-border-router webgui-listen-address='::'
    exec.go:32: [exec] sudo snap stop --disable openthread-border-router
    exec.go:118: [stdout] Stopped.
=== RUN   TestConfig/Set_webgui-port
    exec.go:32: [exec] sudo lsof -nPi :90 || true
    net.go:202: Port 90 is available.
    exec.go:32: [exec] sudo snap set openthread-border-router webgui-port='90'
2024/02/14 15:13:00 [exec] sudo snap start --enable openthread-border-router
2024/02/14 15:13:01 [stdout] Started.
    net.go:96: Retry 1/10: Waiting for ports: 90
    net.go:96: Retry 2/10: Waiting for ports: 90
    exec.go:32: [exec] curl localhost:90
    exec.go:32: [exec] sudo snap set openthread-border-router webgui-port='80'
    exec.go:32: [exec] sudo snap stop --disable openthread-border-router
    exec.go:118: [stdout] Stopped.
=== RUN   TestConfig/Set_autostart
    exec.go:32: [exec] snap services openthread-border-router | awk 'FNR == 2 {print $2}'
    exec.go:118: [stdout] disabled
    exec.go:32: [exec] snap services openthread-border-router | awk 'FNR == 2 {print $3}'
    exec.go:118: [stdout] inactive
    exec.go:32: [exec] sudo snap set openthread-border-router autostart='true'
    exec.go:32: [exec] snap services openthread-border-router.otbr-agent | awk 'FNR == 2 {print $2}'
    exec.go:118: [stdout] enabled
    exec.go:32: [exec] snap services openthread-border-router.otbr-agent | awk 'FNR == 2 {print $3}'
    exec.go:118: [stdout] active
    exec.go:32: [exec] snap services openthread-border-router.otbr-web | awk 'FNR == 2 {print $2}'
    exec.go:118: [stdout] enabled
    exec.go:32: [exec] snap services openthread-border-router.otbr-web | awk 'FNR == 2 {print $3}'
    exec.go:118: [stdout] active
    exec.go:32: [exec] snap services openthread-border-router.otbr-setup | awk 'FNR == 2 {print $2}'
    exec.go:118: [stdout] enabled
    exec.go:32: [exec] snap services openthread-border-router.otbr-setup | awk 'FNR == 2 {print $3}'
    exec.go:118: [stdout] inactive
    exec.go:32: [exec] sudo snap stop --disable openthread-border-router
    exec.go:118: [stdout] Stopped.
=== NAME  TestConfig
    exec.go:32: [exec] sudo snap stop --disable openthread-border-router
    exec.go:118: [stdout] Stopped.
--- PASS: TestConfig (13.77s)
    --- PASS: TestConfig/Set_infra-if (2.63s)
    --- PASS: TestConfig/Set_radio-url (2.94s)
    --- PASS: TestConfig/Set_invalid_thread_interface (0.85s)
    --- PASS: TestConfig/Set_webgui-listen-address (2.06s)
    --- PASS: TestConfig/Set_webgui-port (2.82s)
    --- PASS: TestConfig/Set_autostart (1.76s)
=== RUN   TestThreadNetworkFormation
    exec.go:32: [exec] sudo snap stop --disable openthread-border-router
    exec.go:118: [stdout] Stopped.
2024/02/14 15:13:05 [exec] sudo snap start --enable openthread-border-router
2024/02/14 15:13:06 [stdout] Started.
    exec.go:32: [exec] sudo openthread-border-router.ot-ctl dataset init new
    exec.go:32: [exec] sudo openthread-border-router.ot-ctl dataset commit active
    exec.go:32: [exec] sudo openthread-border-router.ot-ctl ifconfig up
    exec.go:32: [exec] sudo openthread-border-router.ot-ctl thread start
    config.go:59: Retry 1/10: Waiting for expected content in logs: Thread Network
    exec.go:32: [exec] sudo journalctl --since "2024-02-14 15:13:05" --no-pager | grep "openthread-border-router"|| true
    config.go:59: Retry 2/10: Waiting for expected content in logs: Thread Network
    exec.go:32: [exec] sudo journalctl --since "2024-02-14 15:13:05" --no-pager | grep "openthread-border-router"|| true
    config.go:59: Retry 3/10: Waiting for expected content in logs: Thread Network
    exec.go:32: [exec] sudo journalctl --since "2024-02-14 15:13:05" --no-pager | grep "openthread-border-router"|| true
    config.go:59: Retry 4/10: Waiting for expected content in logs: Thread Network
    exec.go:32: [exec] sudo journalctl --since "2024-02-14 15:13:05" --no-pager | grep "openthread-border-router"|| true
    config.go:59: Retry 5/10: Waiting for expected content in logs: Thread Network
    exec.go:32: [exec] sudo journalctl --since "2024-02-14 15:13:05" --no-pager | grep "openthread-border-router"|| true
    config.go:59: Retry 6/10: Waiting for expected content in logs: Thread Network
    exec.go:32: [exec] sudo journalctl --since "2024-02-14 15:13:05" --no-pager | grep "openthread-border-router"|| true
    config.go:59: Retry 7/10: Waiting for expected content in logs: Thread Network
    exec.go:32: [exec] sudo journalctl --since "2024-02-14 15:13:05" --no-pager | grep "openthread-border-router"|| true
    config.go:59: Retry 8/10: Waiting for expected content in logs: Thread Network
    exec.go:32: [exec] sudo journalctl --since "2024-02-14 15:13:05" --no-pager | grep "openthread-border-router"|| true
    config.go:63: Found expected content in logs: Thread Network
    exec.go:32: [exec] sudo openthread-border-router.ot-ctl state | head -n 1
    exec.go:32: [exec] sudo openthread-border-router.ot-ctl thread stop
    exec.go:32: [exec] sudo snap stop --disable openthread-border-router
    exec.go:118: [stdout] Stopped.
--- PASS: TestThreadNetworkFormation (10.39s)
PASS
2024/02/14 15:13:15 [TEARDOWN]
2024/02/14 15:13:15 [exec] (sudo journalctl --since "2024-02-14 15:12:38" --no-pager | grep "openthread-border-router"|| true) > openthread-border-router.log
Wrote snap logs to /home/mengyi/Desktop/883/openthread-border-router-snap/tests/openthread-border-router.log
2024/02/14 15:13:15 Removing installed snap: true
2024/02/14 15:13:15 [exec] sudo snap remove --purge openthread-border-router
2024/02/14 15:13:20 [stdout] openthread-border-router removed
ok      openthread-border-router-snap-tests     41.966s

tests/ot-rcp-simulator-thread-reference-20230119-amd64

MonicaisHer · 2024-02-09T08:55:58Z

The snap test failed on GitHub Action because the error, chain 'OTBR_FORWARD_INGRESS' does not exist, preventing OTBR snap from starting. Here are the logs from Github Action testing:

openthread-border-router.otbr-setup[5302]: + ip6tables -C FORWARD -o wpan0 -j OTBR_FORWARD_INGRESS
openthread-border-router.otbr-setup[5310]: ip6tables v1.8.7 (nf_tables): Chain 'OTBR_FORWARD_INGRESS' does not exist

This error doesn't exist when running the test locally. Here are the logs and iptables version:

openthread-border-router.otbr-setup[157853]: + ip6tables -C FORWARD -o wpan0 -j OTBR_FORWARD_INGRESS
openthread-border-router.otbr-setup[157861]: OTBR_FORWARD_INGRESS  all opt    in * out wpan0  ::/0  -> ::/0

$ iptables --version
iptables v1.8.7 (nf_tables)

It might be possible that the iptables and the GitHub Actions container environments are incompatible. If this is the case, a potential solution for this could be to use the 'iptables-nft' package instead of 'iptables'.

References:

farshidtz · 2024-02-09T15:27:40Z

The snap test failed on GitHub Action because the error, chain 'OTBR_FORWARD_INGRESS' does not exist, preventing OTBR snap from starting. Here are the logs from Github Action testing:
openthread-border-router.otbr-setup[5302]: + ip6tables -C FORWARD -o wpan0 -j OTBR_FORWARD_INGRESS
openthread-border-router.otbr-setup[5310]: ip6tables v1.8.7 (nf_tables): Chain 'OTBR_FORWARD_INGRESS' does not exist
This error doesn't exist when running the test locally. Here are the logs and iptables version:
openthread-border-router.otbr-setup[157853]: + ip6tables -C FORWARD -o wpan0 -j OTBR_FORWARD_INGRESS
openthread-border-router.otbr-setup[157861]: OTBR_FORWARD_INGRESS  all opt    in * out wpan0  ::/0  -> ::/0
$ iptables --version
iptables v1.8.7 (nf_tables)
It might be possible that the iptables and the GitHub Actions container environments are incompatible. If this is the case, a potential solution for this could be to use the 'iptables-nft' package instead of 'iptables'.

References:

Docker doesn't work with iptables v1.8.1 moby/moby#38099 (comment)

Problem with tproxy in docker container when different iptables are present on host and container kumahq/kuma#6334

https://manpages.ubuntu.com/manpages/jammy/en/man8/iptables-nft.8.html

The test isn't failing because of that. The given message isn't even an error. It is the output of ip6tables -C which checks for existence of a rule. This happens because the firewall_stop function is always called before setting up the firewall to check and cleanup the environment. In this case, there is nothing to remove because the chain hasn't been created yet.

The logs, from the artifact of the linked test, show the failure clearly:

Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5266]: + echo 'Setup the firewall'
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5266]: Setup the firewall
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5266]: + /snap/openthread-border-router/x1/bin/script/otbr-firewall start
Feb 05 17:01:51 fv-az1382-620 audit[5306]: AVC apparmor="DENIED" operation="exec" class="file" profile="snap.openthread-border-router.otbr-setup" name="/usr/bin/systemctl" pid=5306 comm="otbr-firewall" requested_mask="x" denied_mask="x" fsuid=0 ouid=0
Feb 05 17:01:51 fv-az1382-620 audit[5306]: AVC apparmor="DENIED" operation="open" class="file" profile="snap.openthread-border-router.otbr-setup" name="/usr/bin/systemctl" pid=5306 comm="otbr-firewall" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Feb 05 17:01:51 fv-az1382-620 audit[5308]: AVC apparmor="DENIED" operation="exec" class="file" profile="snap.openthread-border-router.otbr-setup" name="/usr/bin/plymouth" pid=5308 comm="otbr-firewall" requested_mask="x" denied_mask="x" fsuid=0 ouid=0
Feb 05 17:01:51 fv-az1382-620 audit[5308]: AVC apparmor="DENIED" operation="open" class="file" profile="snap.openthread-border-router.otbr-setup" name="/usr/bin/plymouth" pid=5308 comm="otbr-firewall" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5302]: + case "$1" in
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5302]: + firewall_start
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5302]: + firewall_stop
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5302]: + ip6tables -C FORWARD -o wpan0 -j OTBR_FORWARD_INGRESS
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5310]: ip6tables v1.8.7 (nf_tables): Chain 'OTBR_FORWARD_INGRESS' does not exist
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5310]: Try `ip6tables -h' or 'ip6tables --help' for more information.
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5302]: + ip6tables -L OTBR_FORWARD_INGRESS
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5311]: ip6tables v1.8.7 (nf_tables): chain `OTBR_FORWARD_INGRESS' in table `filter' is incompatible, use 'nft' tool.
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5302]: + ipset_destroy_if_exist otbr-ingress-deny-src
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5302]: + ipset list otbr-ingress-deny-src
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5312]: ipset v7.15: The set with the given name does not exist
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5302]: + ipset_destroy_if_exist otbr-ingress-deny-src-swap
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5302]: + ipset list otbr-ingress-deny-src-swap
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5314]: ipset v7.15: The set with the given name does not exist
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5302]: + ipset_destroy_if_exist otbr-ingress-allow-dst
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5302]: + ipset list otbr-ingress-allow-dst
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5315]: ipset v7.15: The set with the given name does not exist
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5302]: + ipset_destroy_if_exist otbr-ingress-allow-dst-swap
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5302]: + ipset list otbr-ingress-allow-dst-swap
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5316]: ipset v7.15: The set with the given name does not exist
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5302]: + ipset create -exist otbr-ingress-deny-src hash:net family inet6
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5302]: + ipset create -exist otbr-ingress-deny-src-swap hash:net family inet6
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5302]: + ipset create -exist otbr-ingress-allow-dst hash:net family inet6
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5302]: + ipset create -exist otbr-ingress-allow-dst-swap hash:net family inet6
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5302]: + ip6tables -N OTBR_FORWARD_INGRESS
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5302]: + ip6tables -I FORWARD 1 -o wpan0 -j OTBR_FORWARD_INGRESS
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5302]: + ip6tables -A OTBR_FORWARD_INGRESS -m pkttype --pkt-type unicast -i wpan0 -j DROP
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5302]: + ip6tables -A OTBR_FORWARD_INGRESS -m set --match-set otbr-ingress-deny-src src -j DROP
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5302]: + ip6tables -A OTBR_FORWARD_INGRESS -m set --match-set otbr-ingress-allow-dst dst -j ACCEPT
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5302]: + ip6tables -A OTBR_FORWARD_INGRESS -m pkttype --pkt-type unicast -j DROP
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5302]: + ip6tables -A OTBR_FORWARD_INGRESS -j ACCEPT
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5266]: + export PLATFORM=ubuntu
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5266]: + PLATFORM=ubuntu
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5266]: + source /snap/openthread-border-router/x1/bin/_initrc_install
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5266]: ++ set -euxo pipefail
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5266]: ++ [[ ! -n x ]]
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5266]: ++ echo 'Current platform is ubuntu'
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5266]: Current platform is ubuntu
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5266]: + echo 'Setup IP forwarding'
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5266]: Setup IP forwarding
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5266]: + sysctl -w net.ipv6.conf.all.forwarding=1
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5333]: net.ipv6.conf.all.forwarding = 1
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5266]: + sysctl -w net.ipv4.ip_forward=1
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5334]: net.ipv4.ip_forward = 1
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5266]: + echo 'Setup RT Tables for the Backbone Router'
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5266]: Setup RT Tables for the Backbone Router
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5266]: + sysctl net.core.optmem_max=65536
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5335]: net.core.optmem_max = 65536
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5266]: + echo 'Setup NAT44'
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5266]: Setup NAT44
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5266]: + echo 'Set random fwmark bits'
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5266]: Set random fwmark bits
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5266]: + iptables -t mangle -A PREROUTING -i wpan0 -j MARK --set-mark 0x1001 -m comment --comment OTBR
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5266]: + iptables -t nat -A POSTROUTING -m mark --mark 0x1001 -j MASQUERADE -m comment --comment OTBR
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5266]: + echo 'Setup NAT44: Setting firewall rule for wlan0'
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5266]: Setup NAT44: Setting firewall rule for wlan0
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5266]: + iptables -t filter -A FORWARD -o wlan0 -j ACCEPT -m comment --comment OTBR
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5266]: + iptables -t filter -A FORWARD -i wlan0 -j ACCEPT -m comment --comment OTBR
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5266]: + echo 'Setup Border Routing'
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5266]: Setup Border Routing
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5266]: + sysctl -w net.ipv6.conf.wlan0.accept_ra=2
Feb 05 17:01:51 fv-az1382-620 openthread-border-router.otbr-setup[5342]: sysctl: cannot stat /proc/sys/net/ipv6/conf/wlan0/accept_ra: No such file or directory
Feb 05 17:01:51 fv-az1382-620 systemd[1]: snap.openthread-border-router.otbr-setup.service: Main process exited, code=exited, status=255/EXCEPTION
Feb 05 17:01:51 fv-az1382-620 systemd[1]: snap.openthread-border-router.otbr-setup.service: Failed with result 'exit-code'.

The error is sysctl: cannot stat /proc/sys/net/ipv6/conf/wlan0/accept_ra: No such file or directory which has an obvious cause: there is no wlan0 interface on the server.

farshidtz

It is a good start with decent coverage. See inline comments.

Also, snap options aren't being tested explicitly, i.e. testing of the configure hook.

.github/workflows/build-and-test-snap.yml

tests/snap_test.go

.github/workflows/build-and-test-snap.yml

- Update workflow name - Update module name

Improve naming conventions

tests/snap_test.go

farshidtz

Good refactoring and fixes. The testing coverage is reasonable. Thanks.

I've added some comments, which can mostly be taken care of in future work. In that case, please add issues to document them.

tests/snap_services_test.go

farshidtz · 2024-02-15T15:47:00Z

tests/snap_services_test.go

+	utils.SnapStart(nil, otbrSnap)
+
+	// Oneshot service
+	require.False(t, utils.SnapServicesActive(t, otbrSetupApp))
+
+	// Actice services
+	require.True(t, utils.SnapServicesActive(t, otbrWebApp))
+	require.True(t, utils.SnapServicesActive(t, otbrAgentApp))
+}


This starts the services and checks their status. The snap start operation returns once the oneshot setup service exits. These are all operations that are taken care of by snapd. So in a sense, only the successful completion of the oneshot service is being tested here.

What could make this test more useful is to check if the services complete their startup in a reasonable amount of time (by looking for some reference in the logs), or if they fail.

Issue created: #42

farshidtz · 2024-02-15T15:57:38Z

tests/config_test.go

+	command := "sudo snap start openthread-border-router"
+	_, _ = exec.Command("/bin/bash", "-c", command).CombinedOutput()
+	t.Logf("[exec] %s", command)
+	utils.WaitForLogMessage(t, otbrService, expectedLog, start)


Why not use utils.SnapStart?

As explained above

farshidtz · 2024-02-15T15:58:16Z

tests/config_test.go

+
+	utils.SnapSet(t, otbrSnap, configKey, configValue)
+	command := "sudo snap start openthread-border-router"
+	_, _ = exec.Command("/bin/bash", "-c", command).CombinedOutput()


No error checking.

The checkSnapOption() function sometimes sets an invalid config value, such as invalid values for infra-if and radio-url. Therefore, in this case, the test does not handle the error as an error is expected. This is also the reason why it doesn't use Utils.Snapstart(), as the error would be fatal. In this subtest, the crucial aspect is whether the config value has been accepted by the application, rather than whether it is valid.

tests/config_test.go

farshidtz · 2024-02-15T16:01:22Z

tests/config_test.go

+	utils.WaitForLogMessage(t, otbrService, expectedLog, start)
+}


This matches the prinout from the scripts because xtrace is enabled. Instead, it should be checking the application logs to verify if the configuration has been effective.

Issue created: #43

MonicaisHer added 2 commits February 2, 2024 13:29

Tracking and adding LFS artifacts

7bf8e01

Add OTBR snap test (socket, snap services, thread network)

eef5e06

farshidtz reviewed Feb 2, 2024

View reviewed changes

tests/ot-rcp-simulator-thread-reference-20230119-amd64 Outdated Show resolved Hide resolved

MonicaisHer added 3 commits February 2, 2024 17:35

Explain RCP simulator building, update helper function

d903d6e

Add OTBR snap test (setup)

d9b77d6

Add GH workflow for testing

5d6bba2

MonicaisHer added 3 commits February 9, 2024 17:10

Debug - list GitHub Action network interfaces

a99f622

Set network interface for GH

2b0600f

Improve format, fix hardcoded INFRA_IF

1af74a0

MonicaisHer force-pushed the IENG-883 branch from 6706852 to 1af74a0 Compare February 12, 2024 09:31

Make infrastructure interface configurable

3ec89e4

MonicaisHer marked this pull request as ready for review February 12, 2024 14:19

farshidtz requested changes Feb 12, 2024

View reviewed changes

MonicaisHer added 3 commits February 12, 2024 17:45

Trim INFRA_IF, and remove obsolete subtests

da81ec4

- Update workflow name - Update module name

Update const and var names, remove obsolete start

0ea3024

Add explicit tests for snap options, declare time as a local variable

1804e18

Improve naming conventions

MonicaisHer commented Feb 14, 2024

View reviewed changes

tests/snap_test.go Outdated Show resolved Hide resolved

MonicaisHer added 3 commits February 15, 2024 11:41

Update configValue for webgui-listen-address

6fec459

Split main test file to subtest files

33f7bb1

Replace curl with utils.WaitServiceOnline()

ed6f5c4

MonicaisHer requested a review from farshidtz February 15, 2024 14:50

farshidtz approved these changes Feb 15, 2024

View reviewed changes

Update function name, fix typo

a1b041f

This was referenced Feb 16, 2024

Improve service startup test to check the completion #42

Closed

Improve config test to check application logs instead of from syslog #43

Closed

Add comments for intended unhandled error

d1db7fb

farshidtz merged commit e1e3839 into canonical:main Feb 16, 2024
2 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add OTBR snap test #40

Add OTBR snap test #40

MonicaisHer commented Feb 2, 2024 •

edited

Loading

MonicaisHer commented Feb 9, 2024 •

edited

Loading

farshidtz commented Feb 9, 2024

farshidtz left a comment

farshidtz left a comment

farshidtz Feb 15, 2024

MonicaisHer Feb 16, 2024

farshidtz Feb 15, 2024

MonicaisHer Feb 16, 2024

farshidtz Feb 15, 2024

MonicaisHer Feb 16, 2024

farshidtz Feb 15, 2024

MonicaisHer Feb 16, 2024

		utils.WaitForLogMessage(t, otbrService, expectedLog, start)
		}

Add OTBR snap test #40

Add OTBR snap test #40

Conversation

MonicaisHer commented Feb 2, 2024 • edited Loading

Testing locally

MonicaisHer commented Feb 9, 2024 • edited Loading

farshidtz commented Feb 9, 2024

farshidtz left a comment

Choose a reason for hiding this comment

farshidtz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MonicaisHer commented Feb 2, 2024 •

edited

Loading

MonicaisHer commented Feb 9, 2024 •

edited

Loading