Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tweaks to evse test plans based on review(issue #31460) #31901

Merged
merged 15 commits into from
Feb 8, 2024
Merged

Tweaks to evse test plans based on review(issue #31460) #31901

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
5bfa5bf
Added support for test event triggers and handling of reading events …
jamesharrow Feb 4, 2024
a5ecc31
Made TC_EEVSE_Utils.py use the matter_testing_support instead of its …
jamesharrow Feb 4, 2024
1699bbe
Restyled by isort
restyled-commits Feb 4, 2024
ae8d396
Added TC_EEVSE_2_2, 2_4, 2_5 to tests.yaml. Fixed compile warning tre…
jamesharrow Feb 4, 2024
c9baf8c
Updated default min_charge and max_charge in TC_EEVSE_Utils send_enab…
jamesharrow Feb 4, 2024
0a820b3
Fixed test app name
jamesharrow Feb 5, 2024
6ba5534
Moved test runs later in test yaml
jamesharrow Feb 5, 2024
bc0e1e0
Fixed discriminator used in script to match that used in the app.
jamesharrow Feb 5, 2024
09d6301
Merge branch 'master' into Tweaks-to-EVSE-Test-plans-based-on-review(…
jamesharrow Feb 5, 2024
40645a3
Added --endpoint 1 so it tests the correct endpoint
jamesharrow Feb 5, 2024
9358584
Code review comment fixes.
jamesharrow Feb 6, 2024
52bdb70
Fixed trailing whitepace
jamesharrow Feb 6, 2024
e707d8b
Merge branch 'master' into Tweaks-to-EVSE-Test-plans-based-on-review(…
jamesharrow Feb 6, 2024
748cdde
Merge branch 'upstream-master' into Tweaks-to-EVSE-Test-plans-based-o…
jamesharrow Feb 8, 2024
64c83b0
Merged TC_EEVSE tests back in
jamesharrow Feb 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/tests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -452,6 +452,7 @@ jobs:
--target linux-x64-all-clusters-ipv6only-no-ble-no-wifi-tsan-clang-test \
--target linux-x64-lock-ipv6only-no-ble-no-wifi-tsan-clang-test \
--target linux-x64-lit-icd-ipv6only-no-ble-no-wifi-tsan-clang-test \
--target linux-x64-energy-management-ipv6only-no-ble-no-wifi-tsan-clang-test \
--target linux-x64-python-bindings \
build \
--copy-artifacts-to objdir-clone \
Expand All @@ -475,6 +476,9 @@ jobs:
scripts/run_in_python_env.sh out/venv './scripts/tests/run_python_test.py --app out/linux-x64-lock-ipv6only-no-ble-no-wifi-tsan-clang-test/chip-lock-app --factoryreset --app-args "--discriminator 1234 --KVS kvs1 --trace-to json:out/trace_data/app-{SCRIPT_BASE_NAME}.json" --script "src/python_testing/TC_DRLK_2_3.py" --script-args "--storage-path admin_storage.json --commissioning-method on-network --discriminator 1234 --passcode 20202021 --PICS src/app/tests/suites/certification/ci-pics-values --trace-to json:out/trace_data/test-{SCRIPT_BASE_NAME}.json --trace-to perfetto:out/trace_data/test-{SCRIPT_BASE_NAME}.perfetto"'
scripts/run_in_python_env.sh out/venv './scripts/tests/run_python_test.py --app out/linux-x64-all-clusters-ipv6only-no-ble-no-wifi-tsan-clang-test/chip-all-clusters-app --factoryreset --app-args "--discriminator 1234 --KVS kvs1 --trace-to json:out/trace_data/app-{SCRIPT_BASE_NAME}.json" --script "src/python_testing/TC_DeviceBasicComposition.py" --script-args "--storage-path admin_storage.json --manual-code 10054912339 --PICS src/app/tests/suites/certification/ci-pics-values --trace-to json:out/trace_data/test-{SCRIPT_BASE_NAME}.json --trace-to perfetto:out/trace_data/test-{SCRIPT_BASE_NAME}.perfetto"'
scripts/run_in_python_env.sh out/venv './scripts/tests/run_python_test.py --app out/linux-x64-lock-ipv6only-no-ble-no-wifi-tsan-clang-test/chip-lock-app --factoryreset --app-args "--discriminator 1234 --KVS kvs1 --trace-to json:out/trace_data/app-{SCRIPT_BASE_NAME}.json" --script "src/python_testing/TC_DeviceConformance.py" --script-args "--storage-path admin_storage.json --manual-code 10054912339 --bool-arg ignore_in_progress:True allow_provisional:True --PICS src/app/tests/suites/certification/ci-pics-values --trace-to json:out/trace_data/test-{SCRIPT_BASE_NAME}.json --trace-to perfetto:out/trace_data/test-{SCRIPT_BASE_NAME}.perfetto --tests test_TC_IDM_10_2"'
scripts/run_in_python_env.sh out/venv './scripts/tests/run_python_test.py --app out/linux-x64-energy-management-ipv6only-no-ble-no-wifi-tsan-clang-test/chip-energy-management-app --factoryreset --app-args "--discriminator 1234 --KVS kvs1 --trace-to json:out/trace_data/app-{SCRIPT_BASE_NAME}.json --enable-key 000102030405060708090a0b0c0d0e0f" --script "src/python_testing/TC_EEVSE_2_2.py" --script-args "--storage-path admin_storage.json --commissioning-method on-network --discriminator 1234 --passcode 20202021 --hex-arg enableKey:000102030405060708090a0b0c0d0e0f --endpoint 1 --trace-to json:out/trace_data/test-{SCRIPT_BASE_NAME}.json --trace-to perfetto:out/trace_data/test-{SCRIPT_BASE_NAME}.perfetto"'
scripts/run_in_python_env.sh out/venv './scripts/tests/run_python_test.py --app out/linux-x64-energy-management-ipv6only-no-ble-no-wifi-tsan-clang-test/chip-energy-management-app --factoryreset --app-args "--discriminator 1234 --KVS kvs1 --trace-to json:out/trace_data/app-{SCRIPT_BASE_NAME}.json --enable-key 000102030405060708090a0b0c0d0e0f" --script "src/python_testing/TC_EEVSE_2_4.py" --script-args "--storage-path admin_storage.json --commissioning-method on-network --discriminator 1234 --passcode 20202021 --hex-arg enableKey:000102030405060708090a0b0c0d0e0f --endpoint 1 --trace-to json:out/trace_data/test-{SCRIPT_BASE_NAME}.json --trace-to perfetto:out/trace_data/test-{SCRIPT_BASE_NAME}.perfetto"'
scripts/run_in_python_env.sh out/venv './scripts/tests/run_python_test.py --app out/linux-x64-energy-management-ipv6only-no-ble-no-wifi-tsan-clang-test/chip-energy-management-app --factoryreset --app-args "--discriminator 1234 --KVS kvs1 --trace-to json:out/trace_data/app-{SCRIPT_BASE_NAME}.json --enable-key 000102030405060708090a0b0c0d0e0f" --script "src/python_testing/TC_EEVSE_2_5.py" --script-args "--storage-path admin_storage.json --commissioning-method on-network --discriminator 1234 --passcode 20202021 --hex-arg enableKey:000102030405060708090a0b0c0d0e0f --endpoint 1 --trace-to json:out/trace_data/test-{SCRIPT_BASE_NAME}.json --trace-to perfetto:out/trace_data/test-{SCRIPT_BASE_NAME}.perfetto"'
scripts/run_in_python_env.sh out/venv './scripts/tests/run_python_test.py --app out/linux-x64-all-clusters-ipv6only-no-ble-no-wifi-tsan-clang-test/chip-all-clusters-app --factoryreset --app-args "--discriminator 1234 --KVS kvs1 --trace-to json:out/trace_data/app-{SCRIPT_BASE_NAME}.json" --script "src/python_testing/TC_FAN_3_1.py" --script-args "--storage-path admin_storage.json --commissioning-method on-network --discriminator 1234 --passcode 20202021 --trace-to json:out/trace_data/test-{SCRIPT_BASE_NAME}.json --trace-to perfetto:out/trace_data/test-{SCRIPT_BASE_NAME}.perfetto"'
scripts/run_in_python_env.sh out/venv './scripts/tests/run_python_test.py --app out/linux-x64-all-clusters-ipv6only-no-ble-no-wifi-tsan-clang-test/chip-all-clusters-app --factoryreset --app-args "--discriminator 1234 --KVS kvs1 --trace-to json:out/trace_data/app-{SCRIPT_BASE_NAME}.json" --script "src/python_testing/TC_FAN_3_2.py" --script-args "--storage-path admin_storage.json --commissioning-method on-network --discriminator 1234 --passcode 20202021 --trace-to json:out/trace_data/test-{SCRIPT_BASE_NAME}.json --trace-to perfetto:out/trace_data/test-{SCRIPT_BASE_NAME}.perfetto"'
scripts/run_in_python_env.sh out/venv './scripts/tests/run_python_test.py --app out/linux-x64-all-clusters-ipv6only-no-ble-no-wifi-tsan-clang-test/chip-all-clusters-app --factoryreset --app-args "--discriminator 1234 --KVS kvs1 --trace-to json:out/trace_data/app-{SCRIPT_BASE_NAME}.json" --script "src/python_testing/TC_FAN_3_3.py" --script-args "--storage-path admin_storage.json --commissioning-method on-network --discriminator 1234 --passcode 20202021 --trace-to json:out/trace_data/test-{SCRIPT_BASE_NAME}.json --trace-to perfetto:out/trace_data/test-{SCRIPT_BASE_NAME}.perfetto"'
Expand Down
5 changes: 3 additions & 2 deletions examples/energy-management-app/energy-management-common/src/EVSEManufacturerImpl.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -232,8 +232,9 @@ void EVSEManufacturer::FakeReadingsUpdate()

// Update meter values
// Avoid using floats - so we will do a basic rand() call which will generate a integer value between 0 and RAND_MAX
// first compute power as a mean + some random value in range 0 to mPowerRandomness_mW
int64_t power = (rand() % gFakeReadingsData.mPowerRandomness_mW);
// first compute power as a mean + some random value in range +/- mPowerRandomness_mW
int64_t power =
(static_cast<int64_t>(rand()) % (2 * gFakeReadingsData.mPowerRandomness_mW)) - gFakeReadingsData.mPowerRandomness_mW;
power += gFakeReadingsData.mPower_mW; // add in the base power

// TODO call the EPM cluster to send a power reading
Expand Down
28 changes: 14 additions & 14 deletions src/python_testing/TC_EEVSE_2_2.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,9 @@

import chip.clusters as Clusters
from chip.clusters.Types import NullValue
from matter_testing_support import MatterBaseTest, TestStep, async_test_body, default_matter_test_main
from matter_testing_support import EventChangeCallback, MatterBaseTest, TestStep, async_test_body, default_matter_test_main
from mobly import asserts
from TC_EEVSE_Utils import EEVSEBaseTestHelper, EventChangeCallback
from TC_EEVSE_Utils import EEVSEBaseTestHelper

logger = logging.getLogger(__name__)

Expand Down Expand Up @@ -123,7 +123,7 @@ async def test_TC_EEVSE_2_2(self):

self.step("4")
await self.send_test_event_trigger_pluggedin()
event_data = events_callback.WaitForEventReport(Clusters.EnergyEvse.Events.EVConnected)
event_data = events_callback.wait_for_event_report(Clusters.EnergyEvse.Events.EVConnected)

self.step("4a")
await self.check_evse_attribute("State", Clusters.EnergyEvse.Enums.StateEnum.kPluggedInNoDemand)
Expand All @@ -147,7 +147,7 @@ async def test_TC_EEVSE_2_2(self):

self.step("6")
await self.send_test_event_trigger_charge_demand()
event_data = events_callback.WaitForEventReport(Clusters.EnergyEvse.Events.EnergyTransferStarted)
event_data = events_callback.wait_for_event_report(Clusters.EnergyEvse.Events.EnergyTransferStarted)

self.step("6a")
await self.check_evse_attribute("State", expected_state)
Expand All @@ -172,7 +172,7 @@ async def test_TC_EEVSE_2_2(self):
# Sleep for the charging duration plus a couple of seconds to check it has stopped
time.sleep(charging_duration + 2)
# check EnergyTransferredStoped (EvseStopped)
event_data = events_callback.WaitForEventReport(Clusters.EnergyEvse.Events.EnergyTransferStopped)
event_data = events_callback.wait_for_event_report(Clusters.EnergyEvse.Events.EnergyTransferStopped)
expected_reason = Clusters.EnergyEvse.Enums.EnergyTransferStoppedReasonEnum.kEVSEStopped
self.validate_energy_transfer_stopped_event(event_data, session_id, expected_state, expected_reason)

Expand All @@ -188,7 +188,7 @@ async def test_TC_EEVSE_2_2(self):
max_charge_current = 12000

await self.send_enable_charge_command(charge_until=charge_until, min_charge=min_charge_current, max_charge=max_charge_current)
event_data = events_callback.WaitForEventReport(Clusters.EnergyEvse.Events.EnergyTransferStarted)
event_data = events_callback.wait_for_event_report(Clusters.EnergyEvse.Events.EnergyTransferStarted)

self.step("8a")
await self.check_evse_attribute("State", Clusters.EnergyEvse.Enums.StateEnum.kPluggedInCharging)
Expand Down Expand Up @@ -228,7 +228,7 @@ async def test_TC_EEVSE_2_2(self):

self.step("10")
await self.send_test_event_trigger_charge_demand_clear()
event_data = events_callback.WaitForEventReport(Clusters.EnergyEvse.Events.EnergyTransferStopped)
event_data = events_callback.wait_for_event_report(Clusters.EnergyEvse.Events.EnergyTransferStopped)
expected_reason = Clusters.EnergyEvse.Enums.EnergyTransferStoppedReasonEnum.kEVStopped
self.validate_energy_transfer_stopped_event(event_data, session_id, expected_state, expected_reason)

Expand All @@ -239,15 +239,15 @@ async def test_TC_EEVSE_2_2(self):
await self.send_test_event_trigger_charge_demand()
# Check we get EnergyTransferStarted again
await self.send_enable_charge_command(charge_until=charge_until, min_charge=min_charge_current, max_charge=max_charge_current)
event_data = events_callback.WaitForEventReport(Clusters.EnergyEvse.Events.EnergyTransferStarted)
event_data = events_callback.wait_for_event_report(Clusters.EnergyEvse.Events.EnergyTransferStarted)
self.validate_energy_transfer_started_event(event_data, session_id, expected_state, expected_max_charge)

self.step("11a")
await self.check_evse_attribute("State", Clusters.EnergyEvse.Enums.StateEnum.kPluggedInCharging)

self.step("12")
await self.send_test_event_trigger_charge_demand_clear()
event_data = events_callback.WaitForEventReport(Clusters.EnergyEvse.Events.EnergyTransferStopped)
event_data = events_callback.wait_for_event_report(Clusters.EnergyEvse.Events.EnergyTransferStopped)
expected_reason = Clusters.EnergyEvse.Enums.EnergyTransferStoppedReasonEnum.kEVStopped
self.validate_energy_transfer_stopped_event(event_data, session_id, expected_state, expected_reason)

Expand All @@ -256,7 +256,7 @@ async def test_TC_EEVSE_2_2(self):

self.step("13")
await self.send_test_event_trigger_pluggedin_clear()
event_data = events_callback.WaitForEventReport(Clusters.EnergyEvse.Events.EVNotDetected)
event_data = events_callback.wait_for_event_report(Clusters.EnergyEvse.Events.EVNotDetected)
expected_state = Clusters.EnergyEvse.Enums.StateEnum.kPluggedInNoDemand
self.validate_ev_not_detected_event(event_data, session_id, expected_state, expected_duration=0, expected_charged=0)

Expand All @@ -280,13 +280,13 @@ async def test_TC_EEVSE_2_2(self):
session_id = session_id + 1

# Check we get a new EVConnected event with updated session ID
event_data = events_callback.WaitForEventReport(Clusters.EnergyEvse.Events.EVConnected)
event_data = events_callback.wait_for_event_report(Clusters.EnergyEvse.Events.EVConnected)
self.validate_ev_connected_event(event_data, session_id)

self.step("14a")
await self.send_test_event_trigger_charge_demand()
expected_state = Clusters.EnergyEvse.Enums.StateEnum.kPluggedInCharging # This is the value at the event time
event_data = events_callback.WaitForEventReport(Clusters.EnergyEvse.Events.EnergyTransferStarted)
event_data = events_callback.wait_for_event_report(Clusters.EnergyEvse.Events.EnergyTransferStarted)
self.validate_energy_transfer_started_event(event_data, session_id, expected_state, expected_max_charge)

self.step("14b")
Expand All @@ -295,7 +295,7 @@ async def test_TC_EEVSE_2_2(self):
self.step("15")
await self.send_disable_command()
expected_state = Clusters.EnergyEvse.Enums.StateEnum.kPluggedInCharging # This is the value prior to stopping
event_data = events_callback.WaitForEventReport(Clusters.EnergyEvse.Events.EnergyTransferStopped)
event_data = events_callback.wait_for_event_report(Clusters.EnergyEvse.Events.EnergyTransferStopped)
expected_reason = Clusters.EnergyEvse.Enums.EnergyTransferStoppedReasonEnum.kEVSEStopped
self.validate_energy_transfer_stopped_event(event_data, session_id, expected_state, expected_reason)

Expand All @@ -307,7 +307,7 @@ async def test_TC_EEVSE_2_2(self):

self.step("17")
await self.send_test_event_trigger_pluggedin_clear()
event_data = events_callback.WaitForEventReport(Clusters.EnergyEvse.Events.EVNotDetected)
event_data = events_callback.wait_for_event_report(Clusters.EnergyEvse.Events.EVNotDetected)
expected_state = Clusters.EnergyEvse.Enums.StateEnum.kPluggedInNoDemand
self.validate_ev_not_detected_event(event_data, session_id, expected_state, expected_duration=0, expected_charged=0)

Expand Down
18 changes: 9 additions & 9 deletions src/python_testing/TC_EEVSE_2_4.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@

import chip.clusters as Clusters
from chip.clusters.Types import NullValue
from matter_testing_support import MatterBaseTest, TestStep, async_test_body, default_matter_test_main
from TC_EEVSE_Utils import EEVSEBaseTestHelper, EventChangeCallback
from matter_testing_support import EventChangeCallback, MatterBaseTest, TestStep, async_test_body, default_matter_test_main
from TC_EEVSE_Utils import EEVSEBaseTestHelper

logger = logging.getLogger(__name__)

Expand Down Expand Up @@ -99,7 +99,7 @@ async def test_TC_EEVSE_2_4(self):

self.step("4")
await self.send_test_event_trigger_pluggedin()
event_data = events_callback.WaitForEventReport(Clusters.EnergyEvse.Events.EVConnected)
event_data = events_callback.wait_for_event_report(Clusters.EnergyEvse.Events.EVConnected)

self.step("4a")
await self.check_evse_attribute("State", Clusters.EnergyEvse.Enums.StateEnum.kPluggedInNoDemand)
Expand All @@ -117,7 +117,7 @@ async def test_TC_EEVSE_2_4(self):

self.step("6")
await self.send_test_event_trigger_charge_demand()
event_data = events_callback.WaitForEventReport(Clusters.EnergyEvse.Events.EnergyTransferStarted)
event_data = events_callback.wait_for_event_report(Clusters.EnergyEvse.Events.EnergyTransferStarted)

self.step("6a")
await self.check_evse_attribute("State", Clusters.EnergyEvse.Enums.StateEnum.kPluggedInCharging)
Expand All @@ -127,7 +127,7 @@ async def test_TC_EEVSE_2_4(self):

self.step("7")
await self.send_test_event_trigger_evse_ground_fault()
event_data = events_callback.WaitForEventReport(Clusters.EnergyEvse.Events.Fault)
event_data = events_callback.wait_for_event_report(Clusters.EnergyEvse.Events.Fault)
expected_state = Clusters.EnergyEvse.Enums.StateEnum.kPluggedInCharging
previous_fault = Clusters.EnergyEvse.Enums.FaultStateEnum.kNoError
current_fault = Clusters.EnergyEvse.Enums.FaultStateEnum.kGroundFault
Expand All @@ -141,7 +141,7 @@ async def test_TC_EEVSE_2_4(self):

self.step("8")
await self.send_test_event_trigger_evse_over_temperature_fault()
event_data = events_callback.WaitForEventReport(Clusters.EnergyEvse.Events.Fault)
event_data = events_callback.wait_for_event_report(Clusters.EnergyEvse.Events.Fault)
expected_state = Clusters.EnergyEvse.Enums.StateEnum.kFault
previous_fault = Clusters.EnergyEvse.Enums.FaultStateEnum.kGroundFault
current_fault = Clusters.EnergyEvse.Enums.FaultStateEnum.kOverTemperature
Expand All @@ -155,7 +155,7 @@ async def test_TC_EEVSE_2_4(self):

self.step("9")
await self.send_test_event_trigger_evse_fault_clear()
event_data = events_callback.WaitForEventReport(Clusters.EnergyEvse.Events.Fault)
event_data = events_callback.wait_for_event_report(Clusters.EnergyEvse.Events.Fault)
expected_state = Clusters.EnergyEvse.Enums.StateEnum.kFault
previous_fault = Clusters.EnergyEvse.Enums.FaultStateEnum.kOverTemperature
current_fault = Clusters.EnergyEvse.Enums.FaultStateEnum.kNoError
Expand All @@ -169,11 +169,11 @@ async def test_TC_EEVSE_2_4(self):

self.step("10")
await self.send_test_event_trigger_charge_demand_clear()
event_data = events_callback.WaitForEventReport(Clusters.EnergyEvse.Events.EnergyTransferStopped)
event_data = events_callback.wait_for_event_report(Clusters.EnergyEvse.Events.EnergyTransferStopped)

self.step("11")
await self.send_test_event_trigger_pluggedin_clear()
event_data = events_callback.WaitForEventReport(Clusters.EnergyEvse.Events.EVNotDetected)
event_data = events_callback.wait_for_event_report(Clusters.EnergyEvse.Events.EVNotDetected)
expected_state = Clusters.EnergyEvse.Enums.StateEnum.kPluggedInNoDemand
self.validate_ev_not_detected_event(event_data, session_id, expected_state, expected_duration=0, expected_charged=0)

Expand Down
4 changes: 2 additions & 2 deletions src/python_testing/TC_EEVSE_2_5.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@
import chip.clusters as Clusters
from chip.clusters.Types import NullValue
from chip.interaction_model import Status
from matter_testing_support import MatterBaseTest, TestStep, async_test_body, default_matter_test_main
from TC_EEVSE_Utils import EEVSEBaseTestHelper, EventChangeCallback
from matter_testing_support import EventChangeCallback, MatterBaseTest, TestStep, async_test_body, default_matter_test_main
from TC_EEVSE_Utils import EEVSEBaseTestHelper

logger = logging.getLogger(__name__)

Expand Down
Loading
Loading