Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge master to rotating-id-generation #17

Merged
merged 25 commits into from
Feb 10, 2021

Conversation

hnnajh
Copy link
Owner

@hnnajh hnnajh commented Feb 10, 2021

Problem

<<<<<FILL ME IN - the issue this PR is intended to address>>>>>

Summary of Changes

<<<<<FILL ME IN - what's in this PR>>>>>

Fixes #<<<<<FILL ME IN - issue number(s). If no issue, please create one>>>>>

kpschoedel and others added 25 commits February 10, 2021 10:14
* BufferWriter revisions

#### Problem

Followup from PR #4677 _Remove BufBound_
(commit 2d48e75)

#### Summary of Changes

- Revised BufferWriter to add a non-endian version, which only allows
  writing bytes, strings, and blobs.
- Meanwhile, out-lined nontrivial BufferWriter methods to save flash
  (1K+ on current builds).
- Revised PacketBufferWriter to provide the same endianness options.

* forgot to commit new file

* single inheritance

* restyle
* Define a chip-repl script that is intended to provide an interactive console against the python interfaces that chip provides

* Make GetAdapters work on chip repl

* Restyle fixes
…fer.

We have lots of code that basically does encode/decode starting at the
data start of the packet buffer, and bounded to its data length.  We
can make this less error-prone by just defining a helper that does
this.
Get rid of tautological/broken conditions and unused.
variables. Unify the logic for setting Status LED across
applications and platforms.
Bumps [cryptography](https://github.com/pyca/cryptography) from 3.3.1 to 3.3.2.
- [Release notes](https://github.com/pyca/cryptography/releases)
- [Changelog](https://github.com/pyca/cryptography/blob/master/CHANGELOG.rst)
- [Commits](pyca/cryptography@3.3.1...3.3.2)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Since #4401 has been merged, the test pairing/secret is
added to the session manager only if rendezvous is bypassed.
However, we would like to keep the possibility to test
nRF Connect examples with and without the full rendezvous
procedure without the need to recompile a project with
different settings.

Add a setting CHIP_DEVICE_CONFIG_USE_TEST_PAIRING which
allows to initialize the session manager with the test
secret even if rendezvous bypass mode is not used.
* Add Silabs SDK as a submodule for backward compatibility

- Update README

* Restyled by shellharden

* Restyled by shfmt

* update gn_efr32_example script

Co-authored-by: Restyled.io <[email protected]>
* Update SmartThings reviewers

* Update formatting
* Move ChipExceptions into chip.exeptions for python

* Restyle fixes
Use more portable way to find python 3 in the system and
pass the path to GN which uses python to run various helper
scripts.
The reason is that on different operating systems `python`
name, which GN defaults to, may point to either python2 or
python3 or it may not exist at all. On Windows, on the other
hand, python 3 is distributed as python.exe executable, so
`python3` name also could not be used for that purpose.
* Fix and test heap allocation of PacketBuffers

#### Problem

CHIP has to option to use heap-allocated packet buffers
(with `CHIP_SYSTEM_CONFIG_PACKETBUFFER_POOL_SIZE == 0`) but
this configuration has been neglected, and CHIP neither builds
nor runs in this configuration.

#### Summary of Changes

Build fixes:

- Default `CHIP_CONFIG_RMP_RETRANS_TABLE_SIZE` to 15 for heap case.
    15 is the default packet buffer pool size in SystemConfig.h.
    See also issue #4335 - Our definition of the CRMP retransmission
    table size does not really make sense

Runtime fixes:

- Ensure echo server has enough buffer space for the response (and added
  `CHECK_RETURN_VALUE` to `EnsureReservedSize()`).
- Added room for MAC in echo requester.
- Added room for MAC in `chipSendUnicast()`.
- Added parameters to `CloneData()` to request header and trailing
  space.
- Fixed CHIPMem initialization for some tests.
- Fix tests that didn't allocate enough space.

Build and code changes to assist memory troubleshooting (these should have
no cost unless enabled):

- Added a build flag `chip_config_memory_debug_checks` and corresponding
  preprocessor definition `CHIP_CONFIG_MEMORY_DEBUG_CHECKS` to enable extra
  memory checks, at a performance cost. In particular, new functions
  `Platform::MemoryDebugCheckPointer()` and `PacketBufferHandle::Check()`
  operate when this is enabled.
- Added a build flag `chip_config_memory_debug_dmalloc` and corresponding
  preprocessor definition `CHIP_CONFIG_MEMORY_DEBUG_DMALLOC` to use the
  dmalloc debugging malloc library.
- Clarified conditions for the four configurations of PacketBuffer allocation.
- Replaced some PacketBuffer #define constants with scoped constexprs.
- Renamed `CHIP_SYSTEM_CONFIG_PACKETBUFFER_MAXALLOC` to
  `CHIP_SYSTEM_CONFIG_PACKETBUFFER_POOL_SIZE` to remove the ambiguity of
  what a ‘maximum allocation’ is.

Build and automated test changes:

- Made standalone builds default to heap allocation.
- Run a set of unit tests with dmalloc.

fixes #4395 - Heap packet buffer allocation is broken
fixes #4390 - ReliableMessageProtocol assumes a buffer pool

* review

* review 2

* fix messaging shutdown

* review 3

* fix tests

Co-authored-by: Justin Wood <[email protected]>
…4710)

* Use Default Critical (Non-Critical) Marks for CHIP Cert Extensions.

The following certificate extensions, when present, MUST be marked as critical:
  - basic constraints extension
  - key usage extension
  - extended key usage extension

The following certificate extensions, when present, MUST be marked as non-critical:
  - subject key identifier extension
  - authority key identifier extension

These rules should always be enforced when processing X.509 representation of the certificate.
In the CHIP certificate representation the "critical" field is omitted.

Ticket: #4708

* Fixed typo: extention --> extension

* Apply suggestions from code review

Co-authored-by: Boris Zbarsky <[email protected]>

Co-authored-by: Boris Zbarsky <[email protected]>
* [controller] Deprecate old DeviceController in python binding

* Cleanup
…4722)

Regular disconnection after network provisioning is treated
as BLE error and confusing message is printed.

* Added checking if connection is disconnected after successful
provisioning to detect whether it was intended.
* Move tlv implementation into a separate module for python

* Add missed files from previous commit
@hnnajh hnnajh merged commit 9327125 into hnnajh:rotating-id-generation Feb 10, 2021
hnnajh pushed a commit that referenced this pull request Jul 22, 2022
It's not safe to access line editing state from the IO thread while
inside readline() on the main thread.

Remove the code that attempts to redraw readline after printing logs.
This avoids segfaults during logging at the cost of those logs
overwriting the prompt (this is not trivial to fix as readline
is a blocking API).

==================
WARNING: ThreadSanitizer: data race (pid=63005)
  Write of size 1 at 0x55f81c7745ff by main thread:
    #0 InteractiveStartCommand::ParseCommand(char*) ../../examples/chip-tool/commands/interactive/InteractiveCommands.cpp:127 (chip-tool+0x874911)
    #1 InteractiveStartCommand::RunCommand() ../../examples/chip-tool/commands/interactive/InteractiveCommands.cpp:85 (chip-tool+0x874594)
    #2 CHIPCommand::StartWaiting(std::chrono::duration<unsigned int, std::ratio<1l, 1000l> >) ../../examples/chip-tool/commands/common/CHIPCommand.cpp:408 (chip-tool+0x83e478)
    #3 CHIPCommand::Run() ../../examples/chip-tool/commands/common/CHIPCommand.cpp:187 (chip-tool+0x83c839)
    #4 Commands::RunCommand(int, char**, bool) ../../examples/chip-tool/commands/common/Commands.cpp:147 (chip-tool+0x85d4f7)
    #5 Commands::Run(int, char**) ../../examples/chip-tool/commands/common/Commands.cpp:51 (chip-tool+0x85c288)
    #6 main <null> (chip-tool+0x569c0a)

  Previous read of size 1 at 0x55f81c7745ff by thread T5 (mutexes: write M185):
    #0 LoggingCallback ../../examples/chip-tool/commands/interactive/InteractiveCommands.cpp:46 (chip-tool+0x874479)
    #1 chip::Logging::LogV(unsigned char, unsigned char, char const*, __va_list_tag*) ../../src/lib/support/logging/CHIPLogging.cpp:221 (chip-tool+0x8ee4dc)
    #2 chip::Logging::Log(unsigned char, unsigned char, char const*, ...) ../../src/lib/support/logging/CHIPLogging.cpp:172 (chip-tool+0x8ee30a)
    #3 chip::app::ReadClient::RefreshLivenessCheckTimer() <null> (chip-tool+0x8b1746)
    #4 chip::app::ReadClient::ProcessSubscribeResponse(chip::System::PacketBufferHandle&&) ../../src/app/ReadClient.cpp:845 (chip-tool+0x8b20ec)
    #5 chip::app::ReadClient::OnMessageReceived(chip::Messaging::ExchangeContext*, chip::PayloadHeader const&, chip::System::PacketBufferHandle&&) ../../src/app/ReadClient.cpp:409 (chip-tool+0x8ae2a4)
    #6 chip::Messaging::ExchangeContext::HandleMessage(unsigned int, chip::PayloadHeader const&, chip::BitFlags<chip::Messaging::MessageFlagValues, unsigned int>, chip::System::PacketBufferHandle&&) <null> (chip-tool+0xa0517a)
    #7 operator()<chip::Messaging::ExchangeContext> ../../src/messaging/ExchangeMgr.cpp:219 (chip-tool+0xa08c73)
    #8 Call ../../src/lib/support/Pool.h:126 (chip-tool+0xa0912d)
    #9 chip::internal::HeapObjectList::ForEachNode(void*, chip::Loop (*)(void*, void*)) ../../src/lib/support/Pool.cpp:127 (chip-tool+0x8ee05a)
    #10 ForEachActiveObject<chip::Messaging::ExchangeManager::OnMessageReceived(const chip::PacketHeader&, const chip::PayloadHeader&, const chip::SessionHandle&, chip::SessionMessageDelegate::DuplicateMessage, chip::System::PacketBufferHandle&&)::<lambda(auto:2*)> > ../../src/lib/support/Pool.h:396 (chip-tool+0xa08d10)
    #11 chip::Messaging::ExchangeManager::OnMessageReceived(chip::PacketHeader const&, chip::PayloadHeader const&, chip::SessionHandle const&, chip::SessionMessageDelegate::DuplicateMessage, chip::System::PacketBufferHandle&&) ../../src/messaging/ExchangeMgr.cpp:212 (chip-tool+0xa07e91)
    #12 chip::SessionManager::SecureUnicastMessageDispatch(chip::PacketHeader const&, chip::Transport::PeerAddress const&, chip::System::PacketBufferHandle&&) ../../src/transport/SessionManager.cpp:616 (chip-tool+0xa1548b)
    #13 chip::SessionManager::OnMessageReceived(chip::Transport::PeerAddress const&, chip::System::PacketBufferHandle&&) ../../src/transport/SessionManager.cpp:443 (chip-tool+0xa14426)
    #14 chip::TransportMgrBase::HandleMessageReceived(chip::Transport::PeerAddress const&, chip::System::PacketBufferHandle&&) ../../src/transport/TransportMgrBase.cpp:76 (chip-tool+0xa17dfa)
    #15 chip::Transport::Base::HandleMessageReceived(chip::Transport::PeerAddress const&, chip::System::PacketBufferHandle&&) ../../src/transport/raw/Base.h:102 (chip-tool+0xb19728)
    #16 chip::Transport::UDP::OnUdpReceive(chip::Inet::UDPEndPoint*, chip::System::PacketBufferHandle&&, chip::Inet::IPPacketInfo const*) ../../src/transport/raw/UDP.cpp:122 (chip-tool+0xb1a48b)
    #17 chip::Inet::UDPEndPointImplSockets::HandlePendingIO(chip::BitFlags<chip::System::SocketEventFlags, unsigned char>) ../../src/inet/UDPEndPointImplSockets.cpp:688 (chip-tool+0xb00aa0)
    #18 chip::Inet::UDPEndPointImplSockets::HandlePendingIO(chip::BitFlags<chip::System::SocketEventFlags, unsigned char>, long) ../../src/inet/UDPEndPointImplSockets.cpp:569 (chip-tool+0xafff89)
    #19 chip::System::LayerImplSelect::HandleEvents() ../../src/system/SystemLayerImplSelect.cpp:406 (chip-tool+0xb07563)
    #20 chip::DeviceLayer::Internal::GenericPlatformManagerImpl_POSIX<chip::DeviceLayer::PlatformManagerImpl>::_RunEventLoop() ../../src/include/platform/internal/GenericPlatformManagerImpl_POSIX.ipp:181 (chip-tool+0x98a227)
    #21 chip::DeviceLayer::PlatformManager::RunEventLoop() ../../src/include/platform/PlatformManager.h:362 (chip-tool+0x988f75)
    #22 chip::DeviceLayer::Internal::GenericPlatformManagerImpl_POSIX<chip::DeviceLayer::PlatformManagerImpl>::EventLoopTaskMain(void*) ../../src/include/platform/internal/GenericPlatformManagerImpl_POSIX.ipp:205 (chip-tool+0x98a87c)

  Location is global '(anonymous namespace)::gIsCommandRunning' of size 1 at 0x55f81c7745ff (chip-tool+0x000000c485ff)

  Mutex M185 (0x55f81c776180) created at:
    #0 pthread_mutex_lock ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:4240 (libtsan.so.0+0x4f30a)
    #1 chip::DeviceLayer::Internal::GenericPlatformManagerImpl_POSIX<chip::DeviceLayer::PlatformManagerImpl>::_LockChipStack() ../../src/include/platform/internal/GenericPlatformManagerImpl_POSIX.ipp:78 (chip-tool+0x989e90)
    #2 chip::DeviceLayer::PlatformManager::LockChipStack() ../../src/include/platform/PlatformManager.h:410 (chip-tool+0x988fa5)
    #3 chip::DeviceLayer::Internal::GenericPlatformManagerImpl_POSIX<chip::DeviceLayer::PlatformManagerImpl>::_RunEventLoop() ../../src/include/platform/internal/GenericPlatformManagerImpl_POSIX.ipp:170 (chip-tool+0x98a147)
    #4 chip::DeviceLayer::PlatformManager::RunEventLoop() ../../src/include/platform/PlatformManager.h:362 (chip-tool+0x988f75)
    #5 chip::DeviceLayer::Internal::GenericPlatformManagerImpl_POSIX<chip::DeviceLayer::PlatformManagerImpl>::EventLoopTaskMain(void*) ../../src/include/platform/internal/GenericPlatformManagerImpl_POSIX.ipp:205 (chip-tool+0x98a87c)

  Thread T5 (tid=63013, running) created by main thread at:
    #0 pthread_create ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:969 (libtsan.so.0+0x5ad75)
    #1 chip::DeviceLayer::Internal::GenericPlatformManagerImpl_POSIX<chip::DeviceLayer::PlatformManagerImpl>::_StartEventLoopTask() ../../src/include/platform/internal/GenericPlatformManagerImpl_POSIX.ipp:231 (chip-tool+0x98a40a)
    #2 chip::DeviceLayer::PlatformManager::StartEventLoopTask() ../../src/include/platform/PlatformManager.h:375 (chip-tool+0xaacca2)
    #3 chip::Controller::DeviceControllerFactory::ServiceEvents() ../../src/controller/CHIPDeviceControllerFactory.cpp:331 (chip-tool+0xab0417)
    #4 CHIPCommand::StartWaiting(std::chrono::duration<unsigned int, std::ratio<1l, 1000l> >) ../../examples/chip-tool/commands/common/CHIPCommand.cpp:403 (chip-tool+0x83e353)
    #5 CHIPCommand::Run() ../../examples/chip-tool/commands/common/CHIPCommand.cpp:187 (chip-tool+0x83c839)
    #6 Commands::RunCommand(int, char**, bool) ../../examples/chip-tool/commands/common/Commands.cpp:147 (chip-tool+0x85d4f7)
    #7 Commands::Run(int, char**) ../../examples/chip-tool/commands/common/Commands.cpp:51 (chip-tool+0x85c288)
    #8 main <null> (chip-tool+0x569c0a)

SUMMARY: ThreadSanitizer: data race ../../examples/chip-tool/commands/interactive/InteractiveCommands.cpp:127 in InteractiveStartCommand::ParseCommand(char*)
==================
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.