Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stream optimisations #2123

Merged
merged 10 commits into from
Oct 24, 2020
Merged

Conversation

mikee47
Copy link
Contributor

@mikee47 mikee47 commented Oct 23, 2020

  • Simplify Multipart streams

    • Remove nextStream and make stream private.
    • Remove onNextStream method and implement logic within MultipartStream::getNextStream.
    • Deprecate HttpPartResult, rename as MultipartStream::BodyPart.
    • Deprecate HttpPartProducerDelegate, rename as MultipartStream::Producer.
  • Wiring Stream

    • Apply coding style to Stream.h and regularise comments
    • Move trivial Stream methods into header
    • Apply coding style, simplify and tidy up Stream.cpp
    • Also add a couple of extra null ptr checks
    • Stream find methods should take const char*
  • Fix LimitedMemoryStream, and add getStreamPointer method

    • Attempting to write data larger than available space discards all of it.
    • Correct behaviour is to write as much as possible, but still discard the remainder.
    • Make seek operation (for reading) is bi-directional between the start of the buffer and the current write position.
  • Add serialization speed checks in HostTests

@mikee47 mikee47 mentioned this pull request Oct 23, 2020
6 tasks
@slaff slaff added this to the 4.2.0 milestone Oct 23, 2020
@mikee47
Copy link
Contributor Author

mikee47 commented Oct 23, 2020

HostTests has added speed tests using Json6. Here's a summary of the changes running on a NodeMCU.

Times are in microseconds:

  • loadBuffer
Before After
1059 1063
868 868
860 868
860 868
  • loadFlashString
Before After
2386 2342
2267 2240
2267 2236
2267 2236
  • loadFlashString via Stream (cached)
Before After
3665 3656
3444 3439
3444 3440
3444 3440
  • loadFlashString via Stream (un-cached)
Before After
9238 9242
9203 9187
9207 9191
9203 9187
  • loadFile
Before After
4877 4621
4306 4068
4263 4023
4263 4019
  • loadStream
Before After
4154 3903
4134 3872
4127 3872
4127 3872

@slaff

This comment has been minimized.

@mikee47 mikee47 force-pushed the feature/stream-optimisations branch 2 times, most recently from 3f455c1 to 24da373 Compare October 23, 2020 12:24
Remove `nextStream` and make `stream` private.
Remove `onNextStream` method and implement logic within MultipartStream::getNextStream.
Deprecate `HttpPartResult`, rename as `MultipartStream::BodyPart`.
Depreate `HttpPartProducerDelegate`, rename as `MultipartStream::Producer`.
# Conflicts:
#	Sming/Wiring/Stream.h
Also add a couple of extra null ptr checks
Attempting to write data larger than available space discards all of it.
Correct behaviour is to write as much as possible, but still discard the remainder.

Also, seek operation (for reading) is now bi-directional between the start of the buffer and the current write position.
@mikee47 mikee47 force-pushed the feature/stream-optimisations branch from 24da373 to 2e89b92 Compare October 23, 2020 12:56
@mikee47 mikee47 force-pushed the feature/stream-optimisations branch from 2e89b92 to 7d1f68d Compare October 23, 2020 14:42
@slaff
Copy link
Contributor

slaff commented Oct 24, 2020

@mikee47 Is this PR ready for merging?

@mikee47
Copy link
Contributor Author

mikee47 commented Oct 24, 2020

@slaff Yes, all done

@slaff slaff merged commit 7b759af into SmingHub:develop Oct 24, 2020
@mikee47 mikee47 deleted the feature/stream-optimisations branch October 24, 2020 15:20
slaff pushed a commit to mikee47/Sming that referenced this pull request Dec 2, 2020
…in wrong multipart boudary headers and endless loop.

This commit removes the introduced problems.
slaff pushed a commit to mikee47/Sming that referenced this pull request Dec 2, 2020
…in wrong multipart boudary headers and endless loop.

This commit removes the introduced problems.
slaff pushed a commit to mikee47/Sming that referenced this pull request Dec 3, 2020
…in wrong multipart boudary headers and endless loop.

This commit removes the introduced problems.
@slaff slaff removed the 3 - Review label Oct 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants