Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(NODE-6540): Add c++ zstd compression API #30

Merged
merged 21 commits into from
Nov 19, 2024
Merged

feat(NODE-6540): Add c++ zstd compression API #30

merged 21 commits into from
Nov 19, 2024

Conversation

baileympearson
Copy link
Contributor

@baileympearson baileympearson commented Nov 13, 2024 •
edited
Loading

Description

What is changing?

This PR adds a C++ implementation of our compression library's API.

  • Tooling has been added that downloads and builds zstd into a local directory.
  • The gypfile has been updated to statically link against zstd.
  • compress() and decompress() have been implemented in C++.
  • A test matrix has been added that tests compatibility with the 1.x implementation.

Additionally:

  • A lint task has been added that lints C++ and Js code in CI.
  • Tests run on ubuntu (full matrix coming in a follow-up ticket) across our Node version matrix.
Is there new documentation needed for these changes?

What is the motivation for this change?

Double check the following

  • Ran npm run format:js && npm run format:rs script
  • Self-review completed using the steps outlined here
  • PR title follows the correct format: type(NODE-xxxx)[!]: description
    • Example: feat(NODE-1234)!: rewriting everything in coffeescript
  • Changes are covered by tests
  • New TODOs have a related JIRA ticket

@baileympearson baileympearson changed the base branch from main to NODE-6539 November 13, 2024 05:45
@baileympearson baileympearson changed the base branch from NODE-6539 to main November 13, 2024 22:00
@baileympearson baileympearson changed the base branch from main to NODE-6539 November 13, 2024 22:00
Base automatically changed from NODE-6539 to main November 14, 2024 15:30
@baileympearson baileympearson changed the title migrate to c++ feat(NODE-6540): Add c++ zstd compression API Nov 14, 2024
@baileympearson baileympearson marked this pull request as ready for review November 14, 2024 19:19
W-A-James
W-A-James requested changes Nov 14, 2024
View reviewed changes
addon/zstd.cpp Outdated Show resolved Hide resolved
addon/zstd.cpp Outdated Show resolved Hide resolved
package.json Show resolved Hide resolved
addon/compression_worker.h Outdated Show resolved Hide resolved
addon/decompress.h Outdated Show resolved Hide resolved
addon/compress.h Outdated Show resolved Hide resolved
@nbbeeken nbbeeken added the Primary Review In Review with primary reviewer, not yet ready for team's eyes label Nov 14, 2024
addaleax
addaleax requested changes Nov 14, 2024
View reviewed changes
Copy link

@addaleax addaleax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the many comments, I tried to be somewhat exhaustive in my review

lib/index.js Show resolved Hide resolved
etc/install-zstd.sh Outdated Show resolved Hide resolved
addon/compress.h Outdated Show resolved Hide resolved
addon/compress.h Outdated Show resolved Hide resolved
addon/compression_worker.h Outdated Show resolved Hide resolved
addon/compression_worker.h Outdated Show resolved Hide resolved
addon/zstd.cpp Outdated Show resolved Hide resolved
addon/zstd.cpp Outdated Show resolved Hide resolved
addon/zstd.cpp Outdated Show resolved Hide resolved
addon/compression_worker.h Outdated Show resolved Hide resolved
@baileympearson
Compression namespace, no implementation in header files
bb88035 
move compress and decompress into a single compression namespace and move
the implementations into header files.
@baileympearson
migrate away from creating promise in C++
251003c
@baileympearson
other comments
04030a0
@baileympearson
simplify install in test
e795b4b
baileympearson
baileympearson commented Nov 15, 2024
View reviewed changes
Copy link
Contributor Author

@baileympearson baileympearson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the thorough review!

addon/compression_worker.h Show resolved Hide resolved
addon/napi_utils.h Outdated Show resolved Hide resolved
addon/decompress.h Outdated Show resolved Hide resolved
addon/compression_worker.h Outdated Show resolved Hide resolved
addon/compression_worker.h Outdated Show resolved Hide resolved
addon/compression_worker.h Outdated Show resolved Hide resolved
addon/zstd.cpp Outdated Show resolved Hide resolved
addon/compress.h Outdated Show resolved Hide resolved
addon/decompress.h Outdated Show resolved Hide resolved
lib/index.js Show resolved Hide resolved
W-A-James
W-A-James requested changes Nov 15, 2024
View reviewed changes
Copy link
Contributor

@W-A-James W-A-James left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple small changes

addon/compression_worker.h Show resolved Hide resolved
package.json Show resolved Hide resolved
addaleax
addaleax reviewed Nov 18, 2024
View reviewed changes
.eslintrc.json Outdated Show resolved Hide resolved
addon/zstd.cpp Outdated Show resolved Hide resolved
binding.gyp Outdated Show resolved Hide resolved
addon/compression.cpp Outdated Show resolved Hide resolved
addon/compression_worker.h Outdated Show resolved Hide resolved
addon/compression_worker.h Outdated Show resolved Hide resolved
addon/compression_worker.h Outdated Show resolved Hide resolved
nbbeeken
nbbeeken requested changes Nov 19, 2024
View reviewed changes
etc/install-zstd.sh Outdated Show resolved Hide resolved
lib/index.js Outdated Show resolved Hide resolved
test/index.test.js Show resolved Hide resolved
test/index.test.js Outdated Show resolved Hide resolved
.github/workflows/test.yml Show resolved Hide resolved
addaleax
addaleax requested changes Nov 19, 2024
View reviewed changes
lib/index.js Outdated Show resolved Hide resolved
@W-A-James W-A-James added Team Review Needs review from team and removed Primary Review In Review with primary reviewer, not yet ready for team's eyes labels Nov 19, 2024
@W-A-James W-A-James merged commit 6673245 into main Nov 19, 2024
6 checks passed
@W-A-James W-A-James deleted the migrate-to-c++ branch November 19, 2024 20:38
@github-actions github-actions bot mentioned this pull request Nov 25, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@W-A-James W-A-James W-A-James approved these changes

@nbbeeken nbbeeken nbbeeken approved these changes

@addaleax addaleax addaleax approved these changes

Assignees

@W-A-James W-A-James

Labels
Team Review Needs review from team
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@baileympearson @addaleax @nbbeeken @W-A-James