Skip to content

Commit

Permalink
pack: increase buffer reallocation for higher performance (fix #8492)
Browse files Browse the repository at this point in the history
When converting from msgpack to raw JSON or sds-json, the reallocation
strategy is very convervative to keep the memory usage very low, but
in most of scenarios this lead to more fragmentation and high CPU
usage.

This patch changes the memory reallocation with an exponential
strategy for better performance.

Kudos to @LukasJerabek and @lecaros for helping to troubleshoot and
indentify the root cause of this issue.

Signed-off-by: Eduardo Silva <[email protected]>
  • Loading branch information
edsiper committed Mar 16, 2024
1 parent d0f6310 commit 22f25c5
Showing 1 changed file with 3 additions and 2 deletions.
5 changes: 3 additions & 2 deletions src/flb_pack.c
Original file line number Diff line number Diff line change
Expand Up @@ -807,10 +807,11 @@ flb_sds_t flb_msgpack_raw_to_json_sds(const void *in_buf, size_t in_size)
while (1) {
ret = flb_msgpack_to_json(out_buf, out_size, root);
if (ret <= 0) {
realloc_size *= 2;
tmp_buf = flb_sds_increase(out_buf, realloc_size);
if (tmp_buf) {
out_buf = tmp_buf;
out_size += realloc_size;
out_size *= realloc_size;
}
else {
flb_errno();
Expand Down Expand Up @@ -1168,7 +1169,7 @@ char *flb_msgpack_to_json_str(size_t size, const msgpack_object *obj)
ret = flb_msgpack_to_json(buf, size, obj);
if (ret <= 0) {
/* buffer is small. retry.*/
size += 128;
size *= 2;
tmp = flb_realloc(buf, size);
if (tmp) {
buf = tmp;
Expand Down

0 comments on commit 22f25c5

Please sign in to comment.