Release 1.12.0

[neverchanje]

PR	TITLE
XiaoMi/rdsn#332	fix: meta unnecessary assert
XiaoMi/rdsn#290	feat(http): support CPU profiling using gperf
XiaoMi/rdsn#323	refactor: move aio tests out from service_api_c
XiaoMi/rdsn#326	refactor: remove useless functions from binary_reader
XiaoMi/rdsn#327	fix(coldbackup): delay clean request when chkpting
XiaoMi/rdsn#321	feat(http): add http interface for get_app_envs
XiaoMi/rdsn#324	refactor: move replay related codes to mutation_log_replay
XiaoMi/rdsn#317	feat(dup): implement procedure load_from_private_log
XiaoMi/rdsn#314	feat: support table-level slow query on meta server
XiaoMi/rdsn#309	feat(split): child replica learn parent prepare list and checkpoint
XiaoMi/rdsn#315	feat(dup): verify private log validity before starting to duplicate
XiaoMi/rdsn#312	feat(dup): implement ship_mutation stage and mutation_batch
XiaoMi/rdsn#298	feat(throttle): support size-based write throttling
XiaoMi/rdsn#311	refactor: rename disk_aio to aio_context
XiaoMi/rdsn#302	refactor: introduce mutation_log::replay_block
XiaoMi/rdsn#310	refactor: remove empty_aio_provider and posix aio_provider
XiaoMi/rdsn#299	feat(split): parent replica prepare states
XiaoMi/rdsn#304	feat(dup): add interface mutation_duplicator & duplication procedure
XiaoMi/rdsn#307	build: change thrift compiler, set default 3rdlibs
XiaoMi/rdsn#305	add unit tests for task
XiaoMi/rdsn#303	build: remove MY_PROJ_INC_PATH
XiaoMi/rdsn#300	fix(network): use derror rather than dwarn for failed network bootstrap
XiaoMi/rdsn#297	feat(dup): implement duplication_sync on meta server side
XiaoMi/rdsn#291	split: parent replica create child replica
XiaoMi/rdsn#296	http: improvement on http api
XiaoMi/rdsn#295	build: fix dsn.cmake to add thirdparty/output/lib64 into link directories
XiaoMi/rdsn#294	utils: add unlikely for dverify/dreturn/dstop macros
XiaoMi/rdsn#293	utility: enhance fail point functionality
XiaoMi/rdsn#292	dup: complete implementation of replica_duplicator_manager
XiaoMi/rdsn#249	dup: add duplication recovery on meta server
XiaoMi/rdsn#285	CMake: remove MY_PROJ_LIB_PATH
XiaoMi/rdsn#286	split: meta start partition split
XiaoMi/rdsn#280	HTTP service for meta & replica server
XiaoMi/rdsn#289	utils: add test utils on meta server
XiaoMi/rdsn#288	build curl :add without-ssl
XiaoMi/rdsn#287	thirdparty: add curl to help build prometheus
XiaoMi/rdsn#284	add prometheus to thirdparty
XiaoMi/rdsn#281	dup: add duplication_sync_timer
XiaoMi/rdsn#283	*: remove DSN_IN_CORE, improve BOOST linking, fix build.sh
XiaoMi/rdsn#282	minor refactoring on replica-server and meta-server
XiaoMi/rdsn#276	configs: remove useless configs
XiaoMi/rdsn#273	duplication: bugfix on pause_dup
XiaoMi/rdsn#266	http: move http as a submodule of dist
XiaoMi/rdsn#262	Revert "use c++14 mode for compilation (#257)"
XiaoMi/rdsn#257	use c++14 mode for compilation
XiaoMi/rdsn#248	dup: add change_duplication_status on meta_server
XiaoMi/rdsn#246	duplication: add meta_duplication_service (part 3)
XiaoMi/rdsn#245	duplication: add meta_duplication_service (part 2)
XiaoMi/rdsn#244	duplication: add meta_duplication_service (part 1)
XiaoMi/rdsn#242	duplication: add meta_duplication_service (part 0)
XiaoMi/rdsn#237	meta: add duplication_info and refactor test utils
XiaoMi/rdsn#235	ddl_client: add duplication related commands
XiaoMi/rdsn#233	replica: add unit tests for mutation log replay
XiaoMi/rdsn#232	replication: update thrift definitions for duplication

PR	TITLE
#400	feat: table level slow query
#397	build(cmake): fix curl link of prometheus
#398	refactor: change the way to call replication_ddl_client::set_app_envs
#395	feat(shell): add debugging commands for hex and escaped-string conversion
#394	build(cmake): cleanup lib links
#389	build: pack boost.regex with binaries
#386	build: remove MY_PROJ_INC_PATH
#385	fix: unit tests & travis.sh exit with err
#384	fix: unit test failed and update rdsn
#372	benchmark: remove dependency of unexport rocksdb objects
#380	compile: fix dependency error of curl
#379	build: fix travis
#377	fix cmake file; update rdsn
#376	CMake: fix client_lib
#373	CMake: remove MY_PROJ_LIB_PATH, improve client_lib, use rocksdb target to link
#370	configs: minimize config-server.ini to config.min.ini
#358	scripts: travis supports format
#360	HTTP service for replica server
#367	shell & bench: use rocksdb::Statistics to calc histogram
#368	add prometheus for monitor
#362	CMakeList: remove DSN_IN_CORE & improve BOOST linking
#359	update rdsn & fix config
#350	refactor: keep default value in code the same as what it in config.ini
#310	rocksdb: add an option to use rocksdb to reduce intrusive modifications
#306	*: add server config for connection threshold per endpoint
#300	*: add whitelist config & optimize pegasus_offline_node.sh

[neverchanje]

Pegasus v1.12.0-RC1 has released:

• https://github.com/XiaoMi/pegasus/releases/tag/v1.12.0-RC1
• https://github.com/XiaoMi/rdsn/releases/tag/v1.12.0-RC1

[neverchanje]

Release Note

The following are the highlights in this release:

HTTP Support

We exposed many metadata of a Pegasus cluster and some system states of a Pegasus server through HTTP interfaces. Thanks to @Skysheepwang.

Related PR: XiaoMi/rdsn#280, #360, XiaoMi/rdsn#321, XiaoMi/rdsn#296

Related Docs: https://pegasus-kv.github.io/api/http

Multi-tenant Support

In XiaoMi as our user base grows, improvements on our multi-tenant support become increasingly needed for stability. For example, we lack monitoring support for table-level latency (only server-level latency). When a table is observed unreasonably slow, we need to refine our slow query mechanism to dynamically configure the "slow" threshold per table without system reboot. Another requirement is to support size-based write throttling to reduce the influence of a high-throughput-low-QPS table to other tables in the same cluster.

Related PR: XiaoMi/rdsn#314, XiaoMi/rdsn#298, #400, XiaoMi/rdsn#336, XiaoMi/rdsn#340

Docs on table-level latency: TBD

Docs on table-level slow query: TBD

Docs on size-based throttling: TBD

Prometheus Support

Pegasus is continuously lowering our access costs for community users. In this release, we provide our experimental support on Promentheus, thanks to @ChenQShmily.

Related PR: XiaoMi/rdsn#287, XiaoMi/rdsn#284, #397, #368

Related Docs: TBD

CPU Profiling Support

Thanks to @linlinhaohao888.

Related PR: XiaoMi/rdsn#290

Related Docs: TBD

Bug Fixes

• fix: meta unnecessary assert XiaoMi/rdsn#332
• fix(coldbackup): delay clean request when chkpting XiaoMi/rdsn#327

Upgrade from the previous version

No configuration update is needed in this release.

For table-level latency, some new perf-counters are added. Every read/write operation to a table has two perf-counters (p99&p999) for latency.

replica*eon.replica*table.level.RPC_RRDB_RRDB_PUT.latency(ns)@${for.each.table}
replica*eon.replica*table.level.RPC_RRDB_RRDB_PUT.latency(ns)@${for.each.table}.p999
replica*eon.replica*table.level.RPC_RRDB_RRDB_GET.latency(ns)@${for.each.table}
replica*eon.replica*table.level.RPC_RRDB_RRDB_GET.latency(ns)@${for.each.table}.p999

[neverchanje]

PR	TITLE
#411	fix(config): disable auto mem release by default
#416	refactor: remove the assert in get_app_stat
#420	feat(shell): add qps and p99 statistics while list_node
XiaoMi/rdsn#336	feat: add table level latency perf counters
XiaoMi/rdsn#340	fix(bug): resolve static initialization order problem of s_storage_rpc_req_codes

Pegasus v1.12.0-RC2 has released:

• https://github.com/XiaoMi/pegasus/releases/tag/v1.12.0-RC2
• https://github.com/XiaoMi/rdsn/releases/tag/v1.12.0-RC2

[neverchanje]

Pegasus v1.12.0 has released:

https://github.com/XiaoMi/pegasus/releases/tag/v1.12.0
https://github.com/XiaoMi/rdsn/releases/tag/v1.12.0