Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Service Factory + Individual Services' Factory Implementations #257

Merged
merged 7 commits into from
Jun 2, 2020
Merged

Service Factory + Individual Services' Factory Implementations #257

merged 7 commits into from
Jun 2, 2020

Conversation

andrewchoi5
Copy link

@andrewchoi5 andrewchoi5 commented Jun 1, 2020 •
edited
Loading

Service Factory + Individual Services' Factory Implementations

Because of the early design, architectural decisions of Xinfra Monitor, the KafkaMonitor class enforces individual Xinfra Monitor Services to carry the same set of constructor parameters.

This is a large caveat as not all Service classes don't need the identical constructor arguments.

This is not enforced the codebase. It is partially and only silently enforced by convention.

  1. Unmaintainable code + non-robust code.
  2. Non-evolvable code: each Service's constructor implementations aren't allowed to be evolved independently. 3. Change(s) in one Service's constructor absolutely forces the other Services' constructors to also change its parameters, unfortunately.

Signed off by : Andrew Choi

Andrew Choi added 2 commits May 31, 2020 22:10
init
c882deb 
Signed-off-by: Andrew Choi <[email protected]>
initial commit
58a4aeb 
Signed-off-by: Andrew Choi <[email protected]>
@andrewchoi5 andrewchoi5 mentioned this pull request Jun 1, 2020
Merged
@andrewchoi5 andrewchoi5 self-assigned this Jun 1, 2020
@andrewchoi5 andrewchoi5 requested a review from smccauliff June 1, 2020 06:07
@andrewchoi5 andrewchoi5 added the New Feature New Feature Request label Jun 1, 2020
@andrewchoi5 andrewchoi5 added this to the Improvements milestone Jun 1, 2020
@andrewchoi5 andrewchoi5 requested a review from hzxa21 June 1, 2020 06:27
andrewchoi5 pushed a commit that referenced this pull request Jun 2, 2020
Ensure KafkaMetricsReporterSerivce carries the correct constructor pa…
8c1ea3a 
…rameters.

Ensure KafkaMetricsReporterSerivce carries the correct constructor parameters.
Abstract out the helper methods with constructorContainsClass
The way Service Instantiations are done within Kafka Monitor will be updated with #257.
testing:

	produce.treat.zero.throughput.as.unavailable = true
	topic = kafka-monitor-topic1
	zookeeper.connect = localhost:2181
 (com.linkedin.kmf.services.configs.ProduceServiceConfig)
[2020-05-29 14:34:57,457] INFO produce-service/ProduceService is initialized. (com.linkedin.kmf.services.ProduceService)
[2020-05-29 14:34:57,468] INFO KafkaMetricsReporterServiceConfig values:
	bootstrap.servers = localhost:9092
	report.interval.sec = 3
	report.kafka.topic.replication.factor = 1
	report.metrics.list = [kmf.services:type=produce-service,name=*:produce-availability-avg, kmf.services:type=consume-service,name=*:consume-availability-avg, kmf.services:type=produce-service,name=*:records-produced-total, kmf.services:type=consume-service,name=*:records-consumed-total, kmf.services:type=consume-service,name=*:records-lost-total, kmf.services:type=consume-service,name=*:records-duplicated-total, kmf.services:type=consume-service,name=*:records-delay-ms-avg, kmf.services:type=produce-service,name=*:records-produced-rate, kmf.services:type=produce-service,name=*:produce-error-rate, kmf.services:type=consume-service,name=*:consume-error-rate]
	topic = kafka-monitor-topic1
	zookeeper.connect = localhost:2181
 (com.linkedin.kmf.services.configs.KafkaMetricsReporterServiceConfig)
[2020-05-29 14:34:57,694] INFO CreateTopicsResult: {kafka-monitor-topic1=KafkaFuture{value=null,exception=null,done=true}}. (com.linkedin.kmf.common.Utils)
[2020-05-29 14:34:57,694] INFO Created monitoring topic kafka-monitor-topic1 in cluster with 1 partitions and replication factor of 1. (com.linkedin.kmf.common.Utils)
[2020-05-29 14:34:57,694] INFO Completed the topic creation if it doesn't exist for kafka-monitor-topic1. (com.linkedin.kmf.common.Utils)
[2020-05-29 14:34:57,706] INFO produce-service/ProduceService started (com.linkedin.kmf.services.ProduceService)
[2020-05-29 14:34:57,707] INFO reporter-kafka-service/KafkaMetricsReporterService has started. (com.linkedin.kmf.services.KafkaMetricsReporterService)
[2020-05-29 14:34:57,707] INFO Xinfra Monitor (KafkaMonitor) started. (com.linkedin.kmf.KafkaMonitor)
[2020-05-29 14:35:00,722] INFO Kafka Metrics Reporter sending metrics = {
  "kmf.services:name=produce-service,type=produce-service:records-produced-rate" : "0.4514446227929374",
  "kmf.services:name=produce-service,type=produce-service:records-produced-total" : "27.0",
  "kmf.services:name=produce-service,type=produce-service:produce-availability-avg" : "1.0",
  "kmf.services:name=produce-service,type=produce-service:produce-error-rate" : "0.0"
} (com.linkedin.kmf.services.KafkaMetricsReporterService)
[2020-05-29 14:35:03,708] INFO Kafka Metrics Reporter sending metrics = {
  "kmf.services:name=produce-service,type=produce-service:records-produced-rate" : "0.9363765571440515",
  "kmf.services:name=produce-service,type=produce-service:records-produced-total" : "56.0",
  "kmf.services:name=produce-service,type=produce-service:produce-availability-avg" : "1.0",
  "kmf.services:name=produce-service,type=produce-service:produce-error-rate" : "0.0"
} (com.linkedin.kmf.services.KafkaMetricsReporterService)
[2020-05-29 14:35:06,708] INFO Kafka Metrics Reporter sending metrics = {
  "kmf.services:name=produce-service,type=produce-service:records-produced-rate" : "1.4045648357160772",
  "kmf.services:name=produce-service,type=produce-service:records-produced-total" : "84.0",
  "kmf.services:name=produce-service,type=produce-service:produce-availability-avg" : "1.0",
  "kmf.services:name=produce-service,type=produce-service:produce-error-rate" : "0.0"
} (com.linkedin.kmf.services.KafkaMetricsReporterService)
[2020-05-29 14:35:09,709] INFO Kafka Metrics Reporter sending metrics = {
  "kmf.services:name=produce-service,type=produce-service:records-produced-rate" : "1.8894425308497476",
  "kmf.services:name=produce-service,type=produce-service:records-produced-total" : "113.0",
  "kmf.services:name=produce-service,type=produce-service:produce-availability-avg" : "1.0",
  "kmf.services:name=produce-service,type=produce-service:produce-error-rate" : "0.0"
} (com.linkedin.kmf.services.KafkaMetricsReporterService)
[2020-05-29 14:35:12,712] INFO Kafka Metrics Reporter sending metrics = {
  "kmf.services:name=produce-service,type=produce-service:records-produced-rate" : "2.3742246150244948",
  "kmf.services:name=produce-service,type=produce-service:records-produced-total" : "142.0",
  "kmf.services:name=produce-service,type=produce-service:produce-availability-avg" : "1.0",
  "kmf.services:name=produce-service,type=produce-service:produce-error-rate" : "0.0"
} (com.linkedin.kmf.services.KafkaMetricsReporterService)
[2020-05-29 14:35:15,709] INFO Kafka Metrics Reporter sending metrics = {
  "kmf.services:name=produce-service,type=produce-service:records-produced-rate" : "2.842571691330156",
  "kmf.services:name=produce-service,type=produce-service:records-produced-total" : "170.0",
  "kmf.services:name=produce-service,type=produce-service:produce-availability-avg" : "1.0",
  "kmf.services:name=produce-service,type=produce-service:produce-error-rate" : "0.0"
} (com.linkedin.kmf.services.KafkaMetricsReporterService)
f[2020-05-29 14:35:18,712] INFO Kafka Metrics Reporter sending metrics = {
  "kmf.services:name=produce-service,type=produce-service:records-produced-rate" : "3.3273140716960943",
  "kmf.services:name=produce-service,type=produce-service:records-produced-total" : "199.0",
  "kmf.services:name=produce-service,type=produce-service:produce-availability-avg" : "1.0",
  "kmf.services:name=produce-service,type=produce-service:produce-error-rate" : "0.0"
} (com.linkedin.kmf.services.KafkaMetricsReporterService)
Signed-off-by: Andrew Choi <[email protected]>
@andrewchoi5 andrewchoi5 mentioned this pull request Jun 2, 2020
Merged
@andrewchoi5 andrewchoi5 merged commit 10045c5 into master Jun 2, 2020
@andrewchoi5 andrewchoi5 deleted the andchoi_ServiceFactory branch June 2, 2020 06:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@indrajeetkm indrajeetkm indrajeetkm approved these changes

@smccauliff smccauliff Awaiting requested review from smccauliff

@hzxa21 hzxa21 Awaiting requested review from hzxa21

Assignees

@andrewchoi5 andrewchoi5

Labels
New Feature New Feature Request
Projects
None yet
Milestone
New Feature & Improvements
Development

Successfully merging this pull request may close these issues.
2 participants
@andrewchoi5 @indrajeetkm