Add memory usage to worker status in rake evm:status and status_full #15375

jrafanie · 2017-06-14T20:48:34Z

I was using this locally for measuring worker sizes and thought it might be helpful to others.

04:43:06 ~/Code/manageiq (add_memory_information_to_status) (2.4.1) + bin/rake evm:status
** Using session_store: ActionDispatch::Session::MemCacheStore
Checking EVM status...
 Zone    | Server | Status  |            ID |   PID |  SPID | URL                     | Started On           | Last Heartbeat       | Master? | Active Roles
---------+--------+---------+---------------+-------+-------+-------------------------+----------------------+----------------------+---------+-----------------------------------------------------------------------------------------------------------------------------------------------------
 default | EVM    | started | 1000000000001 | 89807 | 89834 | druby://127.0.0.1:60899 | 2017-06-15T20:15:19Z | 2017-06-15T20:17:31Z | true    | automate:database_operations:database_owner:ems_inventory:ems_operations:event:reporting:scheduler:smartstate:user_interface:web_services:websocket

 Worker Type       | Status  |            ID |   PID | SPID  |     Server id | Queue Name / URL    | Started On           | Last Heartbeat       | MB Usage
-------------------+---------+---------------+-------+-------+---------------+---------------------+----------------------+----------------------+----------
 MiqGenericWorker  | started | 1000000000339 | 89838 | 89847 | 1000000000001 | generic             | 2017-06-15T20:15:19Z | 2017-06-15T20:17:46Z |
 MiqScheduleWorker | started | 1000000000340 | 89841 | 89848 | 1000000000001 |                     | 2017-06-15T20:15:19Z | 2017-06-15T20:17:37Z | 239 MB
 MiqUiWorker       | started | 1000000000341 | 89844 |       | 1000000000001 | http://0.0.0.0:3000 | 2017-06-15T20:15:20Z | 2017-06-15T20:17:44Z | 236 MB

Note, any worker with a nil value will show up as an empty string.

Note, this also shows up when you call evm:status_full to see multiple servers and the workers on those servers.

NickLaMuro

Pedantic suggestion only, and not really needed. You probably should just 🐑 🇮🇹 before I get any other bad ideas...

NickLaMuro · 2017-06-14T20:54:11Z

lib/tasks/evm_application.rb

@@ -102,11 +102,12 @@ def self.output_workers_status(servers)
           w.miq_server_id,
           w.queue_name || w.uri,
           w.started_on && w.started_on.iso8601,
-           w.last_heartbeat && w.last_heartbeat.iso8601]
+           w.last_heartbeat && w.last_heartbeat.iso8601,
+           (w.proportional_set_size || w.memory_usage || 0) / 1.megabyte]


I know I had this really gross ternary operator for you in a PM as an alternative:

(mem = (w.proportional_set_size || w.memory_usage)).nil? ? 0 : mem / 1.megabyte

While what you have effectively does the exact same thing without requiring some TUMS™ afterwords, the advantage of the ternary is instead of returning a zero, we could return a "N/A", which probably is more accurate. But this is being a bit pedantic.

yeah, shrug. Is it pedantic? I thought the same thing but couldn't convince myself to make it more complicated.

Is it pedantic?

Well, I said it was, so it is!

But really, my rational is that seeing a 0 Mb process in the status might be confusing to a user, and might give the impression of "the worker is just super efficient memory!" to someone who doesn't know what is going on under the hood. But again, I do think I am being pedantic, and problem more (questionably correct) info is better than none.

Yeah, I agree with you. Maybe blank would be better than N/A. Let me look at that and clean it up.

jrafanie · 2017-06-15T20:22:56Z

Ok, updated as per @NickLaMuro suggestion of not showing 0 and @Fryguy's suggestion of using the number helper.

Fryguy · 2017-06-16T15:01:44Z

lib/tasks/evm_application.rb

@@ -102,11 +102,12 @@ def self.output_workers_status(servers)
           w.miq_server_id,
           w.queue_name || w.uri,
           w.started_on && w.started_on.iso8601,
-           w.last_heartbeat && w.last_heartbeat.iso8601]
+           w.last_heartbeat && w.last_heartbeat.iso8601,
+           ActionView::Base.new.number_to_human_size(w.proportional_set_size || w.memory_usage) || ""]


Prefer .to_s(:human_size) over calling out to ActionView directly. http://edgeguides.rubyonrails.org/active_support_core_extensions.html#formatting

memory_size = w.proportional_set_size || w.memory_usage memory_size ? memory_size.to_s(:human_size) : ""

@Fryguy Not sure that will work with an empty string:

> "".to_s(:to_human) ArgumentError: wrong number of arguments (given 1, expected 0) from (irb):3:in `to_s' from (irb):3 from /Users/nicklamuro/.gem/ruby/2.3.3/gems/railties-5.0.3/lib/rails/commands/console.rb:65:in `start' from /Users/nicklamuro/.gem/ruby/2.3.3/gems/railties-5.0.3/lib/rails/commands/console_helper.rb:9:in `start' from /Users/nicklamuro/.gem/ruby/2.3.3/gems/railties-5.0.3/lib/rails/commands/commands_tasks.rb:78:in `console' from /Users/nicklamuro/.gem/ruby/2.3.3/gems/railties-5.0.3/lib/rails/commands/commands_tasks.rb:49:in `run_command!' from /Users/nicklamuro/.gem/ruby/2.3.3/gems/railties-5.0.3/lib/rails/commands.rb:18:in `<top (required)>' from bin/rails:4:in `require' from bin/rails:4:in `<main>'

But yes, some form of that would look a bit nicer.

yeah, I edited my response probably as you were typing yours.

@Fryguy Ah, so we are back to the really gross ternary I suggested here: #15375 (comment)

I don't even care what color the shed is 🚲 🏠

jrafanie · 2017-06-16T18:01:27Z

lib/tasks/evm_application.rb

@@ -102,11 +102,12 @@ def self.output_workers_status(servers)
           w.miq_server_id,
           w.queue_name || w.uri,
           w.started_on && w.started_on.iso8601,
-           w.last_heartbeat && w.last_heartbeat.iso8601]
+           w.last_heartbeat && w.last_heartbeat.iso8601,
+           (mem = (w.proportional_set_size || w.memory_usage)).nil? ? "" : mem / 1.megabyte]


I feel like I've seen this before...

(☞ﾟヮﾟ)☞

I would use (mem = (w.proportional_set_size || w.memory_usage)).nil? ? "" : mem.to_s(:human_size)], but that's just splitting hairs

LGTM 👍

jrafanie · 2017-06-16T18:01:52Z

Ready to go @NickLaMuro @Fryguy

miq-bot · 2017-06-16T18:02:39Z

Checked commit jrafanie@5be3b45 with ruby 2.2.6, rubocop 0.47.1, and haml-lint 0.20.0
1 file checked, 0 offenses detected
Everything looks fine. 🏆

NickLaMuro · 2017-06-16T18:24:45Z

I approved already... but since you added my "lovely ternary", that gets you 👍 👍 way up!

jrafanie added core/workers enhancement performance labels Jun 14, 2017

jrafanie requested a review from NickLaMuro June 14, 2017 20:50

NickLaMuro reviewed Jun 14, 2017

View reviewed changes

chessbyte assigned gtanzillo Jun 15, 2017

jrafanie force-pushed the add_memory_information_to_evm_status branch 2 times, most recently from 98f41f3 to e5f2f50 Compare June 15, 2017 20:21

NickLaMuro approved these changes Jun 15, 2017

View reviewed changes

Fryguy requested changes Jun 16, 2017

View reviewed changes

Add memory usage to worker status

5be3b45

jrafanie force-pushed the add_memory_information_to_evm_status branch from e5f2f50 to 5be3b45 Compare June 16, 2017 17:59

jrafanie commented Jun 16, 2017

View reviewed changes

Fryguy approved these changes Jun 16, 2017

View reviewed changes

gtanzillo added this to the Sprint 63 Ending Jun 19, 2017 milestone Jun 16, 2017

gtanzillo merged commit e8e6c8d into ManageIQ:master Jun 16, 2017

jrafanie deleted the add_memory_information_to_evm_status branch June 16, 2017 18:46

NickLaMuro mentioned this pull request Jun 16, 2017

[POC][WIP] Slim evm rake tasks (alternative to #14916) #15342

Closed

jrafanie mentioned this pull request Jun 27, 2017

Add server MB usage to rake evm:status and status_full. #15457

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add memory usage to worker status in rake evm:status and status_full #15375

Add memory usage to worker status in rake evm:status and status_full #15375

jrafanie commented Jun 14, 2017 •

edited

Loading

NickLaMuro left a comment

NickLaMuro Jun 14, 2017

jrafanie Jun 14, 2017

NickLaMuro Jun 14, 2017 •

edited

Loading

jrafanie Jun 15, 2017

jrafanie commented Jun 15, 2017 •

edited

Loading

Fryguy Jun 16, 2017 •

edited

Loading

NickLaMuro Jun 16, 2017

NickLaMuro Jun 16, 2017 •

edited

Loading

Fryguy Jun 16, 2017

NickLaMuro Jun 16, 2017 •

edited

Loading

jrafanie Jun 16, 2017

jrafanie Jun 16, 2017

NickLaMuro Jun 16, 2017

Fryguy Jun 16, 2017

jrafanie commented Jun 16, 2017

miq-bot commented Jun 16, 2017

NickLaMuro commented Jun 16, 2017

Add memory usage to worker status in rake evm:status and status_full #15375

Add memory usage to worker status in rake evm:status and status_full #15375

Conversation

jrafanie commented Jun 14, 2017 • edited Loading

NickLaMuro left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NickLaMuro Jun 14, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jrafanie commented Jun 15, 2017 • edited Loading

Fryguy Jun 16, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NickLaMuro Jun 16, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NickLaMuro Jun 16, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jrafanie commented Jun 16, 2017

miq-bot commented Jun 16, 2017

NickLaMuro commented Jun 16, 2017

jrafanie commented Jun 14, 2017 •

edited

Loading

NickLaMuro Jun 14, 2017 •

edited

Loading

jrafanie commented Jun 15, 2017 •

edited

Loading

Fryguy Jun 16, 2017 •

edited

Loading

NickLaMuro Jun 16, 2017 •

edited

Loading

NickLaMuro Jun 16, 2017 •

edited

Loading