Skip to content

Commit

Permalink
pw_bloat: Simple pw bloat CLI command
Browse files Browse the repository at this point in the history
This adds a command to the Pigweed CLI which can be used to run size
reports on binaries without having to go through the GN build.

The command initially supports single binary size reports on ELF files
which are linked using pw_bloat memory region symbols. Additional data
sources can be specified to be displayed hierarchically under the root
memoryregions.

Example usage (no child data sources):

$ pw bloat out/docs/obj/pw_result/size_report/bin/ladder_and_then.elf

 ▒█████▄   █▓  ▄███▒  ▒█    ▒█ ░▓████▒ ░▓████▒ ▒▓████▄
  ▒█░  █░ ░█▒ ██▒ ▀█▒ ▒█░ █ ▒█  ▒█   ▀  ▒█   ▀  ▒█  ▀█▌
  ▒█▄▄▄█░ ░█▒ █▓░ ▄▄░ ▒█░ █ ▒█  ▒███    ▒███    ░█   █▌
  ▒█▀     ░█░ ▓█   █▓ ░█░ █ ▒█  ▒█   ▄  ▒█   ▄  ░█  ▄█▌
  ▒█      ░█░ ░▓███▀   ▒█▓▀▓█░ ░▓████▒ ░▓████▒ ▒▓████▀

+----------------------+---------+
|     memoryregions    |  sizes  |
+======================+=========+
|FLASH                 |1,048,064|
|RAM                   |  196,608|
|VECTOR_TABLE          |      512|
+======================+=========+
|Total                 |1,245,184|
+----------------------+---------+

Change-Id: Icc34a085cc62ce3fcf0697f04aaed50c6d559024
Requires: pigweed-internal:32580
Reviewed-on: https://pigweed-review.googlesource.com/c/pigweed/pigweed/+/112314
Reviewed-by: Armando Montanez <[email protected]>
Reviewed-by: Ewout van Bekkum <[email protected]>
Commit-Queue: Alexei Frolov <[email protected]>
  • Loading branch information
frolv authored and CQ Bot Account committed Oct 21, 2022
1 parent d881950 commit e31b93c
Show file tree
Hide file tree
Showing 8 changed files with 149 additions and 27 deletions.
43 changes: 41 additions & 2 deletions pw_bloat/docs.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,48 @@ Bloat report cards allow tracking the memory usage of a system over time as code
changes are made and provide a breakdown of which parts of the code have the
largest size impact.

``pw bloat`` CLI command
========================
``pw_bloat`` includes a plugin for the Pigweed command line capable of running
size reports on ELF binaries.

.. note::

The bloat CLI plugin is still experimental and only supports a small subset
of ``pw_bloat``'s capabilities. Notably, it currently only runs on binaries
which define memory region symbols; refer to the
:ref:`memoryregions documentation <module-pw_bloat-memoryregions>`
for details.

Basic usage
^^^^^^^^^^^

**Running a size report on a single executable**

.. code-block:: sh
$ pw bloat out/docs/obj/pw_result/size_report/bin/ladder_and_then.elf
▒█████▄ █▓ ▄███▒ ▒█ ▒█ ░▓████▒ ░▓████▒ ▒▓████▄
▒█░ █░ ░█▒ ██▒ ▀█▒ ▒█░ █ ▒█ ▒█ ▀ ▒█ ▀ ▒█ ▀█▌
▒█▄▄▄█░ ░█▒ █▓░ ▄▄░ ▒█░ █ ▒█ ▒███ ▒███ ░█ █▌
▒█▀ ░█░ ▓█ █▓ ░█░ █ ▒█ ▒█ ▄ ▒█ ▄ ░█ ▄█▌
▒█ ░█░ ░▓███▀ ▒█▓▀▓█░ ░▓████▒ ░▓████▒ ▒▓████▀
+----------------------+---------+
| memoryregions | sizes |
+======================+=========+
|FLASH |1,048,064|
|RAM | 196,608|
|VECTOR_TABLE | 512|
+======================+=========+
|Total |1,245,184|
+----------------------+---------+
.. _bloat-howto:

Defining size reports
=====================
Defining size reports in GN
===========================

Diff Size Reports
^^^^^^^^^^^^^^^^^
Expand Down Expand Up @@ -364,6 +402,7 @@ Note that linker scripts are not natively supported by GN and can't be provided
through ``deps``, the ``bloat_macros.ld`` must be passed in the ``includes``
list.

.. _module-pw_bloat-memoryregions:

``memoryregions`` data source
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Expand Down
1 change: 1 addition & 0 deletions pw_bloat/py/BUILD.gn
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ pw_python_package("py") {
]
sources = [
"pw_bloat/__init__.py",
"pw_bloat/__main__.py",
"pw_bloat/bloat.py",
"pw_bloat/bloaty_config.py",
"pw_bloat/label.py",
Expand Down
4 changes: 2 additions & 2 deletions pw_bloat/py/label_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
import os
import logging
import sys
from pw_bloat.label import from_bloaty_tsv, DataSourceMap, Label
from pw_bloat.label import DataSourceMap, Label

LIST_LABELS = [
Label(name='main()', size=30, parents=tuple(['FLASH', '.code'])),
Expand All @@ -35,7 +35,7 @@ def get_test_map():
pw_root = os.environ.get("PW ROOT")
filename = f"{pw_root}/pigweed/pw_bloat/test_label.csv"
with open(filename, 'r') as csvfile:
ds_map = from_bloaty_tsv(csvfile)
ds_map = DataSourceMap.from_bloaty_tsv(csvfile)
capacity_patterns = [("^__TEXT$", 459), ("^_", 920834)]
for cap_pattern, cap_size in capacity_patterns:
ds_map.add_capacity(cap_pattern, cap_size)
Expand Down
78 changes: 78 additions & 0 deletions pw_bloat/py/pw_bloat/__main__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
# Copyright 2022 The Pigweed Authors
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may not
# use this file except in compliance with the License. You may obtain a copy of
# the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations under
# the License.
"""Size reporting utilities."""

import argparse
import logging
from pathlib import Path
import sys

from pw_bloat import bloat
from pw_bloat.label import DataSourceMap
from pw_bloat.label_output import BloatTableOutput
import pw_cli.log

_LOG = logging.getLogger(__name__)


def _parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(description=__doc__)

parser.add_argument('binary',
help='Path to the ELF file to analyze',
type=Path)
parser.add_argument(
'-d',
'--data-sources',
help='Comma-separated list of additional Bloaty data sources to report',
type=lambda s: s.split(','),
default=())
parser.add_argument('-v',
'--verbose',
help=('Print all log messages '
'(only errors are printed by default)'),
action='store_true')

return parser.parse_args()


def main() -> int:
"""Run binary size reports."""

args = _parse_args()

if not args.verbose:
pw_cli.log.set_all_loggers_minimum_level(logging.ERROR)

try:
bloaty_tsv = bloat.memory_regions_size_report(
args.binary,
additional_data_sources=args.data_sources,
extra_args=('--tsv', ))
except bloat.NoMemoryRegions:
_LOG.error('Executable %s does not define any bloat memory regions',
args.binary)
_LOG.error(
'Refer to https://pigweed.dev/pw_bloat/#memoryregions-data-source')
_LOG.error('for information on how to configure them.')
return 1

data_source_map = DataSourceMap.from_bloaty_tsv(bloaty_tsv)

print(BloatTableOutput(data_source_map).create_table())
return 0


if __name__ == '__main__':
sys.exit(main())
18 changes: 10 additions & 8 deletions pw_bloat/py/pw_bloat/bloat.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
import pw_cli.log

from pw_bloat.bloaty_config import generate_bloaty_config
from pw_bloat.label import from_bloaty_tsv
from pw_bloat.label import DataSourceMap
from pw_bloat.label_output import (BloatTableOutput, LineCharset, RstOutput,
AsciiCharset)

Expand Down Expand Up @@ -106,7 +106,7 @@ def memory_regions_size_report(
elf: Path,
additional_data_sources: Iterable[str] = (),
extra_args: Iterable[str] = (),
) -> str:
) -> Iterable[str]:
"""Runs a size report on an ELF file using pw_bloat memory region symbols.
Arguments:
Expand Down Expand Up @@ -137,7 +137,7 @@ def memory_regions_size_report(
bloaty_config.name,
data_sources=('memoryregions', *additional_data_sources),
extra_args=extra_args,
).decode('utf-8')
).decode('utf-8').splitlines()


def write_file(filename: str, contents: str, out_dir_file: str) -> None:
Expand All @@ -162,11 +162,12 @@ def single_target_output(target: str, bloaty_config: str, target_out_file: str,
return 1

single_tsv = single_output.decode().splitlines()
single_report = BloatTableOutput(from_bloaty_tsv(single_tsv),
single_report = BloatTableOutput(DataSourceMap.from_bloaty_tsv(single_tsv),
MAX_COL_WIDTH, LineCharset)

rst_single_report = BloatTableOutput(from_bloaty_tsv(single_tsv),
MAX_COL_WIDTH, AsciiCharset, True)
rst_single_report = BloatTableOutput(
DataSourceMap.from_bloaty_tsv(single_tsv), MAX_COL_WIDTH, AsciiCharset,
True)

single_report_table = single_report.create_table()

Expand Down Expand Up @@ -242,8 +243,9 @@ def main() -> int:
if not single_output_target or not single_output_base:
continue

base_dsm = from_bloaty_tsv(single_output_base.decode().splitlines())
target_dsm = from_bloaty_tsv(
base_dsm = DataSourceMap.from_bloaty_tsv(
single_output_base.decode().splitlines())
target_dsm = DataSourceMap.from_bloaty_tsv(
single_output_target.decode().splitlines())
diff_dsm = target_dsm.diff(base_dsm)

Expand Down
24 changes: 12 additions & 12 deletions pw_bloat/py/pw_bloat/label.py
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,18 @@ class DataSourceMap:
"""
_BASE_TOTAL_LABEL = 'total'

@classmethod
def from_bloaty_tsv(cls, raw_tsv: Iterable[str]) -> 'DataSourceMap':
"""Read in Bloaty TSV output and store in DataSourceMap."""
reader = csv.reader(raw_tsv, delimiter='\t')
top_row = next(reader)
vmsize_index = top_row.index('vmsize')
ds_map_tsv = cls(top_row[:vmsize_index])
for row in reader:
ds_map_tsv.insert_label_hierachy(row[:vmsize_index],
int(row[vmsize_index]))
return ds_map_tsv

def __init__(self, data_sources_names: Iterable[str]):
self._data_sources = list(
_DataSource(name) for name in ['base', *data_sources_names])
Expand Down Expand Up @@ -246,15 +258,3 @@ def has_diff_sublabels(self, top_ds_label: str) -> bool:
== top_ds_label):
return True
return False


def from_bloaty_tsv(raw_tsv: Iterable[str]) -> DataSourceMap:
"""Read in Bloaty TSV output and store in DataSourceMap."""
reader = csv.reader(raw_tsv, delimiter='\t')
top_row = next(reader)
vmsize_index = top_row.index('vmsize')
ds_map_tsv = DataSourceMap(top_row[:vmsize_index])
for row in reader:
ds_map_tsv.insert_label_hierachy(row[:vmsize_index],
int(row[vmsize_index]))
return ds_map_tsv
3 changes: 2 additions & 1 deletion pw_bloat/py/pw_bloat/label_output.py
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ class BloatTableOutput:
"""ASCII Table generator from DataSourceMap."""

_RST_PADDING_WIDTH = 6
_DEFAULT_MAX_WIDTH = 80

class _LabelContent(NamedTuple):
name: str
Expand All @@ -85,7 +86,7 @@ class _LabelContent(NamedTuple):

def __init__(self,
ds_map: Union[DiffDataSourceMap, DataSourceMap],
col_max_width: int,
col_max_width: int = _DEFAULT_MAX_WIDTH,
charset: Union[Type[AsciiCharset],
Type[LineCharset]] = AsciiCharset,
rst_output: bool = False,
Expand Down
5 changes: 3 additions & 2 deletions pw_cli/py/pw_cli/pw_command_plugins.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,14 +29,15 @@ def _register_builtin_plugins(registry: plugins.Registry) -> None:
"""Registers the commands that are included with pw by default."""

# Register these by name to avoid circular dependencies.
registry.register_by_name('bloat', 'pw_bloat.__main__', 'main')
registry.register_by_name('doctor', 'pw_doctor.doctor', 'main')
registry.register_by_name('python-packages',
'pw_env_setup.python_packages', 'main')
registry.register_by_name('format', 'pw_presubmit.format_code', 'main')
registry.register_by_name('keep-sorted', 'pw_presubmit.keep_sorted',
'main')
registry.register_by_name('logdemo', 'pw_cli.log', 'main')
registry.register_by_name('module', 'pw_module.__main__', 'main')
registry.register_by_name('python-packages',
'pw_env_setup.python_packages', 'main')
registry.register_by_name('test', 'pw_unit_test.test_runner', 'main')
registry.register_by_name('watch', 'pw_watch.watch', 'main')

Expand Down

0 comments on commit e31b93c

Please sign in to comment.