Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

create_perf_json: Migrate to pathlib.Path for file operations #122

Merged

Conversation

edwarddavidbaker
Copy link
Contributor

Previously event files were hosted at https://download.01.org/perfmon and fetching them with urllib was necessary. After migrating to GitHub we no longer need to fetch files using urllib.

  • Script arguments
    • Removed base path argument. At this point in time, create_perf_json.py only operates on files in this repository.
    • Add Path type to argument parser.
  • Migrate to Path
    • Use pathlib.Path to construct file and folder paths.
    • Updated type annotations with Path.
    • Replaced urlopen() with open().
    • Removed unused imports os and urllib.
    • Switched a few try: blocks to if <path>.is_file().
  • CI
    • Execute metric tests.

Previously event files were hosted at https://download.01.org/perfmon
and fetching them with urllib was necessary. After migrating to GitHub
we no longer need to fetch event files using urllib.

 - Removed base path argument. At this point in time, create_perf_json
   only operates on the files in this repository.
 - Add Path type to argument parser.
 - Use pathlib.Path to construct file and folder paths.
 - Updated type annotations with Path.
 - Replaced urlopen() with open().
 - Removed unused imports os and urllib.
Execute metric unittests as part of the GitHub action.
@edwarddavidbaker
Copy link
Contributor Author

Thanks Ian. Adding a few notes comparing the console and output directory contents. Output is the same. Console output has slight variations due to paths instead of file://.

# Generate output from main branch.
ed@ebaker-MOBL3:~/perfmon/public/perfmon/scripts$ git checkout main
ed@ebaker-MOBL3:~/perfmon/public/perfmon/scripts$ mkdir main
ed@ebaker-MOBL3:~/perfmon/public/perfmon/scripts$ python create_perf_json.py -vvv --outdir ./main/perf > ./main/output.txt

# Generate output from pull request.
ed@ebaker-MOBL3:~/perfmon/public/perfmon/scripts$ git checkout create_perf_json_pathlib 
ed@ebaker-MOBL3:~/perfmon/public/perfmon/scripts$ mkdir pr
ed@ebaker-MOBL3:~/perfmon/public/perfmon/scripts$ python create_perf_json.py -vvv --outdir ./pr/perf > ./pr/output.txt

Output content is the same.

ed@ebaker-MOBL3:~/perfmon/public/perfmon/scripts$ diff -rup main/perf/ pr/perf/
ed@ebaker-MOBL3:~/perfmon/public/perfmon/scripts$

Verbose logging is slightly different, but essentially the same.

ed@ebaker-MOBL3:~/perfmon/public/perfmon/scripts$ diff -rup main/output.txt pr/output.txt
<snip>
 Parsed models:
 ADL / alderlake
 	models=['GenuineIntel-6-97', 'GenuineIntel-6-9A', 'GenuineIntel-6-B7', 'GenuineIntel-6-BA', 'GenuineIntel-6-BF']
 	files:
-		atom = file:///home/ed/Documents/perfmon/public/perfmon/scripts/../ADL/events/alderlake_gracemont_core.json
-		core = file:///home/ed/Documents/perfmon/public/perfmon/scripts/../ADL/events/alderlake_goldencove_core.json
-		uncore = file:///home/ed/Documents/perfmon/public/perfmon/scripts/../ADL/events/alderlake_uncore.json
-		uncore experimental = file:///home/ed/Documents/perfmon/public/perfmon/scripts/../ADL/events/alderlake_uncore_experimental.json
-		tma metrics = file:///home/ed/Documents/perfmon/public/perfmon/scripts/../TMA_Metrics-full.csv
-		e-core tma metrics = file:///home/ed/Documents/perfmon/public/perfmon/scripts/../E-core_TMA_Metrics.csvADLN / alderlaken
+		atom = /home/ed/Documents/perfmon/public/perfmon/ADL/events/alderlake_gracemont_core.json
+		core = /home/ed/Documents/perfmon/public/perfmon/ADL/events/alderlake_goldencove_core.json
+		uncore = /home/ed/Documents/perfmon/public/perfmon/ADL/events/alderlake_uncore.json
+		uncore experimental = /home/ed/Documents/perfmon/public/perfmon/ADL/events/alderlake_uncore_experimental.json
+		tma metrics = /home/ed/Documents/perfmon/public/perfmon/TMA_Metrics-full.csv
+		e-core tma metrics = /home/ed/Documents/perfmon/public/perfmon/E-core_TMA_Metrics.csvADLN / alderlaken
 	models=['GenuineIntel-6-BE']
 	files:
<snip>
@@ -13508,8 +13508,8 @@ Updated tma_port_4 from
 "UOPS_DISPATCHED_PORT.PORT_4 / tma_info_core_core_clks"
 to
 "tma_store_op_utilization"
-Creating event json for CLX in ./main/perf/cascadelakex
-Generating core events from file:///home/ed/Documents/perfmon/public/perfmon/scripts/../CLX/events/cascadelakex_core.json
+Creating event json for CLX in /home/ed/Documents/perfmon/public/perfmon/scripts/pr/perf/cascadelakex
+Generating core events from /home/ed/Documents/perfmon/public/perfmon/CLX/events/cascadelakex_core.json
 Read perfmon event:
 event_name: INST_RETIRED.ANY, sample_after_value: 2000003, umask: 0x1, brief_description: Instructions retired from execution., public_description: Counts the number of instructions retired from execution. For instructions that consist of multiple micro-ops, Counts the retirement of the last micro-op of the instruction. Counting continues during hardware interrupts, traps, and inside interrupt handlers. Notes: INST_RETIRED.ANY is counted by a designated fixed counter, leaving the four (eight when Hyperthreading is disabled) programmable counters available for other events. INST_RETIRED.ANY_P is counted by a programmable counter and it is an architectural performance event. Counting: Faulting executions of GETSEC/VM entry/VM Exit/MWait will not count as retired instructions., topic: Pipeline
 Read perfmon event:
<snip>

@edwarddavidbaker edwarddavidbaker merged commit 5e5b20f into intel:main Dec 1, 2023
1 check passed
@edwarddavidbaker edwarddavidbaker deleted the create_perf_json_pathlib branch December 1, 2023 16:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants