Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify profile stack trace representation #615

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Simplify profile stack trace representation #615

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 18 additions & 15 deletions opentelemetry/proto/profiles/v1development/profiles.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -163,24 +163,24 @@ message ScopeProfiles {
// - On-disk, the serialized proto must be gzip-compressed.
//
// - The profile is represented as a set of samples, where each sample
// references a sequence of locations, and where each location belongs
// references a stack trace which is a list of locations, each belonging
// to a mapping.
// - There is a N->1 relationship from sample.location_id entries to
// locations. For every sample.location_id entry there must be a
// - There is a N->1 relationship from Stack.location_indices entries to
// locations. For every Stack.location_indices entry there must be a
// unique Location with that index.
// - There is an optional N->1 relationship from locations to
// mappings. For every nonzero Location.mapping_id there must be a
// unique Mapping with that index.

// Represents a complete profile, including sample types, samples,
// mappings to binaries, locations, functions, string table, and additional metadata.
// It modifies and annotates pprof Profile with OpenTelemetry specific fields.
// Represents a complete profile, including sample types, samples, mappings to
// binaries, stacks, locations, functions, string table, and additional
// metadata. It modifies and annotates pprof Profile with OpenTelemetry
// specific fields.
//
// Note that whilst fields in this message retain the name and field id from pprof in most cases
// for ease of understanding data migration, it is not intended that pprof:Profile and
// OpenTelemetry:Profile encoding be wire compatible.
message Profile {

// A description of the samples associated with each Sample.value.
// For a cpu profile this might be:
// [["cpu","nanoseconds"]] or [["wall","seconds"]] or [["syscall","count"]]
Expand All @@ -197,10 +197,10 @@ message Profile {
// If multiple binaries contribute to the Profile and no main
// binary can be identified, mapping[0] has no special meaning.
repeated Mapping mapping_table = 3;
// Locations referenced by samples via location_indices.
// Locations referenced by stacks via location_indices.
repeated Location location_table = 4;
// Array of locations referenced by samples.
repeated int32 location_indices = 5;
// Stacks referenced by samples via stack_index.
repeated Stack stack_table = 5;
// Functions referenced by locations.
repeated Function function_table = 6;
// Lookup table for attributes.
Expand Down Expand Up @@ -374,11 +374,8 @@ message ValueType {
// augmented with auxiliary information like the thread-id, some
// indicator of a higher level request being handled etc.
message Sample {
// locations_start_index along with locations_length refers to to a slice of locations in Profile.location_indices.
int32 locations_start_index = 1;
// locations_length along with locations_start_index refers to a slice of locations in Profile.location_indices.
// Supersedes location_index.
int32 locations_length = 2;
// Reference to stack in Profile.stack_table.
int32 stack_index = 1;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice to have an optional int64 instruction_ip property here that would take the common stack at stack_index, but also add the instruction_ip to it. This would allow for common on-CPU callstacks to be respresented cheaply that are in the same function but hit different parts of it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't be just int64 since that won't play well with symbolization and inlines, would have to be a location ID.

Also, this optimization would only be most useful to profile types like CPU sampling where the leaf frame is indeed very random. For profiles like heap profiling it would not be as valuable since there even the leaf is a callsite, not a sampled IP.

And if we are looking at such "encode the tree more efficiently" optimizations then we should probably look at using the actual tree encoding as discussed a bit here.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, it's only useful for on-CPU sampling with skids, etc. So I don't think it should be required, but definitely useful when needed as an option.

// The type and unit of each value is defined by the corresponding
// entry in Profile.sample_type. All samples must have the same
// number of values, the same as the length of Profile.sample_type.
Expand All @@ -397,6 +394,12 @@ message Sample {
repeated uint64 timestamps_unix_nano = 6;
}

// A Stack represents a stack trace as a list of locations. The first location
// is the leaf frame.
message Stack {
repeated int32 location_indices = 1;
}

// Describes the mapping of a binary in memory, including its address range,
// file offset, and metadata like build ID
message Mapping {
Expand Down
Loading