Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improving multiline context snippet #2288

Merged
merged 6 commits into from
Feb 20, 2021
Merged

Improving multiline context snippet #2288

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
b04dfc1
Improving multiline context snippet
eddynaka Feb 17, 2021
d3886be
changing ConstructMultilineContextSnippet logic
eddynaka Feb 18, 2021
fef1246
updating to charlength property
eddynaka Feb 18, 2021
58cf640
Adding check to verify offset
eddynaka Feb 18, 2021
6f9318c
Reviewing logic to prevent issues
eddynaka Feb 19, 2021
e493b7c
Updating tests.
eddynaka Feb 19, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 30 additions & 3 deletions src/Sarif/FileRegionsCache.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -156,15 +156,42 @@ internal Region ConstructMultilineContextSnippet(Region inputRegion, Uri uri)
return null;
}

const int maxSnippetLength = 512;
const int reasonableSnippetLength = 256;
const int smallSnippetLength = 128;

// If this is big enough, we don't need to create a new snippet.
if (inputRegion.CharLength >= reasonableSnippetLength)
Copy link
Member

@michaelcfanning michaelcfanning Feb 18, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

inputRegion.CharLength [](start = 16, length = 22)

Can we please put a check in place that ensures the region has populated char length and offset properly? Go check the spec on this type, there's a sentinel value/property default that you can use to verify.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, so, we will just check this value if charOffset != -1.

In reply to: 578571743 [](ancestors = 578571743)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait, this isn't right, is it? If CharOffset == -1, this method can't function, can it?

In reply to: 578583524 [](ancestors = 578583524,578571743)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if == -1, then, ur region is using startline / startcolumn, with that, the populateTextRegion handle both cases.

In reply to: 578586111 [](ancestors = 578586111,578583524,578571743)

{
return null;
}
Copy link
Member

@michaelcfanning michaelcfanning Feb 18, 2021 •
edited by eddynaka
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't we actually be checking the region size, not the snippet size? basically, if a region exceeds a certain size, no context region is required. so for this case, the region should have the snippet. your code will permit the following case:

region is 10k in size but has no snippet
we then create a context region + snippet which is smaller than the region! this breaks the SARIF spec. :) #Resolved

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed to the charLength.

In reply to: 578048420 [](ancestors = 578048420)


int maxLineNumber = newLineIndex.MaximumLineNumber;

// Currently, we just grab a single line before and after the region start
// and end lines, respectively. In the future, we could make this configurable.
var region = new Region()
var region = new Region
{
StartLine = inputRegion.StartLine == 1 ? 1 : inputRegion.StartLine - 1,
EndLine = inputRegion.EndLine == maxLineNumber ? maxLineNumber : inputRegion.EndLine + 1
};

Region multilineContextSnippet = this.PopulateTextRegionProperties(region, uri, populateSnippet: true);

// If the new snippet is big enough, we will use. Otherwise, we will
// generate a new snippet
if (multilineContextSnippet.CharLength <= maxSnippetLength)
{
return multilineContextSnippet;
}

// We are going to grab 128 characters from left and from right if possible.
region.CharOffset = inputRegion.CharOffset < smallSnippetLength
? 0
: inputRegion.CharOffset - smallSnippetLength;

region.CharLength = inputRegion.CharLength + inputRegion.CharOffset + smallSnippetLength < newLineIndex.Text.Length
? inputRegion.CharLength + inputRegion.CharOffset + smallSnippetLength
: newLineIndex.Text.Length - region.CharOffset;

return this.PopulateTextRegionProperties(region, uri, populateSnippet: true);
Copy link
Member

@michaelcfanning michaelcfanning Feb 18, 2021 •
edited by eddynaka
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

return [](start = 12, length = 6)

the end of this method could be a good place to introduce a debug assert that verifies we have limited the size of the region that we return. #Resolved

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the debug.Assert to verify if the new length >= the original

In reply to: 578572270 [](ancestors = 578572270)

}

Expand Down
33 changes: 33 additions & 0 deletions src/Test.UnitTests.Sarif/FileRegionsCacheTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -583,5 +583,38 @@ public void FileRegionsCache_PopulatesNullRegion()
region = fileRegionsCache.PopulateTextRegionProperties(inputRegion: null, uri: uri, populateSnippet: true);
region.Should().BeNull();
}

[Fact]
public void FileRegionsCache_PopulatesWithOneLineText()
{
const int maxLength = 128;
Uri uri = new Uri(@"c:\temp\myFile.cpp");
const string fileContent = @"internal Region ConstructMultilineContextSnippet(Region inputRegion, Uri uri){if (inputRegion == null || inputRegion.IsBinaryRegion){return null;}NewLineIndex newLineIndex = GetNewLineIndex(uri, fileText: null);if (newLineIndex == null){return null;}if (inputRegion.Snippet.Text.Length >= 512){return null;}int maxLineNumber = newLineIndex.MaximumLineNumber;var region = new Region{StartLine = inputRegion.StartLine == 1 ? 1 : inputRegion.StartLine - 1,EndLine = inputRegion.EndLine == maxLineNumber ? maxLineNumber : inputRegion.EndLine + 1};Region multilineContextSnippet = this.PopulateTextRegionProperties(region, uri, populateSnippet: true);if (multilineContextSnippet.Snippet.Text.Length <= 512){return multilineContextSnippet;} const int maxLength = 128;region.CharOffset = inputRegion.CharOffset < maxLength? 0: inputRegion.CharOffset - maxLength;region.CharLength = inputRegion.CharLength + inputRegion.CharOffset + maxLength < newLineIndex.Text.Length? inputRegion.CharLength + inputRegion.CharOffset + maxLength: newLineIndex.Text.Length - region.CharOffset;return this.PopulateTextRegionProperties(region, uri, populateSnippet: true);}";

var region = new Region
{
CharOffset = 114,
CharLength = 600,
};

var fileRegionsCache = new FileRegionsCache();
Region newRegion = fileRegionsCache.PopulateTextRegionProperties(region, uri, true, fileContent);
newRegion.Snippet.Text.Should().Be(fileContent.Substring(region.CharOffset, region.CharLength));

Region multilineRegion = fileRegionsCache.ConstructMultilineContextSnippet(newRegion, uri);

// This should be null, since the newRegion already is big enough
multilineRegion.Should().BeNull();

region.CharOffset = 125;
region.CharLength = 22;
newRegion = fileRegionsCache.PopulateTextRegionProperties(region, uri, true, fileContent);
newRegion.Snippet.Text.Should().Be(fileContent.Substring(region.CharOffset, region.CharLength));

multilineRegion = fileRegionsCache.ConstructMultilineContextSnippet(newRegion, uri);

// This should get from 0 -> (125(offset) + 128(maxlength) + 22(charlength))
multilineRegion.Snippet.Text.Should().Be(fileContent.Substring(0, region.CharOffset + region.CharLength + maxLength));
}
}
}