Texture-based paths for thick anti-aliased lines

[ocornut]

We have various changes coming in the low-level drawing pipelines (mostly implemented by Ben aka @ShironekoBen) and for some of them we'll be looking for feedback.

The first one is fairly simple and available in the features/tex_antialiased_lines branch:
https://github.com/ocornut/imgui/tree/features/tex_antialiased_lines

We made it that lines up to a certain thickness are relying on texture data instead of extra polygon.

Essentially:

• Lines that are 1.0f thick now use 50% less indices, 33% less vertices, CPU perfs should be similar.
• Lines thicker than >1.0f now use 66% less indices, 50% less vertices, in addition they are faster to render on CPU (there's too many factor involved but would say roughly 20-30% faster).
• Both paths use same code and indices/vertices count. This is useful as it means future DPI and style related changes can make it easier to scale thickness.

Added in style:
bool style.AntiAliasedLinesUseTex

Added in font atlas:
ImFontAtlas: ImFontAtlasFlags_NoAntiAliasedLines (disable the whole thing which takes about 64x64 pixels or pixel space, for very low memory footprint devices)

Restriction:

• This works if the renderer is using bilinear interpolation on the texture rather than point/nearest filtering. The majority of back-ends and renderers are already doing so, but some embedded/low-end GPU back-ends may not.

Feedback wanted:

• We'd like large or unusual users of lines primitives to be testing this branch if they can. While I think the branch is looking good in most tests we done, the exact pixels output is different and it might affect some. We're curious in particular if this is affecting people using non-integer thickness (e.g. 1.5) or doing unusual scaling.
• You may try to add a live toggle in your main loop, e.g. style.AntiAliasedLinesUseTex = !ImGui::GetIO().KeyShift as a way to easily compare for difference.

This is most probably breaking PR #2964 which we will rework accordingly.

This should merge in docking with 1 minor conflict which should be obvious to fix when merging (conflicting comments in nearby lines).

Taking the liberty to tag people who I suspect may be interested (writers of node editors or plot widgets):

@rokups @thedmd @inflex @wolfpld @epezent @r-lyeh @soulthreads @mkalte666 @Nelarius

Attaching snapshot of some @epezent earlier tests were they stated:

I made the comparison between imgui/master and features/tex_antialiased_lines. I couldn't detect any noticeable visual differences between them. I also toggled ImDrawListFlags_AntiAliasedLinesUseTexData on/off while zooming/panning, moving plot windows, etc. I never saw anything strange. As expected, there is an appreciable performance boost using the new texture anti-aliased method as well.
You can overlay the images below in Photoshop and see that there is a very tiny difference between the two in some areas, but not enough to concern me."

[epezent]

Here are my test results for ImPlot:

Computer

• Windows 10 64-bit
• i7 9700K
• Nvidia RTX 2070
• 32 GB Ram

Test Setup Description

• added ImPlot sources to imgui_test_app Project
• replaced ShowUI with ShowImPlotBenchmark below
• used IMGUI_APP_WIN32_DX11 backend
• enabled 32-bit indices in imconfig.h
• passed -nothrottle option
• ImPlotFlags_AntiAliased flag set (ImPlot will use DrawList.AddLine for each individual segment)

Test Code

float RandomRange(float min, float max) {
    float scale = rand() / (float)RAND_MAX;
    return min + scale * (max - min);
}

struct BenchmarkItem {
    BenchmarkItem() {
        float y = RandomRange(0, 1);
        Data = new ImVec2[1000];
        for (int i = 0; i < 1000; ++i) {
            Data[i].x = i * 0.001f;
            Data[i].y = y + RandomRange(-0.01f, 0.01f);
        }
        Col = ImVec4(RandomRange(0, 1), RandomRange(0, 1), RandomRange(0, 1), 1);
    }
    ~BenchmarkItem() { delete Data; }
    ImVec2* Data;
    ImVec4 Col;
};

static void ShowImPlotBenchmark() {
    ImGui::SetNextWindowPos(ImVec2(0, 0), ImGuiCond_Always);
    ImGui::SetNextWindowSize(ImVec2(1400, 850));
    ImGui::Begin("ImPlot Benchmark");
    static const int n_items = 100;
    srand(0);
    static BenchmarkItem items[n_items];
    ImGui::BulletText("%d lines with %d points each @ %.3f FPS.", n_items, 1000, ImGui::GetIO().Framerate);
    ImPlot::SetNextPlotLimits(0, 1, 0, 1, ImGuiCond_Always);
    if (ImPlot::BeginPlot("##Bench", NULL, NULL, ImVec2(-1, -1), ImPlotFlags_Default)) {
        char buff[16];
        ImPlot::PushStyleVar(ImPlotStyleVar_LineWeight, 1);
        for (int i = 0; i < 100; ++i) {
            sprintf(buff, "item_%d", i);
            ImPlot::PushStyleColor(ImPlotCol_Line, items[i].Col);
            ImPlot::PlotLine(buff, items[i].Data, 1000);
            ImPlot::PopStyleColor();
        }
        ImPlot::PopStyleVar();
        ImPlot::EndPlot();
    }
    ImGui::End();
}

Results

I made the comparison between imgui/master and imgui_private/features/tex_antialiased_lines. I couldn't detect any noticeable visual differences between them. I also toggled ImDrawListFlags_AntiAliasedLinesUseTexData on/off while zooming/panning, moving plot windows, etc. I never saw anything strange. As expected, there is an appreciable performance boost using the new texture anti-aliased method as well.

You can overlay the images below in Photoshop and see that there is a very tiny difference between the two in some areas, but not enough to concern me.

imgui/master

thickness = 1 : FPS = 153

thickness = 2 : FPS = 124

imgui_private/features/tex_antialiased_lines

thickness = 1 : FPS = 169

thickness = 2 : FPS = 167

ImPlot's non-anti aliased solution which bypasses DrawList.AddLine is still quite a bit faster at 370 FPS. My preference is to use this and MSAA, but it's still nice to see an improved software AA solution! Let me know how else I can be useful.

[ocornut]

Thanks (again) for your amazing feedback :)

ImPlot's non-anti aliased solution which bypasses DrawList.AddLine is still quite a bit faster at 370 FPS

Have you tried disabling ImDrawListFlags_AntiAliasedLines in the current drawlist flag? Curious about the cost here.

[wolfpld]

I can see some differences, in the following code:

static void DrawZigZag( ImDrawList* draw, const ImVec2& wpos, double start, double end, double h, uint32_t color, float thickness = 1.f )
{
    const auto spanSz = end - start;
    if( spanSz <= h * 0.5 )
    {
        draw->AddLine( wpos + ImVec2( start, 0 ), wpos + ImVec2( start + spanSz, round( -spanSz ) ), color, thickness );
        return;
    }

    const auto p = wpos + ImVec2( 0.5f, 0.5f );
    const auto h05 = round( h * 0.5 );

    draw->PathLineTo( p + ImVec2( start, 0 ) );
    draw->PathLineTo( p + ImVec2( start + h05, -h05 ) );
    start += h05;

    const auto h2 = h*2;
    int steps = int( ( end - start ) / h2 );
    while( steps-- )
    {
        draw->PathLineTo( p + ImVec2( start + h,   h05 ) );
        draw->PathLineTo( p + ImVec2( start + h2, -h05 ) );
        start += h2;
    }

    if( end - start <= h )
    {
        const auto span = end - start;
        draw->PathLineTo( p + ImVec2( start + span, round( span - h*0.5 ) ) );
    }
    else
    {
        const auto span = end - start - h;
        draw->PathLineTo( p + ImVec2( start + h, h05 ) );
        draw->PathLineTo( p + ImVec2( start + h + span, round( h*0.5 - span ) ) );
    }

    draw->PathStroke( color, false, thickness );
}

Example 1

Flag enabled:

Flag disabled:

Some pixels are missing, doesn't really matter here.

Example 2

Enabled:

Disabled:

Thickness 1.5, rendering becomes blurry.

Other than that, everything looks the same.

[epezent]

@ocornut

Have you tried disabling ImDrawListFlags_AntiAliasedLines in the current drawlist flag? Curious about the cost here.

Yes, I have. My problem was that I couldn't use AddPolyline directly because ImPlot supports offset/circular data (and also at the time AddPolyLine was giving some artifacts for mitered corners). So, naturally I switched to AddLine() to render each individual segment separately. But I noticed this calls PrimReserve each time. Thus, I stole the non-AA part of AddPolyline, and put it in directly in my line rendering for-loop. Approximate performance of each method:

• DrawList.AddLine() w/ ImDrawListFlags_AntiAliasedLines for each segment
  • ~120-150 FPS
• DrawList.AddLine() w/o ImDrawListFlags_AntiAliasedLines for each segment
  • ~250 FPS
• Bypassing multiple calls to DrawList.AddLine and adding vtx/indices manually in a way that I only have to make one call to PrimReserve.
  • ~350 - 400 FPS

My code is here, the first half using AddLine for AA version, and the second half using my custom solution for non-AA.

[ocornut]

@wolfpld:
Thanks for the feedback. Do they look ok with 1.0f or 2.0f thickness? When not magnified and comparing, does current look with 1.5f seem too problematic to you? If you have time could you try messing with the fractional_thickness value in AddPolyline and see if you get better results at e.g. 0.0f or 1.0f or in between?

@epezent:

because ImPlot supports offset/circular data

Could you possibly make two calls to the function, or would the normal artefact at the point of jointure be too much of a problem?

I wonder if we can refactor some of that code to facilitate reusing chunks of the code with less of the fixed cost (I don't know if PrimReserve is a culprit here, most of it should be pretty lightweight provided your ImDrawList has been through one frame). Are you compiling with optimization and inlining?

(By the way any reason you still need to enable 32-bit indices?)

[epezent]

Could you possibly make two calls to the function, or would the normal artefact at the point of jointure be too much of a problem?

The idea crossed my mind but I haven't tried it yet. The main issue was the visual artifacts (small slivers shooting off screen) I saw in AddPolyline, presumably at the mitereed corners. I think it may have only pertained to the AA code however.

I wonder if we can refactor some of that code to facilitate reusing chunks of the code with less of the fixed cost (I don't know if PrimReserve is a culprit here, most of it should be pretty lightweight provided your ImDrawList has been through one frame). Are you compiling with optimization and inlining?

Yes, I have optimizations and inlining enabled. Indeed, PrimReserve should only have an effect the first frame. The biggest gain might be from avoiding the if (Flags & ImDrawListFlags_AntiAliasedLines) branch for each individual segment. I can't say for sure; I didn't profile too deeply.

(By the way any reason you still need to enable 32-bit indices?)

For the test above, I saw artifacting if it wasn't enabled. This was done isnide of your test engine with the default backend enabled.

[wolfpld]

Do they look ok with 1.0f or 2.0f thickness?

Yes, there's no visible change there.

When not magnified and comparing, does current look with 1.5f seem too problematic to you?

Yes, the fuzziness is quite obvious. Attaching un-zoomed comparison below.

If you have time could you try messing with the fractional_thickness value in AddPolyline and see if you get better results at e.g. 0.0f or 1.0f or in between?

Hardcoding this value has the same effect as adjusting fractional part in the thickness value (which is to be expected, as it is calculated as thickness - integer_thickness). With fractional_thickness=0, thickness=1.5 becomes 1; with fractional_thickness=1, thickness=1.5 becomes 2, etc. This visibly changes line weight.

[sergeyn]

(By the way any reason you still need to enable 32-bit indices?)

For the test above, I saw artifacting if it wasn't enabled. This was done isnide of your test engine with the default backend enabled.

I am working on a fix for exactly this problem. I plan to submit it a bit later this week.

[Nelarius]

No visual differences on my end, but I'm not doing anything too fancy. Effectively tested AddBezierCurve, AddCircle, AddQuad, and AddTriangle with integer and non-integer thickness both above and below 1.0.

[ocornut]

thickness both above and below 1.0.

To clarify, thickness below <1.0f are and were always treated the same as =1.0f

Forgot to mention in the initial post that this comes with a restriction:

• This works if the renderer is using bilinear interpolation on the texture rather than point/nearest filtering. The majority of back-ends and renderers are already doing so, but some embedded/low-end GPU back-ends might not. Lines won't look nicely anti-aliased without.

(Added to top-most post now)

[r-lyeh]

No differences here as far as I can tell

[ocornut]

@epezent

(By the way any reason you still need to enable 32-bit indices?)

For the test above, I saw artifacting if it wasn't enabled. This was done isnide of your test engine with the default backend enabled.

Curious if #3163 fixes it for you?
I think the PR is probably correct but I'd like us to add more formal tests before merging because it's a very little exercised piece of code.

[epezent]

@ocornut , sorry for the extremely delayed response. #3163 does not resolve the issue when using 16-bit indices. See my comment on #3232.

[ocornut]

@wolfpld Sorry for delayed reaction. We're not sure what's the best approach here.
If we can somehow find a way to improve non-integer thickness (Ben?). I wonder if we could add a way to manually selectively bake precise thickness into the atlas. Considering future work will simplify atlas texture update it could go toward being fully dynamic..

If we should make it automatic or optional to "degrade" to the existing polygon path when thickness are non-integer (worst case we are degrading to existing code, but the various paths are a little bothering..

Will get back to it..

[ShironekoBen]

Yeah, I'd really like to have a better solution for fractional line widths - geometry-based edges tend to be reasonably "clear" because the geometry is drawn with separate line/anti-aliasing widths, but as you've observed textured edges are more fuzzy as essentially the code just "crushes" the nearest integer-width texture into a smaller space, effectively uniformly scaling (and blending together) all the elements.

We may be able to apply some tweaks (introducing intermediate textures at 0.5 increments for small widths?) to try and minimise that, but the issue I keep coming back to is that regardless of any of that the differences in sampling behavior between geometry edges and texture edges means that any solution based on sometimes drawing lines one way and sometimes drawing them the other is extremely likely to exhibit inconsistencies - we can kinda get away with a bit of that with integer coordinates/widths because (a) in theory the sizes are "right" either way, (b) you have a full pixel worth of step between valid values and (c) they're much more amenable to hand-tuning, but as soon as you have a scenario where there's a cut-off point - e.g. "everything <4px is a texture and everything >=4px is geometry" then there's potential for someone to try and draw a shape that involves a 3.99px (texture) wide line meeting a 4.0px (geometry) wide line with a visible discontinuity.

The least-worst solution I've come up in terms of consistency with thus far is to never allow line edges to be geometry, and always ensure that they are textures even if that means adding extra polys in the middle to "fill out" the line... but in terms of the "fuzziness" problem that's actually a step backwards because it would unify everything onto the texture path... :-(

[mkalte666]

• Linux 64 bit
• nvidia 980ti
• amd ryzen 5 2600

Works fine on my end. Performance wise i am seeing an increase. No obvious visual differences for me.
Great work!

https://imgur.com/a/Zj3uEW3

Rendering takes a roughly half the time as far as i can see, which probably really helps on weaker machines, but i don't have one on had right now :/

[ocornut]

Ben pushed a tweak to make this branch "non-regressing": texture based lines are currently only used when thickness is an integer value, so we are able to merge it. (edit: clarification, this is now indeed merged)

The change would also technically break modification of AA_SIZE as done by thedmd@39cde5c for scaling, so those patches will need to add a && (AA_SIZE == 1.0f) test in AddPolyline() when determining the value of bool use_texture. (ping @thedmd, @rokups)

Also merged separately of the applicable renaming/comments/shallow stuff introduced with this branch, then rebased over, so the merged branch is as lean as possible.

This relatively small changes are not overly exciting right now but they will largely facilitate use of thick borders, and will be one (among many) of the building blocks toward redesigning and improving the style system (thanks to NVIDIA @xteo for their contribution on this and many other upcoming changes aligned with this vision).

@epezent ImPlot may be able to make use of the new UV data provided in TexUvLines[] in its own line renderer (will add a little bit of complexity in your line renderer but you can benefit from cheaper thick lines).

[thedmd]

Fixed my fringe scale. Thanks for an update.

There is a difference in drawing. Look ok to me.

[ocornut]

Minor amend, stripped out dead-code since are only using the texture-bsaed based for integer width: 9801c8c

[ocornut]

Closed as implemented/solved.