[GUI] [linux] Support "ti.GUI(fast_gui=True)" for zero-copy while gui.set_image(img) #1922

archibate · 2020-10-05T07:29:42Z

Related issue = depends #1921

We can now obtain 60fps even with 1920x1080!
This would allow me to implement a real-time soft renderer in Taichi THREE!

Iteravse · 2020-10-05T09:18:13Z

tremendous progress!!!
it solved the question i was gonna ask
waiting for windows version 👍

k-ye

Could you do a merge master first? That should shrink the size of the PR

k-ye · 2020-10-06T09:38:00Z

python/taichi/lang/meta.py

+        u, v, w = min(255, max(0, int(img[i, img.shape[1] - 1 - j] * 255)))
+        # We use i32 for |out| since OpenGL and Metal doesn't support u8 types
+        # TODO: treat Cocoa and Big-endian machines, with XOR logic
+        out[j * 1920 + i] = w + (v << 8) + (u << 16)


img.shape[1]?

img.shape[0] actually.
Btw, I saw you use RGB order in cocoa.cpp, and BGR order in x11.cpp and win32.cpp. I want to know if mac actually use big-endian? If so, no changes need to be done here. Otherwise ti.static(ti.get_os_name() == 'osx') would be used.

k-ye · 2020-10-06T09:38:13Z

python/taichi/lang/meta.py

@@ -21,6 +21,16 @@ def cook_image_type(x):
    return x


+@ti.kernel
+def vector_to_fast_image(img: ti.template(), out: ti.ext_arr()):
+    # FIXME: Why is ``for i, j in img:`` slower than:


Do you know how much it was slower?

Did you reproduce it? I'm building right now. Will tell you asap. This is likely an issue of row-major / col-major, wdyt?

No worries. I was just curious if the slowness was just some noise or not. It does sound like row-major vs column-major could be a factor. Thanks for confirming!

Did you reproduce it?

I tried running the example, but all i got was a black screen.. (Tried both RGB and BGR). Given that the original image was of type ti.f32, maybe that could be the reason? (Or do i actually need to pass fast_buf somewhere for Cocoa?)

python/taichi/misc/gui.py

codecov · 2020-10-10T13:07:50Z

Codecov Report

Merging #1922 into master will increase coverage by 0.85%.
The diff coverage is 23.07%.

@@            Coverage Diff             @@
##           master    #1922      +/-   ##
==========================================
+ Coverage   42.80%   43.65%   +0.85%     
==========================================
  Files          45       45              
  Lines        6413     6226     -187     
  Branches     1101     1107       +6     
==========================================
- Hits         2745     2718      -27     
+ Misses       3498     3334     -164     
- Partials      170      174       +4

Impacted Files	Coverage Δ
python/taichi/misc/gui.py	`8.89% <19.04%> (ø)`
python/taichi/lang/meta.py	`62.82% <40.00%> (ø)`
...ld/taichi-dev/taichi/python/taichi/tools/np2ply.py
...build/taichi-dev/taichi/python/taichi/lang/util.py
...uild/taichi-dev/taichi/python/taichi/tools/file.py
...build/taichi-dev/taichi/python/taichi/lang/meta.py
.../build/taichi-dev/taichi/python/taichi/torch_io.py
...build/taichi-dev/taichi/python/taichi/lang/core.py
...ild/taichi-dev/taichi/python/taichi/lang/kernel.py
...uild/taichi-dev/taichi/python/taichi/cc_compose.py
... and 82 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4fa0d60...0955b07. Read the comment docs.

k-ye

Thanks, LGTM + nits!

python/taichi/misc/gui.py

k-ye · 2020-10-11T07:12:20Z

python/taichi/lang/meta.py

@@ -21,6 +21,16 @@ def cook_image_type(x):
    return x


+@ti.kernel
+def vector_to_fast_image(img: ti.template(), out: ti.ext_arr()):
+    # FIXME: Why is ``for i, j in img:`` slower than:


No worries. I was just curious if the slowness was just some noise or not. It does sound like row-major vs column-major could be a factor. Thanks for confirming!

Did you reproduce it?

I tried running the example, but all i got was a black screen.. (Tried both RGB and BGR). Given that the original image was of type ti.f32, maybe that could be the reason? (Or do i actually need to pass fast_buf somewhere for Cocoa?)

docs/gui.rst

taichi/gui/gui.h

archibate · 2020-10-11T07:58:03Z

Or do i actually need to pass fast_buf somewhere for Cocoa?

Yes, you need. This PR is only for x11.

taichi/python/export_visual.cpp

taichi/gui/gui.h

python/taichi/misc/gui.py

Co-authored-by: Ye Kuang <[email protected]>

archibate · 2020-10-12T09:11:48Z

Merging! FF2 do the Cocoa and Win32 part iapr :)

archibate requested review from yuanming-hu, k-ye, JYLeeLYJ, thinking-tower and taichi-gardener October 5, 2020 07:29

archibate added the depends-other label Oct 5, 2020

archibate marked this pull request as draft October 5, 2020 08:07

archibate self-assigned this Oct 5, 2020

archibate mentioned this pull request Oct 5, 2020

VideoManager doesn't work #1923

Closed

archibate marked this pull request as ready for review October 8, 2020 06:10

k-ye reviewed Oct 9, 2020

View reviewed changes

archibate and others added 5 commits October 10, 2020 21:07

setup fast_gui

324646c

finialize with gui.set_image

5d64429

[skip ci] enforce code format

3a7aa77

[skip ci] update doc

eba8cff

fix 1920

9a8a53e

archibate force-pushed the gui-zerocopy branch from 91507a9 to 9a8a53e Compare October 10, 2020 13:07

archibate requested a review from k-ye October 10, 2020 13:11

k-ye approved these changes Oct 11, 2020

View reviewed changes

archibate commented Oct 11, 2020

View reviewed changes

taichi/python/export_visual.cpp Outdated Show resolved Hide resolved

taichi/gui/gui.h Outdated Show resolved Hide resolved

taichi/gui/gui.h Outdated Show resolved Hide resolved

taichi/gui/gui.h Outdated Show resolved Hide resolved

python/taichi/misc/gui.py Outdated Show resolved Hide resolved

Apply suggestions from code review

0955b07

Co-authored-by: Ye Kuang <[email protected]>

archibate merged commit 1b024a0 into taichi-dev:master Oct 12, 2020

archibate mentioned this pull request Oct 14, 2020

[windows] [gui] Support ti.GUI(fast_gui=True) on Win32 #1953

Merged

yuanming-hu mentioned this pull request Oct 14, 2020

[release] v0.6.41 #1956

Merged

yuanming-hu mentioned this pull request May 7, 2021

[GUI] ImGUI, Separating backend and frontend, zero-transfer presentation, and GUI Python API #2326

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[GUI] [linux] Support "ti.GUI(fast_gui=True)" for zero-copy while gui.set_image(img) #1922

[GUI] [linux] Support "ti.GUI(fast_gui=True)" for zero-copy while gui.set_image(img) #1922

archibate commented Oct 5, 2020

Iteravse commented Oct 5, 2020 •

edited

Loading

k-ye left a comment

k-ye Oct 6, 2020

archibate Oct 10, 2020

k-ye Oct 6, 2020

archibate Oct 10, 2020

k-ye Oct 11, 2020

codecov bot commented Oct 10, 2020 •

edited

Loading

k-ye left a comment

k-ye Oct 11, 2020

archibate commented Oct 11, 2020

archibate commented Oct 12, 2020

[GUI] [linux] Support "ti.GUI(fast_gui=True)" for zero-copy while gui.set_image(img) #1922

[GUI] [linux] Support "ti.GUI(fast_gui=True)" for zero-copy while gui.set_image(img) #1922

Conversation

archibate commented Oct 5, 2020

Iteravse commented Oct 5, 2020 • edited Loading

k-ye left a comment

Choose a reason for hiding this comment

k-ye Oct 6, 2020

Choose a reason for hiding this comment

archibate Oct 10, 2020

Choose a reason for hiding this comment

k-ye Oct 6, 2020

Choose a reason for hiding this comment

archibate Oct 10, 2020

Choose a reason for hiding this comment

k-ye Oct 11, 2020

Choose a reason for hiding this comment

codecov bot commented Oct 10, 2020 • edited Loading

Codecov Report

k-ye left a comment

Choose a reason for hiding this comment

k-ye Oct 11, 2020

Choose a reason for hiding this comment

archibate commented Oct 11, 2020

archibate commented Oct 12, 2020

Iteravse commented Oct 5, 2020 •

edited

Loading

codecov bot commented Oct 10, 2020 •

edited

Loading