Skip to content
This repository has been archived by the owner on Nov 8, 2021. It is now read-only.

Positional tracking and controllers.... BEST streamer VR ever #8

Closed
chapeka opened this issue May 29, 2018 · 29 comments
Closed

Positional tracking and controllers.... BEST streamer VR ever #8

chapeka opened this issue May 29, 2018 · 29 comments

Comments

@chapeka
Copy link

chapeka commented May 29, 2018

Hi. I am very impressed with your work, this is another level of streaming vr, after a lot of tests the application is very functional in OCULUS GO, the next step would be the positioning and the controllers, have you thought of any solution to integrate?
I will be very aware of your development, I would be delighted and it would be an honor to be able to collaborate with you as much as I can, on my YOUTUBE channel I have already made a video of installation and presentation of the application:

https://www.youtube.com/watch?v=UfljMojX17Y

@polygraphene
Copy link
Owner

polygraphene commented May 29, 2018

Thank you for your comment and your great work!
That is a very impressive guide!

Oculus Go 3DoF controller is now usable in latest release.

Positional tracking is very difficult issue. Riftcat has got it with NOLO, but I don't know whether NOLO is the best solution for us. (I heard tracking of NOLO is laggy.)

Positional tracking is also achieved by Google ARCore for Gear VR. (please refer https://www.youtube.com/watch?v=4EWPUdE_kqU )
This is portable (inside-out tracking) solution, but Oculus Go cannot use ARCore because of no camera.

And 6DoF tracking may cause the lag of head tracking, because Asynchronous Timewarp of Oculus Go only supports 3DoF tracking.

@Lightsteed
Copy link

good work on the controller, however you may want to make a way to reposition the controller. im currently having to send a debub pos. of 1 on the y axis to move the hmd to an appropriate height, but the controller stays at or below zero, making it useless in many cases.

@polygraphene
Copy link
Owner

Hmm..
position of controller and HMD is a confusing issue for me.
In addition to your issue, we need to fix bug of recentering (#18).
We need more time to solve.

@polygraphene
Copy link
Owner

I've found the great video.
https://www.youtube.com/watch?v=kHGrOQb1o_o
The author got 6DoF tracking in ALVR with Kinect.

@Lightsteed
Copy link

yep that's actually the video that got me to try out ALVR, i dont know how he's doing though, driver4VR didnt seem to work with ALVR for me, but that may have been an issue with the NOLO that i tried as its quite buggy. in any case, that program is not designed to be able to remap things like vive trackers onto a HMD pose which is what i really want to do. Sidenote: i realised that i need to correctly set up standing room only in steamVR with the correct HMD to floor offset (player height) in order for the HMD to be set off from the floor when using ALVR, unfortunately though it does not offset the controller in steam home for some reason, but when i run apps the controller is in roughly the right position.. it is crashing a lot now though, cant get space pirate trainer to run for more than a few seconds, and when it crashes, it requires a reboot to make it work again as steamVR wont lead properly again. Sorry for mixing up the issues. let me know if you want me to make seperate issue posts.

@penkamaster
Copy link

penkamaster commented May 31, 2018

i got a vive, and I can use the controllers with the Go on steam VR.

I want to use one of the controllers for the pose of the GO, I have tried https://github.com/matzman666/OpenVR-InputEmulator/releases although this program show the possibility it fails when I do the redirection.

Other possibilities is buying a vive tracker, but it presents himself as a controller, I've read in some place about valve giving the documentation in order to flash the tracker to presents himself as a headset, but is all the info I've got.

Any thoughts?

@petermg
Copy link

petermg commented May 31, 2018 via email

@Lightsteed
Copy link

interesting. i can't even get alvr to display if driver4vr is installed on the latest version. steamvr bugs out

@chapeka
Copy link
Author

chapeka commented Jun 1, 2018

The probem with NOLO VR is the perceptible LAG in the tracking, I wait that with antilatency we can solve

@IsaiahN
Copy link

IsaiahN commented Jun 2, 2018

@polygraphene - a user on reddit was able to get a camera working with the oculus go:

https://www.reddit.com/r/OculusGo/comments/8l1j81/adhesived_a_camera_on_my_go_now_what_do/

Without otg needed and got the footage via sideloaded camera app.

Could this combo work with arcore for gearvr? To get positional tracking?

@chapeka
Copy link
Author

chapeka commented Jun 2, 2018

May be possible more resolution in the stream¿?¿, may be 2560x1440¿?? using NVENC it could be possible

@polygraphene
Copy link
Owner

Oculus SDK "suggests" 1024x1024 eye texture. (2048x1024 for both eyes)
https://github.com/polygraphene/ALVRClient/blob/master/app/include/VrApi_Types.h#L251
https://developer.oculus.com/documentation/mobilesdk/1.0.4/concepts/mobile-rendering/

Most game style 3D VR content should target 1024x1024 eye buffers. At this resolution, pixels will be slightly stretched in the center, and only barely compressed at the edges, so mipmap generation is unnecessary. If you have lots of performance headroom, you can experiment with increasing this a bit to take better advantage of the display resolution, but it is costly in power and performance.

If we add supports for larger resolution, the effect is limited.
So we may implement it in future, but the priority is low.

@iamapie13
Copy link

well I've been using PSMoveService together with ALVR and it works great, the PS Move controllers work nice for 6Dof hand tracking and with a third controller you can also get positional tracking, of course the only downside is that not everyone can mount 4 PS3 cameras to the ceiling, but if you can like me this is a great set up, I've been using it for more than a year with riftcat, but now with the integration of this program it works great

@chapeka
Copy link
Author

chapeka commented Jun 7, 2018

I think this version 1.9.2 makes great again this software, its is incredible the new resolutions and the sensors emulation, only need controllers working great like nolo VR and antilatency and yo make gods of VR

@TheJaeger
Copy link

Chiming in to say maybe you can stream 3DOF coordinates to Steam, while offering a position tracking via freetrack. Some of us PS Move Service users use an additional glowing pingpong ball placed on HMD for position tracking for full 6DOF. Riftcat offers an option for this, it's called "Use phone sensors for tracking and freetrack for positon".

Getting this would be golden. The smoothness from this app is unreal. Ideally, ARCore would be the best option as it works without any additional equipment. Someone on YouTube made a post on how this is implemented in Unity for GearVR.

@Lightsteed
Copy link

Yep i want this option so i can use a vive tracker for position. The image isn't smooth when using a tracker for both position and rotation, i got it working with ALVR and open vr input emulator, i guess due to the ATW the dev descibed.

@TheJaeger
Copy link

TheJaeger commented Jun 10, 2018

That behavior is expected because you're completely getting rid of ATW when you use an external tracker for orintation. ATW works only when you use the HMD's sensors for orientation information. It is possible to keep the smoothness of orientation whilst adding an "outside" tracker for position. Freetrack does exactly this, it combines multiple inputs together. If you've used Riftcat with D4VR before, your orientation is tracked at 60 FPS from headset IMUs, but positon is tracked at 30 FPS via kinects. The result is still smooth tracking of orientation because position is independent of orientation for VR scenario.

The 3DOF --> 6DOF transition shouldn't break ATW if done correctly, which can be easily achieved via freetrack. Hopefully the dev can work on this as this would make it a killer app for DIY VR.

@TheJaeger
Copy link

I'm more keen to see ARCore implementation though. I tried an ARcore positional tracking demo on Gear VR and it works wonderfully, even better than my custom ping-pong ball on head. But then again, Oculus GO users will not benefit from this without some hacks.

@polygraphene
Copy link
Owner

polygraphene commented Jun 10, 2018

@IsaiahN It's interesting. It would be nice if ARCore can run on Go.

@TheJaeger
I have uploaded experimental build with ARCore support on here.
This build only supports Galaxy S8 (or maybe? S9).
And this build makes the smartphone very high temperature, so please be careful of use.

@Lightsteed
Copy link

wow, nice one. unfortunately i cant try this build as the camera port on my galaxy s8 is a bit broken and the gearVR wont start up :( what needs to be done to make it work on the Go with a usb camera?

@polygraphene
Copy link
Owner

If USB camera can be used in Oculus Go, the following patch may help us.
https://github.com/tomthecarrot/arcore-for-all

@TheJaeger
Copy link

TheJaeger commented Jun 10, 2018

@polygraphene I am really impressed at the speed of your implementation, RiftCat kept delaying this feature for so long. I'll test this when I can and let you know how it works.

EDIT: WOWOWOW. It seems to track really well in my VR space, far better than the Kinect and ping-pong ball. Once you fix recentering it will be golden. By the time I walk to my play space I'm already out of zone.

@Lightsteed
Copy link

USB camera works on the go, have tried it - followed this guys instructions and worked with both crappy usb camera i own - https://www.reddit.com/r/OculusGo/comments/8l1j81/adhesived_a_camera_on_my_go_now_what_do/

@Lightsteed
Copy link

I tried installing the patched arcore but alvr wouldnt run, a message came up saying i needed the latest version of arcore. Do you think u could build a patched version of arcore for the go with the version your client is using?

@chapeka
Copy link
Author

chapeka commented Jun 13, 2018

what can you do with the camera¿?¿? any kind of possitional tracking or software camera¿?¿

@ChristophGeske
Copy link

I have a quite good ARCore positional tracking experience in the working supporting all devices from S7 to S9 with almost no jittering. Check it out here: https://github.com/ChristophGeske/ARCoreInsideOutTrackingGearVr

@TheJaeger
Copy link

Hey @ChristophGeske, I did try your ARCore tracking couple days back and it worked wonderfully. Even the one posted in ALVR's test build worked well. The results are very promising and can help users achieve room-scale VR without base stations.

@chapeka
Copy link
Author

chapeka commented Jun 15, 2018

Can be used ChristophGeske´s ARCORE with OCULUS GO in the future with an external camera¿? working fine if possible

@ChristophGeske
Copy link

ChristophGeske commented Jun 15, 2018

Hi @chapeka . It is very unlikely that ARCore will run on an external camera because Google combines the camera and the smartphone sensor data to make tracking possible. Only a handfull of devices are supported at the moment because the adjustment needs some extra work from the side of google. This might be improved in the future but right know ARCore is not ready to run on different Cameras. Wikitude and Vuvoria are made for more camera types and phones and they also allow for positional tracking but they are not as accurate as ARCore or ArKit . Here is a project showing Vuvoria for 6DoF tracking. https://youtu.be/KIHTCo69I2I

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

9 participants