Development tools

[Etiennefaivredarcier]

I have created a tool to simplify the development between a glTF file and the unity application.

This tool is still under development but first target the IBC demo deadline (mid september), and has vocation to be merged into the core rt-xr/glTFast repository once done. It resolves the issue here (5G-MAG/rt-xr-unity-player#41)

In order to test this new tool, here is an application code example to use :

This simple script allow to place a sofa model on the detected floor (that will appear in a transparent yellow color) with the help of a mobile device (tested on an android tablet).
Please first:

• Make sure that the StartApplication method is called once the glTF file has loaded completely (after gltfsceneViewer.Load)
• Use the asset from this PR (Development tools rt-xr-content#25)

using rt.xr.unity;
using UnityEngine;
using GLTFast;
using System;
using UnityEngine.XR.ARFoundation;
using System.Collections.Generic;

namespace UnityPlayer.Application
{
    public class Application : MonoBehaviour
    {
        private ARRaycastManager m_RaycastManager;
        private List<ARRaycastHit> m_RaycastHitList;
        private Camera m_Camera;

        private bool m_IsAnchored;
        private bool m_IsPressed;
        private GameObject m_AnchoredGameObject;
        private Vector2 m_LastMousePos;

        public void StartApplication()
        {
            UserInputModule.GetInstance().Register(OnUserInput, 0);
            ActionModule.GetInstance().Register(OnActionManipulate, 0);
            TrackableModule.GetInstance().Register(OnTrackableEvent, 0);
        }

        private void OnActionManipulate(MPEG_ActionEvent _event)
        {
            MPEG_ActionManipulateEvent _manipulate = (MPEG_ActionManipulateEvent)_event;
            m_LastMousePos = _manipulate.inputAction.ReadValue<Vector2>();
        }

        private void OnTrackableEvent(MPEG_TrackableEvent _event)
        {
            if(_event.trackableType == GLTFast.Schema.TrackableType.TRACKABLE_FLOOR)
            {
                if (_event.trackableEventType == TrackableEventType.ADDED)
                {
                    m_IsAnchored = true;
                    m_AnchoredGameObject = VirtualSceneGraph.GetGameObjectFromIndex(3);
                    m_AnchoredGameObject.transform.localScale *= 0.95f;

                    m_RaycastManager = FindObjectOfType<ARRaycastManager>();
                    if(m_RaycastManager == null)
                    {
                        m_RaycastManager = gameObject.AddComponent<ARRaycastManager>();
                    }

                    m_RaycastHitList = new List<ARRaycastHit>();
                    m_Camera = Camera.main;
                }
            }
        }

        private void Update()
        {
            if(!m_IsAnchored)
                return;

            if(!m_IsPressed)
                return;

            Ray _ray = m_Camera.ScreenPointToRay(m_LastMousePos);

            if (m_RaycastManager.Raycast(_ray, m_RaycastHitList, UnityEngine.XR.ARSubsystems.TrackableType.Planes))
            {
                Pose _hitPose = m_RaycastHitList[0].pose;
                Vector3 _offset = new Vector3(0, -0.2f, 0);
                m_AnchoredGameObject.transform.position = _hitPose.position + _offset;
            }
        }

        private void OnUserInput(MPEG_UserInputEvent _event)
        {
            m_IsPressed = _event.isPerformed;
        }
    }
}

[nlsdvl]

Thank you for sharing your progress on the tools you are developing. As discussed during last week's call, you consider this PR to be a draft. What exactly do you plan to add to it in order to have these utility classes integrated ?

I understand the code sample you provide is designed to work witgh a specific scene document, combining interactivity and anchoring. It has hardcoded m_AnchoredGameObject - the object that is signaled to be manipulated in the scene - by getting it from an hardcoded index in the VirtualSceneGraph.

Shouldn't the target for 'action manipulate' be retrieved from the action's definition as signaled in the scene instead - here ?

The MPEG_ActionManipulateEvent holds a reference to a Unity​Engine.​Input​System.Input​Action, it would also make sens for it to hold reference to objects that need to be manipulated, or maybe in your framework, the node indices that can be used to retrieve the gameObject from VirtualSceneGraph. Then the application can manipulate these objects, eventualy taking into account constraints such as anchoring.

[nlsdvl]

Is this covered by a specification ?

[Etiennefaivredarcier]

No, this is to work with Unity input path system, and it need to be updated to support an openXR path

[nlsdvl]

Shouldn't the action hold a reference to the nodes that need to be manipulated as specified in the glltf document ?

[Etiennefaivredarcier]

This is a good idea, this PR provides a base/draft implementation to help for the IBC demo, any improvement is welcome

[Etiennefaivredarcier]

Hello,

Thank you for your review. To answer your questions:

What exactly do you plan to add to it in order to have these utility classes integrated ?

I am planning to add more fields in the events structures, such as game object references, and more handy tools.

Shouldn't the target for 'action manipulate' be retrieved from the action's definition as signaled in the scene instead

Yes it could, and this may be more handy, but for the IBC demo it should be fine