Recording in Ground Observer mode

[ostwind]

Question

What's your question?

While flying the multirotor programmatically, using for example orbit.py, how does one capture videos of the drone from the ground?

I find that when I click to render captured footage (of observing a flying drone from the ground), the playmode prompt for car vs multirotor pops up, forcing a new video capture session.

Include context on what you are trying to achieve

Thank you for a great and extensive library.

My goal is to capture drone videos for a CV application that classifies drone objects as they are flying.

Best guess is this functionality is either in what is described as Ground Observer Mode, or computer vision.

Context details

I am using Linux, python 3.6, most recent version of AirSim and Unreal 4.21

Include details of what you already did to find answers

I have run blocks project while using unreal recording functionalities, but the playmode programming (car vs multirotor prompt followed by new session) has disrupted the intended video capture, at the render movie step after everything has been captured.

[ahmed-elsaharti]

Hi @ostwind , I don't exactly understand if the problem is with the viewmode or the fact that the playmode prompt is popping up so I'll try to address both:

1. Regarding the viewmode:
   You are right. If you set the ViewMode to GroundObserver the camera will chase the drone from 6' above the ground. In ComputerVision mode, there won't be any drone being rendered AFAIK.

2. Regarding the playmode prompt:
   Not sure how you're recording your footage, but if the playmode prompt is disrupting that, you could just prevent it from popping up by setting your SimMode to Multirotor in settings.json

Hope this helps

[ostwind]

Hi @ahmed-elsaharti, thank you for the helpful response and the links. Number 2 was particularly good and I modified settings.json accordingly. Following your description and what's online, I should not use GroundObserver because the camera should be stationary, with drone flying within view. I also do not use ComputerVision mode because the drone should be rendered.

I am still missing something and would be grateful for additional helpful. When the render button in Unreal Editor is clicked, the scene is 'played' or simulated. The difficult part is, since the drone is flying programmatically, it is no longer in the same position in the scene as when I captured it - it is reset to its starting position.

What would be the most straight forward way to set a ground-level stationary camera, fly the drone in its view, then render the film? Thank you in advance.

[ahmed-elsaharti]

Hi again @ostwind, glad you found this helpful.

I'm not sure I understand the issue you're facing here is. Where do you expect the drone to spawn on render? You can change the default spawn position using the X, Y, Z, Yaw, Roll, Pitch in settings.json. If this is not what you're asking please include a screenshot or a video of the problem and I would be happy to help ^_^

Also, for the ground-level stationary camera. I dont think Airsim has a view mode that fixes the camera but i could be wrong.
I think the easiest way around this would be to create a default camera within unreal engine itself maybe.

[ahmed-elsaharti]

@ostwind , if you're still trying to solve this have a look at this comment.

Here's a quick hacky solution:
You can use Multirotor mode to simulate the drone's mission while storing data about its location and orientation (we use multirotor mode here to include physics and all)
After that's done, you could use CV mode and place a dummy drone object in the UE environment then move it using simSetObjectPose based on the data gathered from the physics simulation (this will just move the drone around based on your saved datapoints without physics here).
While that's happening you can use simSetCameraPose along with the current drone position from your stored data to point the camera in the direction of the drone without moving the camera's location.

Essentially what's happening here is that you'd be using AirSim's multirotor mode to simulate and CV mode to 'visualize/animate' the scene based on what was happening to the drone in multirotor mode.

[ostwind]

Hi ahmed,

Thanks for your help. That looks like a viable solution.

I'll close this issue for now.

[ahmed-elsaharti]

I dont think Airsim has a view mode that fixes the camera but i could be wrong.

Turns out I was wrong. With the default view mode on Airsim, if you hit the 'M' button on your keyboard it would fix the viewport camera in place: