ReadSurfaceData_Flush blocking render thread and causing stuttering

[wouter-heerwegh]

Bug report

• AirSim Version/#commit: f1ccd2d
• UE/Unity version: 4.25.4
• autopilot version:
• OS Version: Ubuntu 20.04

What's the issue you encountered?

When running the simulator, I get a steady 60 FPS, nog major drops. When running the ros wrapper (with cameras), the simulator jumps from 60 to 20 fps and back at a very regular frequency. This happens both with binaries and in the UE4editor. When looking at the profiler in Unreal, I see that the render thread is taking most of it's time, trying to flush surface data. From what I understand, this also blocks the game thread because they have to stay in sync.

I am still very new to Airsim and Unreal, so I have no clear idea on how to solve this. I did however find this repository basically explaining how to use cameras without having to flush.

Settings

{
  "SeeDocsAt": "https://github.com/Microsoft/AirSim/blob/master/docs/settings.md",
  "SettingsVersion": 1.2,
  "SimMode": "Car",
  "DefaultCar": {"PawnBP": "Class'/AirSim/VehicleAdv/SUV/SuvCarPawn.SuvCarPawn_C'"},
  "Vehicles": {
  "PhysXCar": {
	"VehicleType": "PhysXCar",
      "DefaultVehicleState": "",
      "EnableCollisionPassthrogh": false,
      "EnableCollisions": true,
      "AllowAPIAlways": true,
      "RC":{
          "RemoteControlID": -1,
		  "AllowAPIWhenDisconnected": false
      },
      "Cameras": {
        "cam1": {
          "CaptureSettings": [
          {
              "ImageType": 0,
              "Width": 785,
              "Height": 785,
              "FOV_Degrees": 90
          }
          ],
          "X": -0.3,
          "Y": -0.16,
          "Z": 0.8,
          "Pitch": 0.0,
          "Roll": 0.0,
          "Yaw": 0
      },
        "cam2": {
          "CaptureSettings": [
            {
              "ImageType": 0,
              "Width": 785,
              "Height": 785,
              "FOV_Degrees": 90
            }
          ],
          "X": -0.3,
          "Y": 0.16,
          "Z": 0.8,
          "Pitch": 0.0,
          "Roll": 0.0,
          "Yaw": 0.0
        }
      },
      "X": 2, "Y": 0, "Z": 0,
      "Pitch": 0, "Roll": 0, "Yaw": 0
    }
  }
}

How can the issue be reproduced?

1. Add cameras to your settings.json file
2. Run the simulator
3. Run the ros wrapper

Include full error message in text form

RHImethod_ReadSurfaceData_Flush taking very long

[rajat2004]

Hmm, this seems like a limitation of AirSim's current image rendering implementation, the repo you linked seems very useful, will check it out. Will probably require a rework of the pipeline being used, don't think it has been touched since a long time. #3018 is currently outdated, will try to revive it to at least get some FPS improvements. The current ROS wrapper is also I think not very optimized, at least according to various issues, that can be improved as well

[wouter-heerwegh]

Thanks for taking a look at this, I will follow the pull request so that I can try to help/test when needed

[Gastastrophe]

Just to link it, I believe this is also the cause of the issue I describe here #4120

[talha-opteran]

@rajat2004 Has there been any progress towards a fix for this?