Cannot Dynamically Disconnect And Reconnect

[ajlewis02]

Describe the bug
On version 7.0 (but not 6.1.0), the pylonsrc plugin is unstable when attempting to dynamically destroy and create plugins within the same pipeline. Specifically, the plugin seems to crash whenever attempting to connect to a camera which had been previously connected to, even if the old plugin has been totally unallocated.
To Reproduce
This python script reproduces the issue:
(Note that this script reproduces the issue in pylonsrc 7.0, but works as intended with pylonsrc 6.1. I haven't tested on other versions of pylonsrc.)

import gi, time, os, sys
from datetime import datetime
from threading import local, Thread, Lock
gi.require_version('Gst', '1.0')

from gi.repository import GLib, Gst

def bus_call(bus, message, loop):
    t = message.type
    if t == Gst.MessageType.EOS:
        sys.stdout.write("End-of-stream\n")
        loop.quit()
    elif t==Gst.MessageType.WARNING:
        err, debug = message.parse_warning()
        sys.stderr.write("Warning: %s: %s\n" % (err, debug))
    elif t == Gst.MessageType.ERROR:
        err, debug = message.parse_error()
        sys.stderr.write("Error: %s: %s\n" % (err, debug))
        now = datetime.utcnow()	
        nowstr = now.strftime('%Y-%m-%dT%H-%M-%S.%f')[:-3]
        sys.stderr.write("Time: {}\n".format(nowstr))
        loop.quit()
    return True

Gst.init(None)
pipeline = Gst.Pipeline()

def create_cam():
    pylon = Gst.ElementFactory.make("pylonsrc", "source")
    pylon.set_property('device-serial-number', '40302373')
    return pylon

def probe(pad, info):
    global pipeline, pylon, sink
    pad.remove_probe(info.id)
    pylon.set_state(Gst.State.NULL)
    pipeline.remove(pylon)
    pylon = None
    time.sleep(1)
    pylon = create_cam()
    pipeline.add(pylon)
    pylon.link(sink)
    return Gst.PadProbeReturn.PASS

sink = Gst.ElementFactory.make("fakesink", "sink")
pylon = create_cam()


pipeline.add(pylon)
pipeline.add(sink)
pylon.link(sink)

loop = GLib.MainLoop()
bus = pipeline.get_bus()
bus.add_signal_watch()
bus.connect ("message", bus_call, loop)
def run_loop_thread(loop):
    try:
        loop.run()
    except Exception as e:
        print(e)

loop_Thread = Thread(target=run_loop_thread, args=(loop,))
loop_Thread.start()

pipeline.set_state(Gst.State.PLAYING)
print("playing")
time.sleep(1)
sink.get_static_pad("sink").add_probe(Gst.PadProbeType.BLOCK, probe)
pipeline.set_state(Gst.State.PLAYING)
time.sleep(1)
pipeline.set_state(Gst.State.NULL)
print("exiting")
loop.quit()
loop_Thread.join()

Expected behavior
The expected behavior was that the new plugin would successfully connect to the camera, given that the old plugin had been unallocated and has disconnected from the camera.

Pylonsrc 6.1.0 works as expected in this regard.

Environment

• Camera: acA1920-48gc
• plugin version info from gst-inspect-1.0 pylonsrc | grep -E "Descr.*pylon.*|\s+Version\s+[0-9]+": 0.7.0-local
• Platform: Jetson Orin Devkit
• OS: operating system and version. Jetpack 5.1.1/Deepstream 6.2

[ajlewis02]

log.txt
Here's the output of GST_DEBUG=4.

[thiesmoeller]

Can confirm, that your script triggers unexpected behavior!

Testing on x86_64 -> segfault during full tear down of buffers.
Will look into it. Could be a destruction order issue...

[thiesmoeller]

Some interim thoughts: One of the big changes in 0.7.0 was to support Nvidia NVMM.
This resulted in bigger changes of the buffer management.....

In your sample code you remove the buffer creating pylonsrc, without waiting that the last buffer to leave the sink.
The segfault, I see on x86_64 is triggered, because the buffer is still in flight in the fakesink when pylon gets destructed.
You might have to wait for the pipeline to finish before removing pylonsrc....

[thiesmoeller]

This was a long one but finally we found the root cause and fixed it.

Please check the latest release that also adds Debian Ubuntu packages

https://github.com/basler/gst-plugin-pylon/releases/tag/v1.0.0