Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Syslog ng ctl attach debugger #327

Merged
merged 9 commits into from
Nov 12, 2024
Merged

Syslog ng ctl attach debugger #327

merged 9 commits into from
Nov 12, 2024

Conversation

bazsi
Copy link
Member

@bazsi bazsi commented Oct 6, 2024

This is a continuation of #326 to add interactive debugger functionality over syslog-ng-ctl

@bazsi bazsi force-pushed the syslog-ng-ctl-attach-debugger branch 3 times, most recently from cd645a3 to acc366e Compare October 12, 2024 20:01
@bazsi bazsi changed the title WIP: Syslog ng ctl attach debugger Syslog ng ctl attach debugger Oct 13, 2024
@bazsi bazsi mentioned this pull request Oct 13, 2024
Merged
@bazsi bazsi force-pushed the syslog-ng-ctl-attach-debugger branch 2 times, most recently from ca8b036 to e058b86 Compare October 27, 2024 11:40
@bazsi bazsi force-pushed the syslog-ng-ctl-attach-debugger branch from e058b86 to 17a2e2e Compare November 7, 2024 10:50
@alltilla alltilla mentioned this pull request Nov 7, 2024
Merged
bazsi added 9 commits November 7, 2024 15:53
@bazsi @alltilla
debugger: add cancellation support to Tracer
597b187 
The tracer is responsible with the communication between the debugger
thread and any worker threads. Up to this point this was not cancellable,
e.g. once a breakpoint fired, the worker stopped with no way of
cancelling this.

With the addition to tracer_cancel() pending breakpoints are let go and
at the same time tracer_wait_for_breakpoint() returns as well with
a gboolean to indicate whether it was cancelled.

Signed-off-by: Balazs Scheidler <[email protected]>
@bazsi @alltilla
debugger: add debugger_exit()
b50b525 
The new debugger_exit() can be called from any thread, triggers the
cancellation of the tracer and then waits for the debugger thread
to finish.

Signed-off-by: Balazs Scheidler <[email protected]>
@bazsi @alltilla
debugger: rename interactive_thread to debugger_thread
3b64bcd 
Signed-off-by: Balazs Scheidler <[email protected]>
@bazsi @alltilla
debugger: attach and deattach the single step hook in a synchronized …
a21b3d9 
…manner

The debugger installed the pipe_single_step_hook without locking, but that
only works if we do that at startup. Since I want to be able to
start the debugger on demand, this needs to sync with any workers that
might be executing.

On x86 it might be safe to just store the pointer, but other less
forgiving architectures (e.g. ARM) may not like that. Let's do this
properly.

Signed-off-by: Balazs Scheidler <[email protected]>
@bazsi @alltilla
debugger: add debugger_stop() call alongside debugger_start()
dd6ba5a 
Signed-off-by: Balazs Scheidler <[email protected]>
@bazsi @alltilla
debugger: make sure that debugger_start_console and debugger_exit() r…
2c138cf 
…un from the main thread

Signed-off-by: Balazs Scheidler <[email protected]>
@bazsi @alltilla
tracer: move the Tracer struct to the implementation file
574d7a9 
Signed-off-by: Balazs Scheidler <[email protected]>
@bazsi @alltilla
debugger: support multi-threaded mode
7fa251e 
The key ingredient to support multi-threaded debugger session is to allow
the breakpoint sites to submit debugger_stop_on_breakpoint() calls
in parallel so that these requests are queued and no variables are
used for the different invocations.

The solution is to introduce the BreakpointSite struct, this is where
we store state that relates to each distinct breakpoint events.

This struct contains both the state the debugger may want to inspect (e.g.
msg, pipe) and also  the state related to inter-thread communication
(e.g. resume_requested).

Each such site instances are queued in an internal queue which is then
processed by the interactive debugger.

Signed-off-by: Balazs Scheidler <[email protected]>
@bazsi @alltilla
syslog-ng-ctl: add support for "attach debugger"
6e93568 
Signed-off-by: Balazs Scheidler <[email protected]>
@alltilla alltilla force-pushed the syslog-ng-ctl-attach-debugger branch from 17a2e2e to 6e93568 Compare November 7, 2024 14:55
alltilla
alltilla reviewed Nov 12, 2024
View reviewed changes
lib/logpipe.c Outdated Show resolved Hide resolved
lib/mainloop-control.c Show resolved Hide resolved
@alltilla
Copy link
Member

I have found a crash:

syslog-ng.conf

log "internal_messages_to_stderr" {
  source{ internal(); };
  destination {
    file("/dev/stderr");
  };
};

log {
  source { example-msg-generator(); };

  rewrite {
    set("syslog-ng-test" value(topic));
  };
};

terminal 1

./install/sbin/syslog-ng -Fe

terminal 2

./install/sbin/syslog-ng-ctl attach debugger

terminal 3

./install/sbin/syslog-ng-ctl reload

terminal 2

c
c

terminal 1

CRASH

Thread 4 "syslog-ng" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffef4006c0 (LWP 90602)]
0x00007ffff7f1f79e in zone_info_get_offset (self=0xeabc27d7444505cf, timestamp=1731398027) at ../lib/timeutils/zoneinfo.c:448
448       if (self->transitions == NULL)
(gdb) bt
#0  0x00007ffff7f1f79e in zone_info_get_offset (self=0xeabc27d7444505cf, timestamp=1731398027) at ../lib/timeutils/zoneinfo.c:448
#1  0x00007ffff7f1fc3e in time_zone_info_get_offset (self=0x555555617c30, stamp=1731398027) at ../lib/timeutils/zoneinfo.c:530
#2  0x00007ffff7eed090 in log_macro_expand_date_time (id=28, options=0x7fffef3fd690, msg=0x55555561d320, result=0x7fffe8002310, type=0x7fffef3fd1c8 "") at ../lib/template/macros.c:347
#3  0x00007ffff7eefec3 in log_macro_expand (id=28, options=0x7fffef3fd690, msg=0x55555561d320, result=0x7fffe8002310, type=0x7fffef3fd4ab "\377") at ../lib/template/macros.c:729
#4  0x00007ffff7ef0a37 in log_template_append_elem_macro (self=0x7fffe8001f10, e=0x7fffe8002040, options=0x7fffef3fd690, msg=0x55555561d320, type=0x7fffef3fd52b "\377\001", result=0x7fffe8002310)
    at ../lib/template/eval.c:92
#5  0x00007ffff7ef1172 in log_template_append_format_value_and_type_with_context (self=0x7fffe8001f10, messages=0x7fffef3fd5c0, num_messages=1, options=0x7fffef3fd690, result=0x7fffe8002310, type=0x0)
    at ../lib/template/eval.c:197
#6  0x00007ffff7ef13e1 in log_template_append_format_value_and_type (self=0x7fffe8001f10, lm=0x55555561d320, options=0x7fffef3fd690, result=0x7fffe8002310, type=0x0) at ../lib/template/eval.c:251
#7  0x00007ffff7ef14a0 in log_template_format_value_and_type (self=0x7fffe8001f10, lm=0x55555561d320, options=0x7fffef3fd690, result=0x7fffe8002310, type=0x0) at ../lib/template/eval.c:265
#8  0x00007ffff7ef1568 in log_template_format (self=0x7fffe8001f10, lm=0x55555561d320, options=0x7fffef3fd690, result=0x7fffe8002310) at ../lib/template/eval.c:280
#9  0x00007ffff7f065b1 in _display_msg_with_template (self=0x7fffe8001e50, msg=0x55555561d320, template=0x7fffe8001f10) at ../lib/debugger/debugger.c:88
#10 0x00007ffff7f06f20 in _handle_interactive_prompt (self=0x7fffe8001e50) at ../lib/debugger/debugger.c:357
#11 0x00007ffff7f06fc5 in _debugger_thread_func (self=0x7fffe8001e50) at ../lib/debugger/debugger.c:380
#12 0x00007ffff7d03c82 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#13 0x00007ffff7a9ca94 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447
#14 0x00007ffff7b29c3c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

@alltilla alltilla merged commit 2d00a2f into axoflow:main Nov 12, 2024
22 checks passed
@alltilla alltilla mentioned this pull request Nov 12, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alltilla alltilla alltilla approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@bazsi @alltilla