[crash] Invalid objects returned from my module function #2

sebastiencs · 2018-04-12T13:27:01Z

Hi,

I'm using emacs-module-rs in some of my projects. I just made a small module that convert a json string to a lisp object.
You can see the code here, it's very small.

When a big json object is returned, it makes emacs crash. It doesn't happen with small objects.
It seems that the lisp object is not valid:

Stack traces


0  in terminate_due_to_signal of ../../src/emacs.c:369
1  in die of ../../src/alloc.c:7434
2  in XSTRING of ../../src/lisp.h:1395
3  in SDATA of ../../src/lisp.h:1446
4  in print_object of ../../src/print.c:1930
5  in print_vectorlike of ../../src/print.c:1713
6  in print_object of ../../src/print.c:2104
7  in print of ../../src/print.c:1150
8  in Fprin1 of ../../src/print.c:648
9  in print_error_message of ../../src/print.c:964
10 in Fcommand_error_default_function of ../../src/keyboard.c:1049
11 in funcall_subr of ../../src/eval.c:2901
12 in Ffuncall of ../../src/eval.c:2821
13 in call3 of ../../src/eval.c:2688
14 in cmd_error_internal of ../../src/keyboard.c:1004
15 in read_process_output_error_handler of ../../src/process.c:5803
16 in internal_condition_case_1 of ../../src/eval.c:1352
17 in read_and_dispose_of_process_output of ../../src/process.c:6008
18 in read_process_output of ../../src/process.c:5919
19 in wait_reading_process_output of ../../src/process.c:5616
20 in kbd_buffer_get_event of ../../src/keyboard.c:3822
21 in read_event_from_main_queue of ../../src/keyboard.c:2157
22 in read_decoded_event_from_main_queue of ../../src/keyboard.c:2221
23 in read_char of ../../src/keyboard.c:2808
24 in read_filtered_event of ../../src/lread.c:674
25 in Fread_event of ../../src/lread.c:786
26 in funcall_subr of ../../src/eval.c:2901
27 in Ffuncall of ../../src/eval.c:2821
28 in exec_byte_code of ../../src/bytecode.c:632
29 in funcall_lambda of ../../src/eval.c:3022
30 in Ffuncall of ../../src/eval.c:2823
31 in exec_byte_code of ../../src/bytecode.c:632
32 in funcall_lambda of ../../src/eval.c:3022
33 in Ffuncall of ../../src/eval.c:2823
34 in exec_byte_code of ../../src/bytecode.c:632
35 in funcall_lambda of ../../src/eval.c:3022
36 in Ffuncall of ../../src/eval.c:2823
37 in exec_byte_code of ../../src/bytecode.c:632
38 in funcall_lambda of ../../src/eval.c:3022
39 in Ffuncall of ../../src/eval.c:2823
40 in exec_byte_code of ../../src/bytecode.c:632
41 in funcall_lambda of ../../src/eval.c:3022
42 in Ffuncall of ../../src/eval.c:2823
43 in exec_byte_code of ../../src/bytecode.c:632
44 in funcall_lambda of ../../src/eval.c:3022
45 in Ffuncall of ../../src/eval.c:2823
46 in exec_byte_code of ../../src/bytecode.c:632
47 in funcall_lambda of ../../src/eval.c:3022
48 in Ffuncall of ../../src/eval.c:2823
49 in Fapply of ../../src/eval.c:2441
50 in funcall_subr of ../../src/eval.c:2876
51 in Ffuncall of ../../src/eval.c:2821
52 in exec_byte_code of ../../src/bytecode.c:632
53 in funcall_lambda of ../../src/eval.c:3022
54 in Ffuncall of ../../src/eval.c:2823
55 in call1 of ../../src/eval.c:2672
56 in timer_check_2 of ../../src/keyboard.c:4333
57 in timer_check of ../../src/keyboard.c:4395
58 in readable_events of ../../src/keyboard.c:3352
59 in get_input_pending of ../../src/keyboard.c:6808
60 in swallow_events of ../../src/keyboard.c:4135
61 in sit_for of ../../src/dispnew.c:5783
62 in read_char of ../../src/keyboard.c:2723
63 in read_key_sequence of ../../src/keyboard.c:9150
64 in command_loop_1 of ../../src/keyboard.c:1370
65 in internal_condition_case of ../../src/eval.c:1332
66 in command_loop_2 of ../../src/keyboard.c:1111
67 in internal_catch of ../../src/eval.c:1097
68 in command_loop of ../../src/keyboard.c:1090
69 in recursive_edit_1 of ../../src/keyboard.c:696
70 in Frecursive_edit of ../../src/keyboard.c:767
71 in main of ../../src/emacs.c:1720

0  in terminate_due_to_signal of ../../src/emacs.c:369
1  in emacs_abort of ../../src/sysdep.c:2426
2  in print_vectorlike of ../../src/print.c:1759
3  in print_object of ../../src/print.c:2104
4  in print of ../../src/print.c:1150
5  in Fprin1 of ../../src/print.c:648
6  in print_error_message of ../../src/print.c:964
7  in Fcommand_error_default_function of ../../src/keyboard.c:1049
8  in funcall_subr of ../../src/eval.c:2901
9  in Ffuncall of ../../src/eval.c:2821
10 in call3 of ../../src/eval.c:2688
11 in cmd_error_internal of ../../src/keyboard.c:1004
12 in read_process_output_error_handler of ../../src/process.c:5803
13 in internal_condition_case_1 of ../../src/eval.c:1352
14 in read_and_dispose_of_process_output of ../../src/process.c:6008
15 in read_process_output of ../../src/process.c:5919
16 in wait_reading_process_output of ../../src/process.c:5616
17 in sit_for of ../../src/dispnew.c:5824
18 in read_char of ../../src/keyboard.c:2723
19 in read_key_sequence of ../../src/keyboard.c:9150
20 in command_loop_1 of ../../src/keyboard.c:1370
21 in internal_condition_case of ../../src/eval.c:1332
22 in command_loop_2 of ../../src/keyboard.c:1111
23 in internal_catch of ../../src/eval.c:1097
24 in command_loop of ../../src/keyboard.c:1090
25 in recursive_edit_1 of ../../src/keyboard.c:696
26 in Frecursive_edit of ../../src/keyboard.c:767
27 in main of ../../src/emacs.c:1720

Am I doing something wrong in my code ?
Is it the conversion from rust to emacs that fails somewhere ?
Or do you think it's a bug in emacs ?

Note that git-tube also makes crash my emacs when the buffer contains lots of diffs

The text was updated successfully, but these errors were encountered:

ubolonton · 2018-04-13T02:56:32Z

Interesting. I'll check it out.

sebastiencs · 2018-04-13T02:57:54Z

Thanks !

ubolonton · 2018-04-13T15:50:46Z

Can you upload an input string that causes the crash somewhere?

ubolonton · 2018-04-14T04:06:15Z

The associated Lisp code that uses the lib would also help.

ubolonton · 2018-04-14T04:22:30Z

I tried this on a large JSON array (10k elements, 7MB) and got a similar crash when trying to print out the whole returned list. It looks like Emacs has problem printing it.

sebastiencs · 2018-04-14T04:22:49Z

Sure, I will provide a reproducible case, I should have done that first.
I will do that tomorrow

sebastiencs · 2018-04-14T09:54:54Z

I have attached a json file, and here is a lisp code that make emacs crash:

(let* ((buffer (find-file-noselect "data.json"))
       (json (emacs-json-parse-string
              (with-current-buffer buffer (buffer-string))))
       (params (gethash "params" json))
       (symbols (gethash "symbols" params)))
  (maphash (lambda (key val)
             (equal key "test")
             (equal val "test"))
           symbols))

The stack trace with this code :

Stack traces

Stack trace of thread 4946:

#0  0x00007efe89845050 raise (libpthread.so.0)
#1  0x000000000050679f terminate_due_to_signal (emacs-27.0.50)
#2  0x0000000000520d30 emacs_abort (emacs-27.0.50)
#3  0x00000000005a64be print_object (emacs-27.0.50)
#4  0x00000000005a4f54 print_object (emacs-27.0.50)
#5  0x00000000005a1d2a Fprin1 (emacs-27.0.50)
#6  0x00000000005a37c0 print_error_message (emacs-27.0.50)
#7  0x00000000005a352c Ferror_message_string (emacs-27.0.50)
#8  0x00000000005856a6 signal_or_quit (emacs-27.0.50)
#9  0x0000000000585308 Fsignal (emacs-27.0.50)
#10 0x0000000000582c7a xsignal2 (emacs-27.0.50)
#11 0x000000000056c507 wrong_type_argument (emacs-27.0.50)
#12 0x000000000059485d Fmaphash (emacs-27.0.50)
#13 0x000000000058283d eval_sub (emacs-27.0.50)
#14 0x000000000058351d FletX (emacs-27.0.50)
#15 0x00000000005825b9 eval_sub (emacs-27.0.50)
#16 0x0000000000585ba2 Feval (emacs-27.0.50)
#17 0x000000000058708f funcall_subr (emacs-27.0.50)
#18 0x00000000005865f1 Ffuncall (emacs-27.0.50)
#19 0x00000000005c5698 exec_byte_code (emacs-27.0.50)
#20 0x0000000000586555 Ffuncall (emacs-27.0.50)
#21 0x00000000005c5698 exec_byte_code (emacs-27.0.50)
#22 0x0000000000586555 Ffuncall (emacs-27.0.50)
#23 0x000000000057ff4e Ffuncall_interactively (emacs-27.0.50)
#24 0x00000000005865f1 Ffuncall (emacs-27.0.50)
#25 0x000000000058125a Fcall_interactively (emacs-27.0.50)
#26 0x00000000005870af funcall_subr (emacs-27.0.50)
#27 0x00000000005865f1 Ffuncall (emacs-27.0.50)
#28 0x00000000005c5698 exec_byte_code (emacs-27.0.50)
#29 0x0000000000586555 Ffuncall (emacs-27.0.50)
#30 0x0000000000586ddd call1 (emacs-27.0.50)
#31 0x000000000050a563 command_loop_1 (emacs-27.0.50)
#32 0x0000000000584c36 internal_condition_case (emacs-27.0.50)
#33 0x00000000005183e4 command_loop_2 (emacs-27.0.50)
#34 0x000000000058431a internal_catch (emacs-27.0.50)
#35 0x00000000005094e2 command_loop (emacs-27.0.50)
#36 0x0000000000509424 recursive_edit_1 (emacs-27.0.50)
#37 0x000000000050966f Frecursive_edit (emacs-27.0.50)
#38 0x0000000000508128 main (emacs-27.0.50)
#39 0x00007efe886ecf2a __libc_start_main (libc.so.6)
#40 0x000000000041b2fa _start (emacs-27.0.50)

Stack trace of thread 4976:
#0  0x00007efe8984082d pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0)
#1  0x00007efe7e6bae0c _ZNSt18condition_variable4waitERSt11unique_lockISt5mutexE (libstdc++.so.6)
#2  0x00007efe8cda9317 _ZN7bmalloc9Scavenger13threadRunLoopEv (libjavascriptcoregtk-4.0.so.18)
#3  0x00007efe7e6c114f execute_native_thread_routine (libstdc++.so.6)
#4  0x00007efe8983a50b start_thread (libpthread.so.0)
#5  0x00007efe887c116f __clone (libc.so.6)

Stack trace of thread 4977:
#0  0x00007efe887b6c6b __poll (libc.so.6)
#1  0x00007efe91ef4e99 g_main_context_iterate.isra.23 (libglib-2.0.so.0)
#2  0x00007efe91ef4fac g_main_context_iteration (libglib-2.0.so.0)
#3  0x00007efe91ef4ff1 glib_worker_main (libglib-2.0.so.0)
#4  0x00007efe91f1c486 g_thread_proxy (libglib-2.0.so.0)
#5  0x00007efe8983a50b start_thread (libpthread.so.0)
#6  0x00007efe887c116f __clone (libc.so.6)

Stack trace of thread 5780:
#0  0x00007efe887b6c6b __poll (libc.so.6)
#1  0x00007efe91ef4e99 g_main_context_iterate.isra.23 (libglib-2.0.so.0)
#2  0x00007efe91ef4fac g_main_context_iteration (libglib-2.0.so.0)
#3  0x00007efe712a642d dconf_gdbus_worker_thread (libdconfsettings.so)
#4  0x00007efe91f1c486 g_thread_proxy (libglib-2.0.so.0)
#5  0x00007efe8983a50b start_thread (libpthread.so.0)
#6  0x00007efe887c116f __clone (libc.so.6)

Stack trace of thread 5779:
#0  0x00007efe887b6c6b __poll (libc.so.6)
#1  0x00007efe91ef4e99 g_main_context_iterate.isra.23 (libglib-2.0.so.0)
#2  0x00007efe91ef5232 g_main_loop_run (libglib-2.0.so.0)
#3  0x00007efe924ddb56 gdbus_shared_thread_func (libgio-2.0.so.0)
#4  0x00007efe91f1c486 g_thread_proxy (libglib-2.0.so.0)
#5  0x00007efe8983a50b start_thread (libpthread.so.0)
#6  0x00007efe887c116f __clone (libc.so.6)

There is a wrong-type-argument with a (corrupted ?) data, emacs try to print it and crash

data.json.zip

ubolonton · 2018-04-14T10:47:23Z

I think I know the problem.

I assumed all raw emacs_value's obtained from emacs_env live (protected from GC) as long as the emacs_env itself. Turns out that's not the case, at least in 25.3.1. I'm working on the fix now.

This is potentially an Emacs's bug (or bad design) though. If you have access to Emacs 26, can you help me check if it also crashes there?

ubolonton · 2018-04-14T10:52:31Z

FWIW, that assumption was in line with this https://phst.github.io/emacs-modules#value-lifetime. That's not an official doc though.

ubolonton · 2018-04-14T10:58:47Z

This temp workaround works for me (I got a different way to reproduce the issue locally):

After this line https://github.com/ubolonton/emacs-module-rs/blob/master/src/lib.rs#L168, add

// Leak all values.
raw_call_no_exit!(env, make_global_ref, raw);

sebastiencs · 2018-04-14T11:16:59Z

If you have access to Emacs 26, can you help me check if it also crashes there?

I'm using emacs' master branch so it would crash there too.

FWIW, that assumption was in line with this https://phst.github.io/emacs-modules#value-lifetime. That's not an official doc though.

I think that it's the person who wrote emacs module system, so it's a legit source.
@phst, any thoughts on this ?

After this line https://github.com/ubolonton/emacs-module-rs/blob/master/src/lib.rs#L168, add

Yes that fix it for me too :)
Thanks for working on this so promptly, I wouldn't have found out by myself

phst · 2018-04-15T11:38:12Z

I assumed all raw emacs_value's obtained from emacs_env live (protected from GC) as long as the emacs_env itself. Turns out that's not the case, at least in 25.3.1.
This is potentially an Emacs's bug (or bad design) though.

Yes, that would definitely be a bug. Can you file a bug against Emacs with a minimal example? Thanks.

ubolonton · 2018-04-16T09:40:06Z

That's great to hear. I'll try to make a minimal example (without json parsing).

ubolonton · 2018-04-18T01:33:02Z

I'm getting closer to a minimal example https://github.com/ubolonton/emacs-module-rs/blob/crash/test-module/src/test_basics.rs#L130

It could be some weird interaction between Emacs, Rust compiler and this lib.

This will crash

(|| {
    v.push("0".into_lisp(env).unwrap());
})();
env.message("")?;
env.call("garbage-collect", &[])?;
env.call("print", &[v[0]])?;

This will not

v.push("0".into_lisp(env)?);
env.message("")?;
env.call("garbage-collect", &[]);
env.call("print", &[v[0]])?;

I'll try to reproduce this in C.

ubolonton · 2018-04-22T11:36:49Z

@phst I'm not sure if it's the same thing, but I got minimal examples in https://github.com/ubolonton/emacs-module-crash.

I've filed an Emacs bug report.

monnier · 2018-09-21T22:51:52Z

FWIW, that assumption was in line with this https://phst.github.io/emacs-modules#value-lifetime. That's not an official doc though.

Indeed, the life-time described there reflects an early design. It was changed before emacs-modules were integrated into an Emacs release. The system used instead is that emacs_values that live on the C stack are safe (thanks to the conservative stack scanning) but those that live elsewhere need to be manually registered/unregistered.

phst · 2019-01-17T22:55:10Z

Indeed, the life-time described there reflects an early design.

It reflects the current and intended design. The Emacs implementation is simply buggy.

This fixes potential memory leaks caused by #3, which was the fix for #2

ubolonton added a commit that referenced this issue Apr 18, 2018

Try to create a minimal reproduction of #2

e38f82d

ubolonton added a commit that referenced this issue Sep 9, 2018

Add failing tests for #2

2eab48e

ubolonton added a commit that referenced this issue Sep 9, 2018

Add failing tests for #2

6d29f92

ubolonton added a commit that referenced this issue Sep 12, 2018

Fix #2 by protecting new values returned from Emacs runtime

a73b584

ubolonton mentioned this issue Sep 15, 2018

Fix memory safety issues #3

Merged

ubolonton closed this as completed in #3 Sep 15, 2018

ubolonton mentioned this issue Jun 12, 2019

Put memory safety fix behind a feature flag #13

Closed

ubolonton added a commit that referenced this issue Mar 10, 2020

Fix Env's temporary global refs not being freed on error

a118de9

This fixes potential memory leaks caused by #3, which was the fix for #2

ubolonton added a commit that referenced this issue Mar 10, 2020

Fix Env's temporary global refs not being freed on non-local exit

066ddc5

This fixes potential memory leaks caused by #3, which was the fix for #2

ubolonton mentioned this issue Dec 23, 2020

Disable the workaround for Emacs GC bug if it's fixed (i.e. Emacs 27+) #35

Merged

ubolonton mentioned this issue Dec 20, 2024

Fix crashes with invalid utf-8. #59

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[crash] Invalid objects returned from my module function #2

[crash] Invalid objects returned from my module function #2

sebastiencs commented Apr 12, 2018

ubolonton commented Apr 13, 2018

sebastiencs commented Apr 13, 2018

ubolonton commented Apr 13, 2018

ubolonton commented Apr 14, 2018

ubolonton commented Apr 14, 2018

sebastiencs commented Apr 14, 2018

sebastiencs commented Apr 14, 2018 •

edited

Loading

ubolonton commented Apr 14, 2018

ubolonton commented Apr 14, 2018

ubolonton commented Apr 14, 2018

sebastiencs commented Apr 14, 2018 •

edited

Loading

phst commented Apr 15, 2018

ubolonton commented Apr 16, 2018

ubolonton commented Apr 18, 2018

ubolonton commented Apr 22, 2018 •

edited

Loading

monnier commented Sep 21, 2018

phst commented Jan 17, 2019

[crash] Invalid objects returned from my module function #2

[crash] Invalid objects returned from my module function #2

Comments

sebastiencs commented Apr 12, 2018

ubolonton commented Apr 13, 2018

sebastiencs commented Apr 13, 2018

ubolonton commented Apr 13, 2018

ubolonton commented Apr 14, 2018

ubolonton commented Apr 14, 2018

sebastiencs commented Apr 14, 2018

sebastiencs commented Apr 14, 2018 • edited Loading

ubolonton commented Apr 14, 2018

ubolonton commented Apr 14, 2018

ubolonton commented Apr 14, 2018

sebastiencs commented Apr 14, 2018 • edited Loading

phst commented Apr 15, 2018

ubolonton commented Apr 16, 2018

ubolonton commented Apr 18, 2018

ubolonton commented Apr 22, 2018 • edited Loading

monnier commented Sep 21, 2018

phst commented Jan 17, 2019

sebastiencs commented Apr 14, 2018 •

edited

Loading

sebastiencs commented Apr 14, 2018 •

edited

Loading

ubolonton commented Apr 22, 2018 •

edited

Loading