Add diagnostic for stack allocations of 1 GB or more #119798

iSwapna · 2024-01-10T06:00:10Z

Add diagnostic for stack allocations of 1 GB or more

Zulip conversation [here](Issue 83060 - Regression with large stack arrays (2-4GB))

Do I generate an ICE or issue a warning?

cc #83060

rustbot · 2024-01-10T06:00:19Z

r? @wesleywiser

(rustbot has picked a reviewer for you, use r? to override)

compiler/rustc_codegen_ssa/src/mir/mod.rs

iSwapna · 2024-01-13T06:11:23Z

I have addressed all the comments, but not sure how to add the stdout file for 32 bit :
Testing stage2 compiletest suite=ui mode=ui (x86_64-unknown-linux-gnu -> i686-unknown-linux-gnu)
fails in CI:

failures:

  ---- [ui] tests/ui/codegen/issue-83060-large-stack-size.rs stdout ----
  diff of stderr:

  - warning: Dangerous stack allocation of size: 1 GiB exceeds most architecture limits
  -
  - warning: 1 warning emitted
  -
  -

iSwapna · 2024-01-13T06:37:37Z

@rustbot review

iSwapna · 2024-01-13T06:52:06Z

I have addressed all the comments, but not sure how to add the stdout file for 32 bit : Testing stage2 compiletest suite=ui mode=ui (x86_64-unknown-linux-gnu -> i686-unknown-linux-gnu) fails in CI:
failures:

  ---- [ui] tests/ui/codegen/issue-83060-large-stack-size.rs stdout ----
  diff of stderr:

  - warning: Dangerous stack allocation of size: 1 GiB exceeds most architecture limits
  -
  - warning: 1 warning emitted
  -
  -

compiler/rustc_codegen_ssa/messages.ftl

compiler/rustc_codegen_ssa/src/mir/mod.rs

Dylan-DPC · 2024-02-21T07:25:01Z

@iSwapna any updates on this? thanks

iSwapna · 2024-02-22T03:54:02Z

@iSwapna any updates on this? thanks

Sorry, I am taking a class @Stanford CS103, taking every bit of my time outside of work. That's towards wanting a good grounding on CS/compilers as well, to do a better job here!

There was a CI failure (fluent related) which I have not got an answer on how to address, the Zulip discussion is here: Issue 83060 - Regression with large stack arrays (2-4GB)

If I could get an answer on how to address the CI issue, I can take care of this in a couple of weeks (after impending midterm)

Dylan-DPC · 2024-02-22T04:12:23Z

Sure, Thanks for the update. Just wanted to know if you are still working on it and have any updates. You should get a reply on zulip else you can bump it i guess

iSwapna · 2024-02-22T04:14:02Z

Sure, Thanks for the update. Just wanted to know if you are still working on it and have any updates. You should get a reply on zulip else you can bump it i guess

will do! Thank you for checking!

tests/ui/codegen/issue-83060-large-stack-size.rs

tests/ui/codegen/large-stack-size-issue-83060.rs

rustbot · 2024-10-26T09:02:06Z

Some changes occurred in tests/ui/sanitizer

cc @rust-lang/project-exploit-mitigations, @rcvalle

use ilog2 Update compiler/rustc_codegen_ssa/messages.ftl Co-authored-by: Michael Goulet <[email protected]> Run test only on 64 bit

rust-log-analyzer · 2024-11-02T19:11:26Z

The job x86_64-gnu-llvm-18 failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

------
 > importing cache manifest from ghcr.io/rust-lang/rust-ci-cache:c32c805632780b5c1de330e3f44561b336c2efe163bc0990acb392390157a8e1d9f855d75914a239aa40c49d77f4a837247d05d2f8d46f554b98e1f46712a3e3:
------
##[endgroup]
Setting extra environment values for docker:  --env ENABLE_GCC_CODEGEN=1 --env GCC_EXEC_PREFIX=/usr/lib/gcc/
[CI_JOB_NAME=x86_64-gnu-llvm-18]
debug: `DISABLE_CI_RUSTC_IF_INCOMPATIBLE` configured.
---
sccache: Starting the server...
##[group]Configure the build
configure: processing command line
configure: 
configure: build.configure-args := ['--build=x86_64-unknown-linux-gnu', '--llvm-root=/usr/lib/llvm-18', '--enable-llvm-link-shared', '--set', 'rust.randomize-layout=true', '--set', 'rust.thin-lto-import-instr-limit=10', '--enable-verbose-configure', '--enable-sccache', '--disable-manage-submodules', '--enable-locked-deps', '--enable-cargo-native-static', '--set', 'rust.codegen-units-std=1', '--set', 'dist.compression-profile=balanced', '--dist-compression-formats=xz', '--set', 'rust.lld=false', '--disable-dist-src', '--release-channel=nightly', '--enable-debug-assertions', '--enable-overflow-checks', '--enable-llvm-assertions', '--set', 'rust.verify-llvm-ir', '--set', 'rust.codegen-backends=llvm,cranelift,gcc', '--set', 'llvm.static-libstdcpp', '--enable-new-symbol-mangling']
configure: target.x86_64-unknown-linux-gnu.llvm-config := /usr/lib/llvm-18/bin/llvm-config
configure: llvm.link-shared     := True
configure: rust.randomize-layout := True
configure: rust.thin-lto-import-instr-limit := 10
---
  Downloaded boml v0.3.1
   Compiling boml v0.3.1
   Compiling y v0.1.0 (/checkout/compiler/rustc_codegen_gcc/build_system)
    Finished `release` profile [optimized] target(s) in 3.79s
     Running `/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-codegen/x86_64-unknown-linux-gnu/release/y test --use-system-gcc --use-backend gcc --out-dir /checkout/obj/build/x86_64-unknown-linux-gnu/stage1-tools/cg_gcc --release --mini-tests --std-tests`
Using system GCC
[BUILD] example
[AOT] mini_core_hello_world
/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-tools/cg_gcc/mini_core_hello_world
abc
---

failures:

---- compiler/rustc_lint_defs/src/builtin.rs - builtin::DANGEROUS_STACK_ALLOCATION (line 718) stdout ----
warning: allocation of size: 1 GiB  exceeds most system architecture limits
  |
  |
4 |     let mut x: [u8; CAP>>1] = [0; CAP>>1];
  |
  = note: `#[warn(dangerous_stack_allocation)]` on by default

warning: 1 warning emitted
warning: 1 warning emitted

Test executable failed (signal: 6 (SIGABRT) (core dumped)).
stderr:

thread '<unknown>' has overflowed its stack
fatal runtime error: stack overflow
---
    compiler/rustc_lint_defs/src/builtin.rs - builtin::DANGEROUS_STACK_ALLOCATION (line 718)

test result: FAILED. 116 passed; 1 failed; 24 ignored; 0 measured; 0 filtered out; finished in 1.14s

error: doctest failed, to rerun pass `-p rustc_lint_defs --doc`
  local time: Sat Nov  2 19:11:17 UTC 2024
  network time: Sat, 02 Nov 2024 19:11:17 GMT
##[error]Process completed with exit code 1.
Post job cleanup.

bors · 2024-11-04T04:14:27Z

☔ The latest upstream changes (presumably #132581) made this pull request unmergeable. Please resolve the merge conflicts.

iSwapna · 2024-11-22T19:00:30Z

@estebank I will get back to this after exams - late Dec. Do let me know your comments.
Best,
Swapna

estebank · 2024-11-22T21:51:27Z

compiler/rustc_codegen_ssa/src/mir/mod.rs

+                    decl.source_info.span,
+                    |lint| {
+                        lint.primary_message(format!(
+                            "allocation of size: {:.2} {}  exceeds most system architecture limits",


Suggested change

"allocation of size: {:.2} {} exceeds most system architecture limits",

"allocation of {:.2} {} exceeds most system architecture limits",

estebank · 2024-11-22T21:52:30Z

compiler/rustc_codegen_ssa/src/mir/mod.rs

+/// Formats a number of bytes into a human readable SI-prefixed size.
+/// Returns a tuple of `(quantity, units)`.
+pub fn human_readable_bytes(bytes: u64) -> (u64, &'static str) {
+    static UNITS: [&str; 7] = ["B", "KiB", "MiB", "GiB", "TiB", "PiB", "EiB"];
+    let i = ((bytes.checked_ilog2().unwrap_or(0) / 10) as usize).min(UNITS.len() - 1);
+    (bytes >> (10 * i), UNITS[i])
+}


Could have sworn that we did this already, but alas it is in cargo 😅

estebank · 2024-11-22T21:52:56Z

compiler/rustc_lint_defs/src/builtin.rs

+    /// Large arras may cause stack overflow due to the limited size of the
+    /// stack on most platforms.


Suggested change

/// Large arras may cause stack overflow due to the limited size of the

/// stack on most platforms.

/// Large arrays may cause stack overflow due to the limited size of the

/// stack on most platforms.

We should probably extend this to be more explicit about which platforms this is an issue in (no need to be exhaustive).

estebank · 2024-11-22T21:54:14Z

compiler/rustc_lint_defs/src/builtin.rs

+    /// stack on most platforms.
+    pub DANGEROUS_STACK_ALLOCATION,
+    Warn,
+    "Detects dangerous stack allocations at the limit of most architectures"


Suggested change

"Detects dangerous stack allocations at the limit of most architectures"

"detects dangerously large stack allocations at the limit of most architectures"

estebank · 2024-11-22T21:54:35Z

compiler/rustc_codegen_ssa/src/mir/mod.rs

@@ -29,6 +31,8 @@ use self::debuginfo::{FunctionDebugContext, PerLocalVarDebugInfo};
 use self::operand::{OperandRef, OperandValue};
 use self::place::PlaceRef;

+const MIN_DANGEROUS_SIZE: u64 = 1024 * 1024 * 1024 * 1; // 1 GB


Suggested change

const MIN_DANGEROUS_SIZE: u64 = 1024 * 1024 * 1024 * 1; // 1 GB

const MIN_DANGEROUS_ALLOC_SIZE: u64 = 1024 * 1024 * 1024 * 1; // 1 GB

Not to change, but to discuss: would making this lower than 1GB be worth it, so that people that we can guide people that are not hitting the limits, but are on their way to do so get some early warning?

estebank · 2024-11-22T21:54:46Z

compiler/rustc_codegen_ssa/src/mir/mod.rs

@@ -234,6 +238,21 @@ pub fn codegen_mir<'a, 'tcx, Bx: BuilderMethods<'a, 'tcx>>(
            let layout = start_bx.layout_of(fx.monomorphize(decl.ty));
            assert!(!layout.ty.has_erasable_regions());

+            if layout.size.bytes() >= MIN_DANGEROUS_SIZE {


Suggested change

if layout.size.bytes() >= MIN_DANGEROUS_SIZE {

if layout.size.bytes() >= MIN_DANGEROUS_ALLOC_SIZE {

estebank · 2024-11-22T21:55:45Z

compiler/rustc_codegen_ssa/src/errors.rs

+#[derive(Diagnostic)]
+#[diag(codegen_ssa_dangerous_stack_allocation)]
+pub struct DangerousStackAllocation {
+    #[primary_span]
+    pub span: Span,
+    pub output: String,
+}


This and the ftl change are no longer used, right?

estebank · 2024-11-22T21:58:49Z

compiler/rustc_codegen_ssa/src/mir/mod.rs

+                    lint::builtin::DANGEROUS_STACK_ALLOCATION,
+                    CRATE_HIR_ID,
+                    decl.source_info.span,
+                    |lint| {


We could add more context, like in the lint description, about some common platform limits.

estebank · 2024-11-22T22:00:49Z

compiler/rustc_lint_defs/src/builtin.rs

+    /// warning: allocation of size: 1 GiB  exceeds most system architecture limits
+    /// --> $DIR/large-stack-size-issue-83060.rs:7:9
+    /// |
+    /// LL |     let mut x: [u8; CAP>>1] = [0; CAP>>1];
+    ///  |         ^^^^^
+    ///  |
+    ///  = note: `#[warn(dangerous_stack_allocation)]` on by default


This will likely fail because of the textual formatting not being correctly left aligned.

estebank · 2024-11-22T22:03:22Z

compiler/rustc_lint_defs/src/builtin.rs

+    /// warning: allocation of size: 1 GiB  exceeds most system architecture limits
+    /// --> $DIR/large-stack-size-issue-83060.rs:7:9
+    /// |
+    /// LL |     let mut x: [u8; CAP>>1] = [0; CAP>>1];
+    ///  |         ^^^^^
+    ///  |
+    ///  = note: `#[warn(dangerous_stack_allocation)]` on by default


Suggested change

/// warning: allocation of size: 1 GiB exceeds most system architecture limits

/// --> $DIR/large-stack-size-issue-83060.rs:7:9

/// |

/// LL | let mut x: [u8; CAP>>1] = [0; CAP>>1];

/// | ^^^^^

/// |

/// = note: `#[warn(dangerous_stack_allocation)]` on by default

/// warning: allocation of size: 1 GiB exceeds most system architecture limits

/// --> $DIR/large-stack-size-issue-83060.rs:7:9

/// |

/// LL | let mut x: [u8; CAP>>1] = [0; CAP>>1];

/// | ^^^^^

/// |

/// = note: `#[warn(dangerous_stack_allocation)]` on by default

rustbot assigned wesleywiser Jan 10, 2024

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Jan 10, 2024

This comment has been minimized.

Sign in to view

kadiwa4 reviewed Jan 10, 2024

View reviewed changes

compiler/rustc_codegen_ssa/src/mir/mod.rs Outdated Show resolved Hide resolved

This comment has been minimized.

Sign in to view

iSwapna force-pushed the issue-83060-fix branch from fd9f9a6 to 8ee58b2 Compare January 11, 2024 06:32

This comment has been minimized.

Sign in to view

iSwapna force-pushed the issue-83060-fix branch 2 times, most recently from 9fb4f4f to 91dcd42 Compare January 13, 2024 04:20

This comment has been minimized.

Sign in to view

iSwapna requested a review from kadiwa4 January 13, 2024 06:11

This comment has been minimized.

Sign in to view

wesleywiser reviewed Jan 16, 2024

View reviewed changes

compiler/rustc_codegen_ssa/messages.ftl Outdated Show resolved Hide resolved

compiler/rustc_codegen_ssa/src/mir/mod.rs Outdated Show resolved Hide resolved

compiler/rustc_codegen_ssa/src/mir/mod.rs Outdated Show resolved Hide resolved

wesleywiser added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 22, 2024

wesleywiser reviewed Mar 5, 2024

View reviewed changes

tests/ui/codegen/issue-83060-large-stack-size.rs Outdated Show resolved Hide resolved

iSwapna force-pushed the issue-83060-fix branch from 3305a06 to 96c95df Compare March 30, 2024 19:23

This comment has been minimized.

Sign in to view

workingjubilee reviewed Oct 4, 2024

View reviewed changes

tests/ui/codegen/large-stack-size-issue-83060.rs Outdated Show resolved Hide resolved

iSwapna force-pushed the issue-83060-fix branch from d9232a0 to 72aa684 Compare October 4, 2024 05:40

cjgillot changed the title ~~Issue 83060 fix~~ Add diagnostic for stack allocations of 1 GB or more Oct 4, 2024

iSwapna force-pushed the issue-83060-fix branch from 72aa684 to bece062 Compare October 23, 2024 23:50

rustbot added the PG-exploit-mitigations Project group: Exploit mitigations label Oct 26, 2024

iSwapna force-pushed the issue-83060-fix branch from 991265a to 96da2f1 Compare October 26, 2024 09:05

iSwapna requested a review from estebank October 26, 2024 09:06

This comment has been minimized.

Sign in to view

iSwapna force-pushed the issue-83060-fix branch from f9ad136 to d7480c4 Compare October 31, 2024 23:56

This comment has been minimized.

Sign in to view

iSwapna force-pushed the issue-83060-fix branch from d7480c4 to fa8402a Compare November 1, 2024 17:38

This comment has been minimized.

Sign in to view

iSwapna added 7 commits November 2, 2024 08:48

Add span_bug for locals larger than 1GB

48975e3

use ilog2 Update compiler/rustc_codegen_ssa/messages.ftl Co-authored-by: Michael Goulet <[email protected]> Run test only on 64 bit

Lint instead of warn

1dbc746

tidy

3cb7556

mingw fix

9953a42

tidy!

5d77f9b

fix doctest

91c22d1

text for doc

a0338cf

iSwapna force-pushed the issue-83060-fix branch from 265cfea to a0338cf Compare November 2, 2024 15:52

This comment has been minimized.

Sign in to view

Another attempt

deb7c8c

iSwapna requested review from cjgillot and workingjubilee November 6, 2024 21:13

estebank reviewed Nov 22, 2024

View reviewed changes

	"allocation of size: {:.2} {} exceeds most system architecture limits",
	"allocation of {:.2} {} exceeds most system architecture limits",

		/// Large arras may cause stack overflow due to the limited size of the
		/// stack on most platforms.

	"Detects dangerous stack allocations at the limit of most architectures"
	"detects dangerously large stack allocations at the limit of most architectures"

	const MIN_DANGEROUS_SIZE: u64 = 1024 * 1024 * 1024 * 1; // 1 GB
	const MIN_DANGEROUS_ALLOC_SIZE: u64 = 1024 * 1024 * 1024 * 1; // 1 GB

	if layout.size.bytes() >= MIN_DANGEROUS_SIZE {
	if layout.size.bytes() >= MIN_DANGEROUS_ALLOC_SIZE {

Add diagnostic for stack allocations of 1 GB or more #119798

Are you sure you want to change the base?

Add diagnostic for stack allocations of 1 GB or more #119798

Conversation

iSwapna commented Jan 10, 2024 • edited by cjgillot Loading

rustbot commented Jan 10, 2024

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

iSwapna commented Jan 13, 2024 • edited Loading

iSwapna commented Jan 13, 2024

iSwapna commented Jan 13, 2024 • edited Loading

This comment has been minimized.

Dylan-DPC commented Feb 21, 2024

iSwapna commented Feb 22, 2024

Dylan-DPC commented Feb 22, 2024

iSwapna commented Feb 22, 2024

This comment has been minimized.

rustbot commented Oct 26, 2024

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

rust-log-analyzer commented Nov 2, 2024

bors commented Nov 4, 2024

iSwapna commented Nov 22, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

iSwapna commented Jan 10, 2024 •

edited by cjgillot

Loading

iSwapna commented Jan 13, 2024 •

edited

Loading

iSwapna commented Jan 13, 2024 •

edited

Loading