Introduce context failure threshold support in the TestContext framework

This set of commits introduces ApplicationContext "failure threshold"
support in the Spring TestContext Framework (TCF).

Specifically, this new feature avoids repeated attempts to load a
failing ApplicationContext in the TCF, based on a failure threshold
which defaults to 1 but can be configured via a system property.

See individual commits for details.

Closes gh-14182

framework-docs/modules/ROOT/nav.adoc

@@ -130,6 +130,7 @@
 **** xref:testing/testcontext-framework/ctx-management/web.adoc[]
 **** xref:testing/testcontext-framework/ctx-management/web-mocks.adoc[]
 **** xref:testing/testcontext-framework/ctx-management/caching.adoc[]
+**** xref:testing/testcontext-framework/ctx-management/failure-threshold.adoc[]
 **** xref:testing/testcontext-framework/ctx-management/hierarchies.adoc[]
 *** xref:testing/testcontext-framework/fixture-di.adoc[]
 *** xref:testing/testcontext-framework/web-scoped-beans.adoc[]

framework-docs/modules/ROOT/pages/appendix.adoc

@@ -64,6 +64,11 @@ on a test class. See xref:testing/annotations/integration-junit-jupiter.adoc#int
 | The maximum size of the context cache in the _Spring TestContext Framework_. See
 xref:testing/testcontext-framework/ctx-management/caching.adoc[Context Caching].
 
+| `spring.test.context.failure.threshold`
+| The failure threshold for errors encountered while attempting to load an `ApplicationContext`
+in the _Spring TestContext Framework_. See
+xref:testing/testcontext-framework/ctx-management/failure-threshold.adoc[Context Failure Threshold].
+
 | `spring.test.enclosing.configuration`
 | The default _enclosing configuration inheritance mode_ to use if
 `@NestedTestConfiguration` is not present on a test class. See

framework-docs/modules/ROOT/pages/testing/testcontext-framework/ctx-management.adoc

@@ -119,5 +119,6 @@ advanced use cases.
 * xref:testing/testcontext-framework/ctx-management/dynamic-property-sources.adoc[Context Configuration with Dynamic Property Sources]
 * xref:testing/testcontext-framework/ctx-management/web.adoc[Loading a `WebApplicationContext`]
 * xref:testing/testcontext-framework/ctx-management/caching.adoc[Context Caching]
+* xref:testing/testcontext-framework/ctx-management/failure-threshold.adoc[Context Failure Threshold]
 * xref:testing/testcontext-framework/ctx-management/hierarchies.adoc[Context Hierarchies]
 

framework-docs/modules/ROOT/pages/testing/testcontext-framework/ctx-management/failure-threshold.adoc

@@ -0,0 +1,23 @@
+[[testcontext-ctx-management-failure-threshold]]
+= Context Failure Threshold
+
+As of Spring Framework 6.1, a context _failure threshold_ policy is in place which helps
+avoid repeated attempts to load a failing `ApplicationContext`. By default, the failure
+threshold is set to `1` which means that only one attempt will be made to load an
+`ApplicationContext` for a given context cache key (see
+xref:testing/testcontext-framework/ctx-management/caching.adoc[Context Caching]). Any
+subsequent attempt to load the `ApplicationContext` for the same context cache key will
+result in an immediate `IllegalStateException` with an error message which explains that
+the attempt was preemptively skipped. This behavior allows individual test classes and
+test suites to fail faster by avoiding repeated attempts to load an `ApplicationContext`
+that will never successfully load -- for example, due to a configuration error or a missing
+external resource that prevents the context from loading in the current environment.
+
+You can configure the context failure threshold from the command line or a build script
+by setting a JVM system property named `spring.test.context.failure.threshold` with a
+positive integer value. As an alternative, you can set the same property via the
+xref:appendix.adoc#appendix-spring-properties[`SpringProperties`] mechanism.
+
+NOTE: If you wish to effectively disable the context failure threshold, you can set the
+property to a very large value. For example, from the command line you could set the
+system property via `-Dspring.test.context.failure.threshold=1000000`.

spring-test/src/main/java/org/springframework/test/context/CacheAwareContextLoaderDelegate.java

@@ -36,6 +36,31 @@
  */
 public interface CacheAwareContextLoaderDelegate {
 
+	/**
+	 * The default failure threshold for errors encountered while attempting to
+	 * load an application context: {@value}.
+	 * @since 6.1
+	 * @see #CONTEXT_FAILURE_THRESHOLD_PROPERTY_NAME
+	 */
+	int DEFAULT_CONTEXT_FAILURE_THRESHOLD = 1;
+
+	/**
+	 * System property used to configure the failure threshold for errors
+	 * encountered while attempting to load an application context: {@value}.
+	 * <p>May alternatively be configured via the
+	 * {@link org.springframework.core.SpringProperties} mechanism.
+	 * <p>Implementations of {@code CacheAwareContextLoaderDelegate} are not
+	 * required to support this feature. Consult the documentation of the
+	 * corresponding implementation for details. Note, however, that the standard
+	 * {@code CacheAwareContextLoaderDelegate} implementation in Spring supports
+	 * this feature.
+	 * @since 6.1
+	 * @see #DEFAULT_CONTEXT_FAILURE_THRESHOLD
+	 * @see #loadContext(MergedContextConfiguration)
+	 */
+	String CONTEXT_FAILURE_THRESHOLD_PROPERTY_NAME = "spring.test.context.failure.threshold";
+
+
 	/**
 	 * Determine if the {@linkplain ApplicationContext application context} for
 	 * the supplied {@link MergedContextConfiguration} has been loaded (i.e.,
@@ -72,6 +97,13 @@ default boolean isContextLoaded(MergedContextConfiguration mergedConfig) {
 	 * mechanism, catch any exception thrown by the {@link ContextLoader}, and
 	 * delegate to each of the configured failure processors to process the context
 	 * load failure if the exception is an instance of {@link ContextLoadException}.
+	 * <p>As of Spring Framework 6.1, implementations of this method are encouraged
+	 * to support the <em>failure threshold</em> feature. Specifically, if repeated
+	 * attempts are made to load an application context and that application
+	 * context consistently fails to load &mdash; for example, due to a configuration
+	 * error that prevents the context from successfully loading &mdash; this
+	 * method should preemptively throw an {@link IllegalStateException} if the
+	 * configured failure threshold has been exceeded.
 	 * <p>The cache statistics should be logged by invoking
 	 * {@link org.springframework.test.context.cache.ContextCache#logStatistics()}.
 	 * @param mergedConfig the merged context configuration to use to load the
@@ -81,6 +113,7 @@ default boolean isContextLoaded(MergedContextConfiguration mergedConfig) {
 	 * the application context
 	 * @see #isContextLoaded
 	 * @see #closeContext
+	 * @see #CONTEXT_FAILURE_THRESHOLD_PROPERTY_NAME
 	 */
 	ApplicationContext loadContext(MergedContextConfiguration mergedConfig);
 

spring-test/src/main/java/org/springframework/test/context/cache/ContextCacheUtils.java

@@ -1,5 +1,5 @@
 /*
- * Copyright 2002-2016 the original author or authors.
+ * Copyright 2002-2023 the original author or authors.
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
@@ -17,10 +17,11 @@
 package org.springframework.test.context.cache;
 
 import org.springframework.core.SpringProperties;
+import org.springframework.test.context.CacheAwareContextLoaderDelegate;
 import org.springframework.util.StringUtils;
 
 /**
- * Collection of utilities for working with {@link ContextCache ContextCaches}.
+ * Collection of utilities for working with context caching.
  *
  * @author Sam Brannen
  * @since 4.3
@@ -30,25 +31,48 @@ public abstract class ContextCacheUtils {
 	/**
 	 * Retrieve the maximum size of the {@link ContextCache}.
 	 * <p>Uses {@link SpringProperties} to retrieve a system property or Spring
-	 * property named {@code spring.test.context.cache.maxSize}.
-	 * <p>Falls back to the value of the {@link ContextCache#DEFAULT_MAX_CONTEXT_CACHE_SIZE}
+	 * property named {@value ContextCache#MAX_CONTEXT_CACHE_SIZE_PROPERTY_NAME}.
+	 * <p>Defaults to {@value ContextCache#DEFAULT_MAX_CONTEXT_CACHE_SIZE}
 	 * if no such property has been set or if the property is not an integer.
 	 * @return the maximum size of the context cache
 	 * @see ContextCache#MAX_CONTEXT_CACHE_SIZE_PROPERTY_NAME
 	 */
 	public static int retrieveMaxCacheSize() {
+		String propertyName = ContextCache.MAX_CONTEXT_CACHE_SIZE_PROPERTY_NAME;
+		int defaultValue = ContextCache.DEFAULT_MAX_CONTEXT_CACHE_SIZE;
+		return retrieveProperty(propertyName, defaultValue);
+	}
+
+	/**
+	 * Retrieve the <em>failure threshold</em> for application context loading.
+	 * <p>Uses {@link SpringProperties} to retrieve a system property or Spring
+	 * property named {@value CacheAwareContextLoaderDelegate#CONTEXT_FAILURE_THRESHOLD_PROPERTY_NAME}.
+	 * <p>Defaults to {@value CacheAwareContextLoaderDelegate#DEFAULT_CONTEXT_FAILURE_THRESHOLD}
+	 * if no such property has been set or if the property is not an integer.
+	 * @return the failure threshold
+	 * @since 6.1
+	 * @see CacheAwareContextLoaderDelegate#CONTEXT_FAILURE_THRESHOLD_PROPERTY_NAME
+	 * @see CacheAwareContextLoaderDelegate#DEFAULT_CONTEXT_FAILURE_THRESHOLD
+	 */
+	public static int retrieveContextFailureThreshold() {
+		String propertyName = CacheAwareContextLoaderDelegate.CONTEXT_FAILURE_THRESHOLD_PROPERTY_NAME;
+		int defaultValue = CacheAwareContextLoaderDelegate.DEFAULT_CONTEXT_FAILURE_THRESHOLD;
+		return retrieveProperty(propertyName, defaultValue);
+	}
+
+	private static int retrieveProperty(String key, int defaultValue) {
 		try {
-			String maxSize = SpringProperties.getProperty(ContextCache.MAX_CONTEXT_CACHE_SIZE_PROPERTY_NAME);
-			if (StringUtils.hasText(maxSize)) {
-				return Integer.parseInt(maxSize.trim());
+			String value = SpringProperties.getProperty(key);
+			if (StringUtils.hasText(value)) {
+				return Integer.parseInt(value.trim());
 			}
 		}
 		catch (Exception ex) {
 			// ignore
 		}
 
 		// Fallback
-		return ContextCache.DEFAULT_MAX_CONTEXT_CACHE_SIZE;
+		return defaultValue;
 	}
 
 }

spring-test/src/main/java/org/springframework/test/context/cache/DefaultCacheAwareContextLoaderDelegate.java

@@ -16,7 +16,9 @@
 
 package org.springframework.test.context.cache;
 
+import java.util.HashMap;
 import java.util.List;
+import java.util.Map;
 
 import org.apache.commons.logging.Log;
 import org.apache.commons.logging.LogFactory;
@@ -41,7 +43,7 @@
 import org.springframework.util.Assert;
 
 /**
- * Default implementation of the {@link CacheAwareContextLoaderDelegate} interface.
+ * Default implementation of the {@link CacheAwareContextLoaderDelegate} strategy.
  *
  * <p>To use a static {@link DefaultContextCache}, invoke the
  * {@link #DefaultCacheAwareContextLoaderDelegate()} constructor; otherwise,
@@ -53,14 +55,18 @@
  * SpringFactoriesLoader} mechanism and delegates to them in
  * {@link #loadContext(MergedContextConfiguration)} to process context load failures.
  *
+ * <p>As of Spring Framework 6.1, this class supports the <em>failure threshold</em>
+ * feature described in {@link CacheAwareContextLoaderDelegate#loadContext},
+ * delegating to {@link ContextCacheUtils#retrieveContextFailureThreshold()} to
+ * obtain the threshold value to use.
+ *
  * @author Sam Brannen
  * @since 4.1
  */
 public class DefaultCacheAwareContextLoaderDelegate implements CacheAwareContextLoaderDelegate {
 
 	private static final Log logger = LogFactory.getLog(DefaultCacheAwareContextLoaderDelegate.class);
 
-
 	/**
 	 * Default static cache of Spring application contexts.
 	 */
@@ -74,27 +80,53 @@ public class DefaultCacheAwareContextLoaderDelegate implements CacheAwareContext
 
 	private final ContextCache contextCache;
 
+	/**
+	 * Map of context keys to context load failure counts.
+	 * @since 6.1
+	 */
+	private final Map<MergedContextConfiguration, Integer> failureCounts = new HashMap<>(32);
 
 	/**
-	 * Construct a new {@code DefaultCacheAwareContextLoaderDelegate} using
-	 * a static {@link DefaultContextCache}.
-	 * <p>This default cache is static so that each context can be cached
-	 * and reused for all subsequent tests that declare the same unique
-	 * context configuration within the same JVM process.
+	 * The configured failure threshold for errors encountered while attempting to
+	 * load an {@link ApplicationContext}.
+	 * @since 6.1
+	 */
+	private final int failureThreshold;
+
+
+	/**
+	 * Construct a new {@code DefaultCacheAwareContextLoaderDelegate} using a
+	 * static {@link DefaultContextCache}.
+	 * <p>The default cache is static so that each context can be cached and
+	 * reused for all subsequent tests that declare the same unique context
+	 * configuration within the same JVM process.
 	 * @see #DefaultCacheAwareContextLoaderDelegate(ContextCache)
 	 */
 	public DefaultCacheAwareContextLoaderDelegate() {
 		this(defaultContextCache);
 	}
 
 	/**
-	 * Construct a new {@code DefaultCacheAwareContextLoaderDelegate} using
-	 * the supplied {@link ContextCache}.
+	 * Construct a new {@code DefaultCacheAwareContextLoaderDelegate} using the
+	 * supplied {@link ContextCache} and the default or user-configured context
+	 * failure threshold.
 	 * @see #DefaultCacheAwareContextLoaderDelegate()
+	 * @see ContextCacheUtils#retrieveContextFailureThreshold()
 	 */
 	public DefaultCacheAwareContextLoaderDelegate(ContextCache contextCache) {
+		this(contextCache, ContextCacheUtils.retrieveContextFailureThreshold());
+	}
+
+	/**
+	 * Construct a new {@code DefaultCacheAwareContextLoaderDelegate} using the
+	 * supplied {@link ContextCache} and context failure threshold.
+	 * @since 6.1
+	 */
+	private DefaultCacheAwareContextLoaderDelegate(ContextCache contextCache, int failureThreshold) {
 		Assert.notNull(contextCache, "ContextCache must not be null");
+		Assert.isTrue(failureThreshold > 0, "'failureThreshold' must be positive");
 		this.contextCache = contextCache;
+		this.failureThreshold = failureThreshold;
 	}
 
 
@@ -112,6 +144,13 @@ public ApplicationContext loadContext(MergedContextConfiguration mergedConfig) {
 		synchronized (this.contextCache) {
 			ApplicationContext context = this.contextCache.get(mergedConfig);
 			if (context == null) {
+				Integer failureCount = this.failureCounts.getOrDefault(mergedConfig, 0);
+				if (failureCount >= this.failureThreshold) {
+					throw new IllegalStateException("""
+							ApplicationContext failure threshold (%d) exceeded: \
+							skipping repeated attempt to load context for %s"""
+								.formatted(this.failureThreshold, mergedConfig));
+				}
 				try {
 					if (mergedConfig instanceof AotMergedContextConfiguration aotMergedConfig) {
 						context = loadContextInAotMode(aotMergedConfig);
@@ -126,6 +165,7 @@ public ApplicationContext loadContext(MergedContextConfiguration mergedConfig) {
 					this.contextCache.put(mergedConfig, context);
 				}
 				catch (Exception ex) {
+					this.failureCounts.put(mergedConfig, ++failureCount);
 					Throwable cause = ex;
 					if (ex instanceof ContextLoadException cle) {
 						cause = cle.getCause();