Skip to content

Commit

Permalink
Add thread group controller based on RPS to dynamically create threads
Browse files Browse the repository at this point in the history
Remove deprecated methods to cleanup code and add exception to DashboardVisualizer.showInGui to get a more descriptive error when used.
  • Loading branch information
rabelenda-abstracta committed Oct 22, 2021
1 parent 5c0b8b5 commit e378c53
Show file tree
Hide file tree
Showing 13 changed files with 586 additions and 97 deletions.
29 changes: 26 additions & 3 deletions docs/classes.puml
Original file line number Diff line number Diff line change
Expand Up @@ -57,16 +57,16 @@ package core {

EmbeddedJmeterEngine ..> JMeterEnvironment

class DslThreadGroup extends TestElementContainer implements TestPlanChild
class DslThreadGroup extends TestElementContainer implements TestPlanChild {
List<Stage> stages
}

class Stage {
int threads
int iterations
Duration duration
}

DslThreadGroup -> "*" Stage

interface ThreadGroupChild extends DslTestElement

abstract class DslSampler extends TestElementContainer implements ThreadGroupChild
Expand Down Expand Up @@ -94,6 +94,29 @@ package core {
String Label
}

package threadgroups {

class RpsThreadGroup extends TestElementContainer implements TestPlanChild {
List<TimerSchedule> schedules
EventType counting
int initThreads
int maxThreads
double spareThreads
}

class TimerSchedule {
double fromRps
double toRps
Duration duration
}

enum EventType {
REQUESTS
ITERATIONS
}

}

package configs {

class DslCsvDataSet extends BaseTestElement implements MultiLevelTestElement {
Expand Down
57 changes: 49 additions & 8 deletions docs/guide/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -202,7 +202,9 @@ threadGroup(10, 20) // 10 threads for 20 iterations each
threadGroup(10, Duration.ofSeconds(20)) // 10 threads for 20 seconds each
```

But, these options are not good when working with many threads or when trying to configure some complex test scenarios (like when doing incremental or peak tests).
But these options are not good when working with many threads or when trying to configure some complex test scenarios (like when doing incremental or peak tests).

### Thread ramps and holds

When working with many threads, it is advisable to configure a ramp up period, to avoid starting all threads at once affecting performance metrics and generation.

Expand All @@ -217,21 +219,24 @@ Additionally, you can use and combine these same methods to configure more compl

```java
threadGroup()
.rampToAndHold(10, Duration.ofSeconds(5), Duration.ofSeconds(20))
.rampToAndHold(100, Duration.ofSeconds(10), Duration.ofSeconds(30))
.rampTo(200, Duration.ofSeconds(10))
.rampToAndHold(100, Duration.ofSeconds(10), Duration.ofSeconds(30))
.rampTo(0, Duration.ofSeconds(5))
.rampToAndHold(10, Duration.ofSeconds(5), Duration.ofSeconds(20))
.rampToAndHold(100, Duration.ofSeconds(10), Duration.ofSeconds(30))
.rampTo(200, Duration.ofSeconds(10))
.rampToAndHold(100, Duration.ofSeconds(10), Duration.ofSeconds(30))
.rampTo(0, Duration.ofSeconds(5))
.children(
httpSampler("http://my.service")
)
```

Which would translate in the following threads' timeline:

![ThreadGroup Chart](./images/complex-thread-group-chart.png)
![Thread Group Timeline](./images/ultimate-thread-group-timeline.png)

Check [DslThreadGroup](../../jmeter-java-dsl/src/main/java/us/abstracta/jmeter/javadsl/core/DslThreadGroup.java) for more details.

::: tip
To visualize threads timeline, for complex thread group configurations like previous one, you can get a chart like previous one by using provided `DslThreadGroup.showThreadsTimeline()` method.
To visualize threads timeline, for complex thread group configurations like previous one, you can get a chart like previous one by using provided `DslThreadGroup.showTimeline()` method.
:::

::: tip
Expand All @@ -246,6 +251,42 @@ For example, for above test plan you would get a window like the following one:
When using multiple thread groups in a test plan, consider setting a name on them to properly identify associated requests in statistics & jtl results.
:::

### Throughput based thread group

Sometimes you want to focus just on the number of requests per second to generate and don't want to be concerned about how many concurrent threads/users, and pauses between requests, are needed. For these scenarios you can use `rpsThreadGroup` like in following example:

```java
rpsThreadGroup()
.maxThreads(500)
.rampTo(20, Duration.ofSeconds(10))
.rampTo(10, Duration.ofSeconds(10))
.rampToAndHold(1000, Duration.ofSeconds(5), Duration.ofSeconds(10))
.children(
httpSampler("http://my.service")
)
```

This will internally use JMeter [Concurrency Thread Group](https://jmeter-plugins.org/wiki/ConcurrencyThreadGroup/) element in combination with [Throughput
Shaping Time](https://jmeter-plugins.org/wiki/ThroughputShapingTimer/).

::: tip
`rpsThreadGroup` will dynamically create and remove threads and add delays between requests to match the traffic to the expected RPS. You can also specify to control iterations per second (number of times the flow in the thread group runs per second) instead of threads by using `.counting(RpsThreadGroup.EventType.ITERATIONS)`.
:::

::: warning
When no `maxThreads` are specified, `rpsThreadGroup` will use as many threads as needed. In such scenarios, you might end with unexpected number of threads with associated CPU and Memory requirements, which may affect the performance test metrics. **You should always set maximum threads to use** to avoid such scenarios.

You can use following formula to calculate a value for `maxThreads`: `T*R`, being `T` the maximum RPS that you want to achieve and `R` the maximum expected response time (or iteration time if you use `.counting(RpsThreadGroup.EventType.ITERATIONS)`) in seconds.
:::

::: tip
As with default thread group, with `rpsThreadGroup` you can use `showTimeline` to get a chart of configured RPS profile for easy visualization. An example chart:

![RPS Thread Group Timeline](./images/rps-thread-group-timeline.png)
:::

Check [RpsThreadGroup](../../jmeter-java-dsl/src/main/java/us/abstracta/jmeter/javadsl/core/threadgroups/RpsThreadGroup.java) for more details.

## Test plan debugging

A usual requirement while building a test plan is to be able to debug for potential issues in configuration or behavior of service under test. With jmeter-java-dsl you have several options for this purpose.
Expand Down
Binary file added docs/guide/images/rps-thread-group-timeline.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
Expand Up @@ -183,4 +183,9 @@ protected void showTestElementGui(Supplier<Component> guiBuilder, Runnable close
showFrameWith(guiBuilder.get(), "Dashboard", 1080, 600, closeListener);
}

@Override
public void showInGui() {
throw new UnsupportedOperationException("Dashboard has no built-in JMeter GUI");
}

}
5 changes: 5 additions & 0 deletions jmeter-java-dsl/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,11 @@
<artifactId>jmeter-plugins-casutg</artifactId>
<version>2.10</version>
</dependency>
<dependency>
<groupId>kg.apc</groupId>
<artifactId>jmeter-plugins-tst</artifactId>
<version>2.5</version>
</dependency>
<dependency>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-http</artifactId>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
import us.abstracta.jmeter.javadsl.core.preprocessors.DslJsr223PreProcessor;
import us.abstracta.jmeter.javadsl.core.preprocessors.DslJsr223PreProcessor.PreProcessorScript;
import us.abstracta.jmeter.javadsl.core.preprocessors.DslJsr223PreProcessor.PreProcessorVars;
import us.abstracta.jmeter.javadsl.core.threadgroups.RpsThreadGroup;
import us.abstracta.jmeter.javadsl.core.timers.DslUniformRandomTimer;
import us.abstracta.jmeter.javadsl.http.DslCacheManager;
import us.abstracta.jmeter.javadsl.http.DslCookieManager;
Expand Down Expand Up @@ -133,10 +134,10 @@ public static DslThreadGroup threadGroup(String name, int threads, Duration dura
* Eg:
* <pre>{@code
* threadGroup()
* .rampTo(10, Duration.seconds(10))
* .rampDown(5, Duration.seconds(10))
* .rampToAndHold(20, Duration.seconds(5), Duration.seconds(10))
* .rampTo(0, Duration.seconds(5))
* .rampTo(10, Duration.ofSeconds(10))
* .rampTo(5, Duration.ofSeconds(10))
* .rampToAndHold(20, Duration.ofSeconds(5), Duration.ofSeconds(10))
* .rampTo(0, Duration.ofSeconds(5))
* .children(...)
* }</pre>
*
Expand All @@ -159,6 +160,45 @@ public static DslThreadGroup threadGroup(String name) {
return new DslThreadGroup(name);
}

/**
* Builds a thread group that dynamically adapts thread count and pauses to match a given RPS.
*
* Internally this element uses
* <a href="https://jmeter-plugins.org/wiki/ConcurrencyThreadGroup/">Concurrency Thread Group</a>
* in combination with <a href="https://jmeter-plugins.org/wiki/ThroughputShapingTimer/">Throughput
* Shaping Timer</a>.
*
* Eg:
* <pre>{@code
* rpsThreadGroup()
* .maxThreads(500)
* .rampTo(20, Duration.ofSeconds(10))
* .rampTo(10, Duration.ofSeconds(10))
* .rampToAndHold(1000, Duration.ofSeconds(5), Duration.ofSeconds(10))
* .rampTo(0, Duration.ofSeconds(5))
* .children(...)
* }</pre>
*
* @return the thread group instance.
* @see RpsThreadGroup
* @since 0.26
*/
public static RpsThreadGroup rpsThreadGroup() {
return new RpsThreadGroup(null);
}

/**
* Same as {@link #rpsThreadGroup()} but allowing to set a name on the thread group.
* <p>
* Setting a proper name allows to properly identify the requests generated in each thread group.
*
* @see #rpsThreadGroup()
* @since 0.26
*/
public static RpsThreadGroup rpsThreadGroup(String name) {
return new RpsThreadGroup(name);
}

/**
* Builds a new transaction controller with the given name.
*
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
import java.beans.PropertyDescriptor;
import java.io.IOException;
import java.lang.reflect.InvocationTargetException;
import java.time.Duration;
import java.util.function.Supplier;
import javax.swing.JFrame;
import javax.swing.WindowConstants;
Expand Down Expand Up @@ -53,10 +54,14 @@ public HashTree buildTreeUnder(HashTree parent, BuildTreeContext context) {

protected TestElement buildConfiguredTestElement() {
TestElement ret = buildTestElement();
return configureTestElement(ret, name, guiClass, getBeanInfo());
}

protected static TestElement configureTestElement(TestElement ret, String name,
Class<? extends JMeterGUIComponent> guiClass, BeanInfoSupport beanInfo) {
ret.setName(name);
ret.setProperty(TestElement.GUI_CLASS, guiClass.getName());
ret.setProperty(TestElement.TEST_CLASS, ret.getClass().getName());
BeanInfoSupport beanInfo = getBeanInfo();
if (beanInfo != null) {
loadBeanProperties(ret, beanInfo);
}
Expand All @@ -69,7 +74,7 @@ protected BeanInfoSupport getBeanInfo() {
return null;
}

private void loadBeanProperties(TestElement bean, BeanInfoSupport beanInfo) {
private static void loadBeanProperties(TestElement bean, BeanInfoSupport beanInfo) {
for (PropertyDescriptor prop : beanInfo.getPropertyDescriptors()) {
if (TestBeanHelper.isDescriptorIgnored(prop)) {
continue;
Expand Down Expand Up @@ -135,4 +140,8 @@ public void windowClosed(WindowEvent e) {
frame.setVisible(true);
}

protected static long durationToSeconds(Duration duration) {
return Math.round(Math.ceil((double) duration.toMillis() / 1000));
}

}
Loading

0 comments on commit e378c53

Please sign in to comment.