How to write tests?

[tobiashort]

In my opinion, tests in the form of unit tests are also essential in order to write a stable extension. Unfortunately, the API does not seem to be designed in such a way that it can be easily tested. For example, it is not possible to create an HttpRequest because HttpRequest.httpRequest() calls FACTORY.httpRequest(), but FACTORY is null because FACTORY has not been initialised. It is initialised when the extension is loaded, but how do I do this in the test context?

[SeanBurnsUK]

You can use mocks or your own implementation, everything is interfaces. Just assign the public static factory to your implementation.

import burp.api.montoya.http.message.requests.HttpRequest;
import burp.api.montoya.internal.MontoyaObjectFactory;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;

import static burp.api.montoya.internal.ObjectFactoryLocator.FACTORY;
import static org.assertj.core.api.Assertions.assertThat;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;

public class ExampleTest {
    @BeforeEach
    void setup() {
        FACTORY = mock(MontoyaObjectFactory.class);
        HttpRequest testRequest = mock(HttpRequest.class);
        when(FACTORY.httpRequest()).thenReturn(testRequest);
    }

    @Test
    void exampleTest() {
        assertThat(HttpRequest.httpRequest()).isNotNull();
    }
}

[tobiashort]

Using mocks or my own implementation is pretty much the same. I do not think this will make my extension more reliable, because it will not actually test against a concrete implementation, but against my own, where everything Burpsuite actually does has to be assumed.
If you try to mock things away, you will soon find that you have to mock away every little bit, from logging to utilities. This does not make much sense to me.

Wouldn't it be possible to provide a JUnit Jupiter test extension that initialises everything so that it is possible to write real tests against real implementations?

[Hannah-PortSwigger]

Hi

Yes, using mocks can get quite complicated quite quickly. You can find a good example of a tested extension using mocks here: https://github.com/PortSwigger/jwt-editor/tree/main/src/test/java

Overall, where possible, we'd suggest splitting your own code out from Burp's native code as much as possible to test just your own functionality.

Another alternative is creating a test extension you can load into Burp, which would mean you would not have to mock anything and can fully test functionality.

We do have a feature request to improve the testability of extensions, so we'll add your +1 to that ticket.

[tobiashort]

Another alternative is creating a test extension you can load into Burp, which would mean you would not have to mock anything and can fully test functionality.

This may work for manual testing, but I would like to have automated tests.

We do have a feature request to improve the testability of extensions, so we'll add your +1 to that ticket.

Is this an internal ticket or can you provide a link to it?

[Hannah-PortSwigger]

This is an internal ticket that we are currently monitoring. Adding +1s helps us to prioritize tickets like this in the future, as we can understand the level of interest in the functionality from the community.

We do not have any plans to make any changes to the testability of extensions in the short term.

[tobiashort]

For those interested. I have implemented a workaround. Our extension now displays a separate global tab when run in debug mode. With a single click, custom test cases can be run against concrete implementations. This is done by implementing a rudimentary test framework that uses reflection to look for new test classes and methods and displays the results in a JTextPane.

Reference:

• https://github.com/CompassSecurity/SAMLRaider/blob/master/doc/hacking.md#LiveTesting
• https://github.com/CompassSecurity/SAMLRaider/blob/master/src/main/java/gui/LiveTestingTab.java
• https://github.com/CompassSecurity/SAMLRaider/tree/master/src/main/java/livetesting