Skip to content

Glean testing

Jump to bottom
Cyndi Chin edited this page Jan 23, 2025 · 6 revisions

This page is broken into two parts, both related to testing Glean telemetry. The first part goes over testing telemetry calls manually and verifying that telemetry is sent properly through a dashboard. The second part goes over unit testing telemetry.

Verifying Glean Ping Data in the Debug Dashboard

In order to verify what data is being sent through Glean, you can use the glean debug dashboard here: https://glean-debug-view-dev-237806.firebaseapp.com/

Steps to test:

  • Decide what your tag will be for you to track your Glean pings. It can be anything, but it should not contain spaces or characters that would need to be url escaped and it should be shorter than 20 characters.
    • Example of valid tag: "Kayla-Glean-Test"
  • Add your tag to the end of this Glean debug deeplinking url: "firefox://glean?logPings=true&debugViewTag=" 
    • With the above tag, the final url would look like this: "firefox://glean?logPings=true&debugViewTag=Kayla-Glean-Test"
  • If you don't see the pings, try to use the following to confirm:
- Metrics Ping - > firefox://glean?logPings=true&debugViewTag=exampleTagName&sendPing=metrics
- Events Ping -> firefox://glean?logPings=true&debugViewTag=exampleTagName&sendPing=events
- Baseline Ping -> firefox://glean?logPings=true&debugViewTag=exampleTagName&sendPing=baseline
  • Open safari on either the simulator that has firefox installed with the build you are trying to test, or on a device that has the version of firefox you are trying to test installed
  • Paste the deeplink url into the safari navigation bar and it should prompt you to open firefox, accept this prompt
  • Use firefox to do things that would result in Glean data being sent
  • Navigate to the glean debug dashboard here: https://debug-ping-preview.firebaseapp.com/
  • You should see data appear with the date received, ping type, and payload. You can view the json here to see if your data is being sent in the way you expect.

You can read more about this in Glean debugging docs as well.

Unit Testing Glean Telemetry

We should unit test our telemetry using a mock instead of relying on the Glean dependency directly.

Background Context

Previously, we were unit testing our telemetry using Glean directly. However, we want to stop depending on Glean for our unit tests since relying on Glean caused flaky and unreliable tests that would crash. We created a GleanWrapper protocol that allows us to mock the dependency of Glean in unit tests.

  • The DefaultGleanWrapper is a concrete wrapper that abstracts Glean from our application and conforms to the GleanWrapper protocol. It is used in production.
  • The MockGleanWrapper is a struct that also conforms to the GleanWrapper protocol, but does not depend on Glean and is used for unit testing.

Setup Telemetry for Unit Testing

For our new telemetry, we are creating specific structs related to a feature or project. We want to use dependency injection so that we can test this struct using a mock. In production, we initialize our gleanWrapper using the DefaultGleanWrapper(), which abstracts Glean. Then we call the appropriate methods in gleanWrapper specified by the product requirements. See below as an example of what a telemetry struct should look like.

struct ExampleTelemetry {
    private let gleanWrapper: GleanWrapper

    init(gleanWrapper: GleanWrapper = DefaultGleanWrapper()) {
        self.gleanWrapper = gleanWrapper
    }

    func sendExampleTelemetry(value: Bool) {
        let exampleExtra = GleanMetrics.Example.ButtonTappedExtra(value: value)
        gleanWrapper.recordEvent(for: GleanMetrics.Example.buttonTapped, extras: exampleExtra)
    }
}

Using MockGleanWrapper in the Tests

When testing the methods in the telemetry struct, we want to inject the MockGleanWrapper in the initializer. Instead of using DefaultGleanWrapper, we want to rely on the mock, which we can test to verify what methods are being called and what values are being sent.

final class ExampleTelemetryTests: XCTestCase {
    var gleanWrapper: MockGleanWrapper!

    override func setUp() {
        super.setUp()
        gleanWrapper = MockGleanWrapper()
    }

    override func tearDown() {
        gleanWrapper = nil
        super.tearDown()
    }

    func testExample_withExtraIsTrue_sendsCorrectEvent() throws {
        let subject = createSubject()

        subject.sendExtraTelemetry(value: true)

        let savedEvent = try XCTUnwrap(
            gleanWrapper.savedEvent as? EventMetricType<GleanMetrics.Example.ButtonTappedExtra>
        )
        let savedExtras = try XCTUnwrap(
            gleanWrapper.savedExtras as? GleanMetrics.Example.ButtonTappedExtra
        )
        let expectedMetricType = type(of: GleanMetrics.Example.buttonTapped)
        let resultMetricType = type(of: savedEvent)
        let message = TelemetryDebugMessage(expectedMetric: expectedMetricType, resultMetric: resultMetricType)

        XCTAssert(resultMetricType == expectedMetricType, message.text)
        XCTAssertEqual(gleanWrapper.recordEventCalled, 1)
        XCTAssertEqual(savedExtras.value, true)
    }

    func testExample_withExtraIsFalse_sendsCorrectEvent() throws {
        let subject = createSubject()

        subject.sendExtraTelemetry(value: false)

        let savedEvent = try XCTUnwrap(
            gleanWrapper.savedEvent as? EventMetricType<GleanMetrics.Example.ButtonTappedExtra>
        )
        let savedExtras = try XCTUnwrap(
            gleanWrapper.savedExtras as? GleanMetrics.Example.ButtonTappedExtra
        )
        let expectedMetricType = type(of: GleanMetrics.Example.buttonTapped)
        let resultMetricType = type(of: savedEvent)
        let message = TelemetryDebugMessage(expectedMetric: expectedMetricType, resultMetric: resultMetricType)

        XCTAssert(resultMetricType == expectedMetricType, message.text)
        XCTAssertEqual(gleanWrapper.recordEventCalled, 1)
        XCTAssertEqual(savedExtras.value, false)
    }

    func createSubject() -> ExampleTelemetry {
       return ExampleTelemetry(gleanWrapper: gleanWrapper)
    }
}

Note: Currently, this example is testing that our mock is being called properly with the proper extra values, but we also want to create tests for when an action has been taken and causes the appropriate telemetry event to be sent.

Resources

Clone this wiki locally