Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When span time is small (e.g., less than 1s), discard the span and do not display the span in the trace #1552

Closed
yzpangu opened this issue May 20, 2019 · 16 comments

Comments

@yzpangu
Copy link

yzpangu commented May 20, 2019

When span time is small (e.g., less than 1s), discard the span and do not display the span in the trace

@yzpangu
Copy link
Author

yzpangu commented May 20, 2019

How can I do that

@yurishkuro
Copy link
Member

Jaeger does not support that. If you like to propose an enhancement, please describe business requirement / user story (this was asked in the issue template, btw).

@yzpangu
Copy link
Author

yzpangu commented May 21, 2019

When jaeger was used in our project, it produced a lot of spans, and we wanted to reduce the number of spans. So we want to remove the low time consuming span from the traces, such as the [mysql select] span, except for the high time consuming span we want to keep

@black-adder
Copy link
Contributor

@tiffon @everett980 this would be a pretty nice filter to have on the UI.

@yurishkuro
Copy link
Member

@yzpangu have you tried expand/collapse functionality in the UI? It allows you to focus on important spans already.

@yzpangu
Copy link
Author

yzpangu commented May 22, 2019

I mean it's not just no need shown on the UI, it's not stored. These low time consuming spans don't make much sense to us, and storage costs are high.I want discard it when span time is small,no stored

@yurishkuro
Copy link
Member

In the current architecture it is not easy to suppress the storage of such spans. Each span records a reference to its parent, so if you have a chain A->B->C and span B meets your criteria of being "too short", if we simply drop it then span C will become orphaned and the UI will have to show it as the top-level span. To be able to cleanly exclude certain spans from the graph you first need to collect the full trace.

We have some plans to look into pre-aggregating traces before storing (because that would allow batched writes, which are more efficient in Cassandra), but I can't give any timeline of when this will be implemented.

@yzpangu
Copy link
Author

yzpangu commented May 22, 2019

I see. Thank you

@jpkrohling
Copy link
Contributor

We have some plans to look into pre-aggregating traces before storing

How do you determine that a trace is "done" ?

@yurishkuro
Copy link
Member

for the purpose of storage it's not necessary to know if the trace is "done", since it's just a performance optimization.

@wuyupengwoaini
Copy link

wuyupengwoaini commented May 28, 2019

@yzpangu I suppose that you only need to customize the Reporter interface to meet your needs.But it only can meet your part of needs because of the reasons mentioned above by @yurishkuro .

e.g. If the span has no child span like [mysql select] span,you can customize reporter.

`
import io.jaegertracing.internal.JaegerSpan;
import io.jaegertracing.spi.Reporter;
import io.opentracing.tag.Tags;

import java.util.Map;

public class MySqlCustomeizeReporter implements Reporter {
private Reporter realReporter;

private int minDuration;

public MySqlCustomeizeReporter(Reporter realReporter, int minDuration) {
    this.realReporter = realReporter;
    this.minDuration = minDuration;
}

@Override
public void report(JaegerSpan span) {
    if (!isDBSpan(span) || span.getDuration() < minDuration) {
        realReporter.report(span);
    }
}

private boolean isDBSpan(JaegerSpan span) {
    Map<String, Object> tags = span.getTags();
    if (tags == null || tags.size() == 0) {
        return false;
    }
    String dbStatement = (String) tags.get(Tags.DB_STATEMENT);
    return dbStatement != null && dbStatement.length() > 0;
}

@Override
public void close() {
    realReporter.close();
}

}
`

@yzpangu
Copy link
Author

yzpangu commented Jun 2, 2019

@yurishkuro maybe jaeger should support customize the Reporter interface and Sampler interface, as @wuyupengwoaini said. make it More scalable

import com.yunzong.taoist.common.customize.reporter.MyReporter;
import io.jaegertracing.Configuration;
import io.jaegertracing.internal.JaegerTracer;
import io.jaegertracing.spi.Reporter;

public class MyConfiguration extends Configuration{

	public MyConfiguration(String serviceName) {
		super(serviceName);
		// TODO Auto-generated constructor stub
	}

	@Override
	public JaegerTracer.Builder getTracerBuilder() {
		JaegerTracer.Builder builder = super.getTracerBuilder();
		Reporter realReporter = builder.getReporter(); // The parent class does not define this method,  I defined and recompile the jar
		MyReporter reporter = new MyReporter(realReporter,20);
		builder.withReporter(reporter);
		return builder;
	}
}

@quaff
Copy link

quaff commented Jun 26, 2019

Here is my solution

import java.util.function.Predicate;

import io.jaegertracing.internal.JaegerSpan;
import io.jaegertracing.spi.Reporter;
import io.opentracing.tag.Tags;

public class DelegatingReporter implements Reporter {

	private final Reporter delegate;

	private final Predicate<JaegerSpan> predicate;

	public DelegatingReporter(Reporter delegate, Predicate<JaegerSpan> predicate) {
		this.delegate = delegate;
		this.predicate = predicate;
	}

	@Override
	public void report(JaegerSpan span) {
		if (span.context().isDebug() || Boolean.TRUE.equals(span.getTags().get(Tags.ERROR.getKey()))
				|| span.getTags().get(Tags.SAMPLING_PRIORITY.getKey()) != null || span.getTags().get("async") != null
				|| predicate.test(span))
			delegate.report(span);
	}

	@Override
	public void close() {
		delegate.close();
	}

}

Reporter reporter = new RemoteReporter.Builder().withSender(sender).build();
reporter = new DelegatingReporter(reporter, span -> span.getDuration() >= 5 * 1000);

@yurishkuro
Copy link
Member

@quaff your solution can easily lead to broken traces, per my earlier #1552 (comment)

@quaff
Copy link

quaff commented Jul 5, 2019

@yurishkuro I realize that, so I add some tag conditions to mitigate it.

@yurishkuro
Copy link
Member

now duplicate of jaegertracing/jaeger-ui#435

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants