Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase truncate limit for keywords in the Metricbeat templates from 1024 #8076

Closed
marius-dr opened this issue Aug 24, 2018 · 3 comments
Closed
Labels

Comments

@marius-dr
Copy link
Member

In the current metricbeat index template, the ignore_above limit is set as 1024. This is consistently passed in the system.process.cmdline field by Elasticsearch itself (on Windows).
A typical cmdline for ES on windows looks something like this:

"C:\Program Files\Java\jdk1.8.0_121\bin\java.exe" -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.io.tmpdir=C:\Users\ratonbox\AppData\Local\Temp\elasticsearch -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=data -XX:ErrorFile=logs/hs_err_pid%p.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:logs/gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=32 -XX:GCLogFileSize=64m -Delasticsearch -Des.path.home="C:\Users\ratonbox\Desktop\elasticsearch-6.4.0" -Des.path.conf="C:\Users\ratonbox\Desktop\elasticsearch-6.4.0\config" -Des.distribution.flavor="default" -Des.distribution.type="zip" -cp "C:\Users\ratonbox\Desktop\elasticsearch-6.4.0\lib\*" "org.elasticsearch.bootstrap.Elasticsearch"
which has 1126 characters.

This makes the whole field unsearchable, without the user really being aware of it.

@ruflin
Copy link
Collaborator

ruflin commented Aug 24, 2018

Would it solve the problem if we set it to 2048 for cmdline or do you expect this also be exceeded in some cases?

@marius-dr
Copy link
Member Author

The windows cmd.exe and batch files only allow 8129 chars to be entered, while the windows API shows a max limit of 32767 ( USHORT / 2 ).
But for our use case 2048 should be more than enough. We barely get past the 1024 limit with ES, which has pretty complex params, I doubt anything else would go higher.

ruflin added a commit to ruflin/beats that referenced this issue Aug 27, 2018
Current default for ignore_above is 1024. This is too short for some command line entries on Windows. This increases it to 2048.

Closes elastic#8076
ruflin added a commit to ruflin/beats that referenced this issue Aug 27, 2018
Current default for ignore_above is 1024. This is too short for some command line entries on Windows. This increases it to 2048.

Closes elastic#8076
@ruflin
Copy link
Collaborator

ruflin commented Aug 27, 2018

I opened #8101 with 2048.

@ph ph closed this as completed in #8101 Aug 27, 2018
ph pushed a commit that referenced this issue Aug 27, 2018
Current default for ignore_above is 1024. This is too short for some command line entries on Windows. This increases it to 2048.

Closes #8076
ruflin added a commit to ruflin/beats that referenced this issue Aug 28, 2018
Current default for ignore_above is 1024. This is too short for some command line entries on Windows. This increases it to 2048.

Closes elastic#8076

(cherry picked from commit e1d8c15)
ruflin added a commit to ruflin/beats that referenced this issue Aug 28, 2018
Current default for ignore_above is 1024. This is too short for some command line entries on Windows. This increases it to 2048.

Closes elastic#8076

(cherry picked from commit e1d8c15)
exekias pushed a commit that referenced this issue Aug 28, 2018
Current default for ignore_above is 1024. This is too short for some command line entries on Windows. This increases it to 2048.

Closes #8076

(cherry picked from commit e1d8c15)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants