Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gauge 0.1.8 broken on mac - Incorrect value for runner_request_timeout #222

Closed
barrysvee opened this issue Oct 6, 2015 · 22 comments
Closed
Labels

Comments

@barrysvee
Copy link

Test were running with 0.1.7 on OS X but fail after upgrading to 0.1.8 with the following error. Looks like a conversion from null to a time value.

[WARNING] Incorrect value for runner_request_timeout in property file.Cannot convert to time

@kashishm
Copy link
Contributor

kashishm commented Oct 7, 2015

This warning message is shown if the value of runner_request_timeout is not set in gauge.properties file. The documentation covers how to set values for the internal configurations.

Gauge doesn't fail tests because of the [WARNING] messages. Tests may be failing because of some other error, the whole console output/stack trace would be helpful in debugging this.

@barrysvee
Copy link
Author

Thank you for the reply. I have added both "runner_request_timeout = 10000" and "plugin_connection_timeout = 10000" to my gauge.properties and I see the warnings:

"Incorrect value for plugin_kill_timeout in property file.Cannot convert to time"
and
"[WARNING] Incorrect value for runner_request_timeout in property file.Cannot convert to time"

I realize that these are warnings, but the values in my gauge.properties are being ignored. Regardless, the tests all went from passing to failing after upgrading from 0.1.7 to 0.1.8 from a terminal window in OS X.

I started IDEA and was able to run the specs no problem. This seemed to correct execution from the command line since I re-ran my previous statement in the terminal window and the spec ran. This is not an ideal fix for our Mac testing VMs that will not IDEA installed.

Also, from IDEA the xml-report and html-report plugins generated their output but the following occurs from the command line with no reports being generated:

[WARNING] Error starting plugin Html Report 1.0.4. pipe: too many open files
[WARNING] Error starting plugin xml-report. Failed to get plugin.json. Plugin xml-report is not installed. To install, run gauge --install xml-report.

I killed orphaned gauge processes before re-running the spec.

@mahendrakariya
Copy link
Contributor

Sorry to hear about the issue that you are facing.

The one thing that should have changed when you ran the specs from IntelliJ is that it would have recompiled the java files. While running tests on VMs, if you face this issue, just delete the out and gauge_bin folder before running gauge specs.

Thanks for reporting the issue that you faced with html-report generation. We will take a look at it.

Gauge was not able to generate the xml reports because the xml-report plugin is not installed. You can install it using gauge --install xml-report.

Regarding Incorrect value for plugin_kill_timeout error, could you please let us know what is the encoding of gauge.properties file? It should ideally be utf8. We just want to cross check.

@apoorvam
Copy link
Contributor

apoorvam commented Oct 8, 2015

@barrysvee Please make sure you have read access to the gauge.properties file from command-line.

The warning you have mentioned happens if Gauge is not able to read the values to certain variables in gauge.properties file. But this would be set to default in case of this problem.
Also, please check if there are any logs related to this issue. You can find logs inside the project directory.

@vshanve
Copy link

vshanve commented Oct 14, 2015

@apoorvam I am able to find gauge.properties under program files in Windows machine. But unable to see this in MAC. Do you have the path where I can find this file in mac? Also how can I set the property file values to default ?

@apoorvam
Copy link
Contributor

@vshanve Gauge plugins are installed at ~/.gauge/plugins directory in case of MacOS. You can check the Installation Troubleshooting here.

During Gauge installation, the default values in gauge.properties file are already set. You need not alter this file unless required.

@mahendrakariya
Copy link
Contributor

On Mac, the location of properties files depends on how you installed it.

If you installed from source or used the .pkg installer, you will find the gauge.properties file in /usr/local/share/gauge.

In case you used homebrew, the file will be in /usr/local/Cellar/gauge/0.1.8/share/gauge/.

As Apoorva said, the default values are already set in the file. But just for your reference, here they are.

gauge_repository_url = http://raw.github.com/getgauge/gauge-repository/master
runner_connection_timeout = 30000
plugin_connection_timeout = 10000
plugin_kill_timeout = 4000
runner_request_timeout = 30000

@vshanve
Copy link

vshanve commented Oct 14, 2015

@apoorvam @mahendrakariya
Thank You. Updated the property file with default values. I ran these install commands as well 'gauge --install html-report' and 'gauge --install xml-report'. Below are the observations I can see with Gauge version 0.1.7 and 0.2.0.

Gauge 0.1.7:

*Though I have these below warning the specs are running in this gauge version.

[WARN] Incorrect value for runner_request_timeout in property file.Cannot convert to time
[WARN] Error starting plugin html-report. Failed to get plugin.json.Plugin html-report is not installed.To install,run 'gauge --install html-report'.
[WARN] Error starting plugin xml-report. Failed to get plugin.json.Plugin xml-report is not installed.To install,run 'gauge --install xml-report'.
[WARN] Incorrect value for plugin_kill_timeout in propert file.Cannot convert to time

Gauge 0.2.0:

*Here specs are not running and I don’t see timeout errors on property file.

[INFO] process /Library/Java/JavaVirtualMachines/jdk1.7.0_79.jdk/Contents/Home/jre/bin/java with pid 482 quit unexpectedly. exit status 1
[WARNING] Error starting plugin Html Report 1.1.0. pipe: too many open files
[WARNING] Error starting plugin xml Report 0.1.0. pipe: too many open files
[ERROR] Failed to initialize suite datastore. Error: use of closed network connection

@mahendrakariya
Copy link
Contributor

Warnings about timeout values are not fatal, since gauge will internally take the default values. That won't stop the execution.

But errors that you got with Gauge 0.2.0 are fatal. Could you please ensure that you have the latest versions of all the plugins? The minimum plugin versions compatible with Gauge 0.2.0 are mentioned in this announcement email.

In case you have older versions of the plugins, please run gauge --update-all. This will update all the installed plugins.

Please note that the newer plugin versions are not compatible with Gauge 0.1.7.

@mahendrakariya
Copy link
Contributor

To solve the pipe: too many open files error, could you please ensure that there are no redundant processes running on your system? Also close applications, files, etc. that you don't need. If you still face this issue, please let us know.

Context about this error:
By default, maximum number of files that Mac OS X can open is 12288. If any process tries to cross this limit, the operating system throws pipe: too many open files error. This limit can be increased. But you should do that only if its absolutely required. See this answer on stackoverflow for more details.

To know how many files are currently open, you can run the command lsof | wc -l.

@vshanve
Copy link

vshanve commented Oct 14, 2015

@mahendrakariya

While running Gauge 0.1.7 the below versions of plugins are used.
Java - 0.2.2
xml - 0.0.2
html - 1.0.4

While running Gauge 0.2.0 the below versions of plugins are used.
Java - 0.3.1
xml - 0.1.0
html - 1.1.0


1)With Gauge 0.2.0, though I used the latest plugin versions. I typed in the command 'gauge --update-all'. It executed and ends with warning saying,

  • for html 1.1.0 is already installed
  • for xml 0.1.0 is already installed
  • for java 0.3.1 is already installed
    Then I restarted my Mac.

2)Now on running my specs for the first time it ran successfully and generated reports.Just with Plugin_kill_timeout incorrect value error.

3)From second time onwards it starts giving the same error message as mentioned in my previous post.

[INFO] process /Library/Java/JavaVirtualMachines/jdk1.7.0_79.jdk/Contents/Home/jre/bin/java with pid 482 quit unexpectedly. exit status 1
[WARNING] Error starting plugin Html Report 1.1.0. pipe: too many open files
[WARNING] Error starting plugin xml Report 0.1.0. pipe: too many open files
[ERROR] Failed to initialize suite datastore. Error: use of closed network connection

Now I can see these warnings as well:
[WARN] Failed to get configuration from Gauge property file.Error: open/usr/local/share/gauge/gauge.properties:too many open files
[WARN] Incorrect value for plugin_kill_timeout in propert file.Cannot convert to time


On executing below command I get this value 1883

lsof | wc -l
1883

@mahendrakariya
Copy link
Contributor

@vshanve Thanks for this info.
Could you also provide us the entire logs for this error? That will help us narrow down the issue.

Also, what number do you get when you run the command ulimit -n? If this number is small, can you run the command ulimit -S -n 2048 and try running gauge specs again?

@mahendrakariya
Copy link
Contributor

We have raised a separate issue for the incorrect timeout values.
getgauge/common#1

@vshanve
Copy link

vshanve commented Oct 15, 2015

@mahendrakariya

Unable to attach the logs for failure as txt file.So pasting it in my next post.


I got 256 on running the below command

ulimit -n
256

Then I ran this command.
ulimit -S -n 2048

*Now on running the specs its running successfully.Tried running different specs repeatedly. Its running good now.I dont see any error message now and the html reports got generated.

*If I close my terminal and reopen a new one, I start seeing the issues again. Again if I pass those ulimit command I can run the specs successfully.

*I can see this command is to set upper limit 'ulimit -S -n 2048' How it resolved everything?

Thank you.

@vshanve
Copy link

vshanve commented Oct 15, 2015

@mahendrakariya

09:17:23.334 [INFO] Note: Some input files use or override a deprecated API.
09:17:23.334 [INFO] Note: Recompile with -Xlint:deprecation for details.
09:17:23.334 [INFO] Note: Some input files use unchecked or unsafe operations.
09:17:23.334 [INFO] Note: Recompile with -Xlint:unchecked for details.
09:17:26.196 [INFO] 1 [main] WARN org.reflections.Reflections - could not create Dir using com.thoughtworks.gauge.scan.ClasspathScanner$1@49b96230 from url file:/Users/.gauge/plugins/java/0.3.1/. skipping.
java.lang.NullPointerException
at com.thoughtworks.gauge.scan.ClasspathScanner$1.createDir(ClasspathScanner.java:53)
at org.reflections.vfs.Vfs.fromURL(Vfs.java:99)
at org.reflections.vfs.Vfs.fromURL(Vfs.java:91)
at org.reflections.Reflections.scan(Reflections.java:237)
at org.reflections.Reflections.scan(Reflections.java:204)
at org.reflections.Reflections.(Reflections.java:129)
at com.thoughtworks.gauge.scan.ClasspathScanner.createReflections(ClasspathScanner.java:60)
at com.thoughtworks.gauge.scan.ClasspathScanner.scan(ClasspathScanner.java:37)
at com.thoughtworks.gauge.GaugeRuntime.main(GaugeRuntime.java:36)
09:17:26.196 [INFO] 3 [main] WARN org.reflections.Reflections - could not create Dir using directory from url file:/Users/.gauge/plugins/java/0.3.1/
. skipping.
java.lang.NullPointerException
at org.reflections.vfs.Vfs$DefaultUrlTypes$3.matches(Vfs.java:239)
at org.reflections.vfs.Vfs.fromURL(Vfs.java:98)
at org.reflections.vfs.Vfs.fromURL(Vfs.java:91)
at org.reflections.Reflections.scan(Reflections.java:237)
at org.reflections.Reflections.scan(Reflections.java:204)
at org.reflections.Reflections.(Reflections.java:129)
at com.thoughtworks.gauge.scan.ClasspathScanner.createReflections(ClasspathScanner.java:60)
at com.thoughtworks.gauge.scan.ClasspathScanner.scan(ClasspathScanner.java:37)
at com.thoughtworks.gauge.GaugeRuntime.main(GaugeRuntime.java:36)
09:17:26.196 [INFO] 3 [main] WARN org.reflections.Reflections - could not create Vfs.Dir from url. ignoring the exception and continuing
org.reflections.ReflectionsException: could not create Vfs.Dir from url, no matching UrlType was found [file:/Users/.gauge/plugins/java/0.3.1/*]
either use fromURL(final URL url, final List urlTypes) or use the static setDefaultURLTypes(final List urlTypes) or addDefaultURLTypes(UrlType urlType) with your specialized UrlType.
at org.reflections.vfs.Vfs.fromURL(Vfs.java:109)
at org.reflections.vfs.Vfs.fromURL(Vfs.java:91)
at org.reflections.Reflections.scan(Reflections.java:237)
at org.reflections.Reflections.scan(Reflections.java:204)
at org.reflections.Reflections.(Reflections.java:129)
at com.thoughtworks.gauge.scan.ClasspathScanner.createReflections(ClasspathScanner.java:60)
at com.thoughtworks.gauge.scan.ClasspathScanner.scan(ClasspathScanner.java:37)
at com.thoughtworks.gauge.GaugeRuntime.main(GaugeRuntime.java:36)
09:17:26.687 [INFO] java.net.SocketException: Connection reset
09:17:26.687 [INFO] at java.net.SocketInputStream.read(SocketInputStream.java:196)
at java.net.SocketInputStream.read(SocketInputStream.java:122)
09:17:26.687 [INFO] at com.google.protobuf.CodedInputStream.tryRefillBuffer(CodedInputStream.java:1121)
09:17:26.687 [INFO] at com.google.protobuf.CodedInputStream.refillBuffer(CodedInputStream.java:1081)
09:17:26.687 [INFO]
09:17:26.687 [INFO] at com.google.protobuf.CodedInputStream.readRawByte(CodedInputStream.java:1149)
09:17:26.687 [INFO]
09:17:26.687 [INFO] at com.google.protobuf.CodedInputStream.readRawVarint64SlowPath(CodedInputStream.java:778)
09:17:26.687 [INFO]
09:17:26.687 [INFO] at com.google.protobuf.CodedInputStream.readRawVarint64(CodedInputStream.java:770)
09:17:26.687 [INFO] at com.thoughtworks.gauge.connection.GaugeConnection.getMessageLength(GaugeConnection.java:51)
09:17:26.687 [INFO]
09:17:26.687 [INFO] at com.thoughtworks.gauge.connection.GaugeConnection.getAPIResponse(GaugeConnection.java:150)
09:17:26.687 [INFO] at com.thoughtworks.gauge.connection.GaugeConnection.getStepValue(GaugeConnection.java:190)
09:17:26.687 [INFO]
09:17:26.687 [INFO] at com.thoughtworks.gauge.connection.GaugeConnection.getStepValue(GaugeConnection.java:176)
09:17:26.687 [INFO]
09:17:26.687 [INFO] at com.thoughtworks.gauge.scan.StepsScanner.buildStepRegistry(StepsScanner.java:47)
09:17:26.687 [INFO]
09:17:26.687 [INFO] at com.thoughtworks.gauge.scan.StepsScanner.scan(StepsScanner.java:39)
09:17:26.687 [INFO]
09:17:26.687 [INFO] at com.thoughtworks.gauge.scan.ClasspathScanner.scan(ClasspathScanner.java:39)
09:17:26.687 [INFO] at com.thoughtworks.gauge.GaugeRuntime.main(GaugeRuntime.java:36)
09:17:26.687 [INFO]
09:17:26.688 [INFO] Exception in thread "main" java.lang.NullPointerException
09:17:26.688 [INFO] at com.thoughtworks.gauge.registry.StepRegistry.addStepImplementation(StepRegistry.java:29)
09:17:26.688 [INFO] at com.thoughtworks.gauge.scan.StepsScanner.buildStepRegistry(StepsScanner.java:48)
09:17:26.688 [INFO] at com.thoughtworks.gauge.scan.StepsScanner.scan(StepsScanner.java:39)
09:17:26.688 [INFO] at com.thoughtworks.gauge.scan.ClasspathScanner.scan(ClasspathScanner.java:39)
09:17:26.688 [INFO]
09:17:26.688 [INFO] at com.thoughtworks.gauge.GaugeRuntime.main(GaugeRuntime.java:36)
09:17:26.688 [INFO]
09:17:26.710 [ERROR] Validation failed. The following steps have errors
09:17:26.710 [ERROR] /Users/dev/apps/AthenaBrowser/main/Gauge/DriverTest/specs/DefaultRC - ToolBar.spec:18: use of closed network connection. Verify
09:17:26.710 [ERROR] /Users/dev/apps/AthenaBrowser/main/Gauge/DriverTest/specs/DefaultRC - ToolBar.spec:19: use of closed network connection. Navigate to the next item
09:17:26.710 [ERROR] /Users/dev/apps/AthenaBrowser/main/Gauge/DriverTest/specs/DefaultRC - ToolBar.spec:20: use of closed network connection. Verify
09:17:26.711 [ERROR] /Users/dev/apps/AthenaBrowser/main/Gauge/DriverTest/specs/DefaultRC - ToolBar.spec:21: use of closed network connection. Wait "25" seconds
09:17:26.711 [ERROR] /Users/dev/apps/AthenaBrowser/main/Gauge/DriverTest/specs/DefaultRC - ToolBar.spec:22: use of closed network connection. Verify
09:17:26.711 [ERROR] /Users/dev/apps/AthenaBrowser/main/Gauge/DriverTest/specs/DefaultRC - ToolBar.spec:23: use of closed network connection. Verify
09:17:26.711 [ERROR] /Users/dev/apps/AthenaBrowser/main/Gauge/DriverTest/specs/DefaultRC - ToolBar.spec:24: use of closed network connection. Click dialog button
09:17:26.711 [ERROR] /Users/dev/apps/AthenaBrowser/main/Gauge/DriverTest/specs/DefaultRC - ToolBar.spec:25: use of closed network connection. Wait "1" seconds
09:17:26.711 [ERROR] /Users/dev/apps/AthenaBrowser/main/Gauge/DriverTest/specs/DefaultRC - ToolBar.spec:26: use of closed network connection. Verify
09:17:26.711 [ERROR] /Users/dev/apps/AthenaBrowser/main/Gauge/DriverTest/specs/DefaultRC - ToolBar.spec:27: use of closed network connection. Verify
09:17:26.711 [ERROR] /Users/dev/apps/AthenaBrowser/main/Gauge/DriverTest/specs/DefaultRC - ToolBar.spec:28: use of closed network connection. Verify button
09:17:26.712 [ERROR] /Users/dev/apps/AthenaBrowser/main/Gauge/DriverTest/specs/DefaultRC - ToolBar.spec:38: use of closed network connection. EndExam
09:17:26.712 [WARNING] Error starting plugin Html Report 1.1.0. pipe: too many open files
09:17:26.712 [WARNING] Error starting plugin Xml Report 0.1.0. pipe: too many open files
09:17:26.712 [ERROR] Failed to initialize suite datastore. Error: use of closed network connection
09:17:26.712 [WARNING] Failed to get configuration from Gauge properties file. Error: open /usr/local/share/gauge/gauge.properties: too many open files
09:17:26.712 [WARNING] Incorrect value for plugin_kill_timeout in property file. Cannot convert to time
09:17:26.712 [INFO] process /Library/Java/JavaVirtualMachines/jdk1.7.0_79.jdk/Contents/Home/jre/bin/java with pid 6999 quit unexpectedly. exit status 1
09:17:26.712 [DEBUG] Runner exited with error: exit status 1
09:17:26.712 [INFO] Specifications: -1 executed, -1 passed, 0 failed, 1 skipped
09:17:26.712 [INFO] Scenarios: -1 executed, -1 passed, 0 failed, 1 skipped
09:17:26.712 [INFO] Total time taken: 0

@mahendrakariya
Copy link
Contributor

Glad to know it is working fine now!

ulimit sets the upper limit over resources available to shell and the processes started by it. The -n flag is for the maximum number of open file descriptors. The command ulimit -S -n 2048 increased the soft upper limit of file descriptors available to shell from 256 to 2048, for the current session. If you want to set it permanently, you can add this command to .bashrc or .zshrc depending on what shell you are using. If you want to see the upper limit for other resources, you can run ulimit -a.

Thanks for the stacktrace. We have created a separate issue for that. I think reflections utility is not able to create new directories because of the upper limit set on number of files. But we need to confirm that.

@vshanve
Copy link

vshanve commented Oct 15, 2015

@mahendrakariya

Thanks. I don't see this 'setting ulimit' issue if I run the specs using gauge 0.1.7. So is it an issue related only to 0.2.0 version?

@mahendrakariya
Copy link
Contributor

We have added a bunch of new features in 0.1.8 and in 0.2.0. We have also updated the dependencies in the new versions of Gauge. It's very likely that this may have increased the number of files that needs to be accessed.

But we do not see it as a concern. Because to add more features, we have to add more files.

Moreover, default upper limit of 256 is pretty low, especially in today's time. Most of the other unix based operating systems, including popular linux distros like ubuntu, fedora etc have 1024 as the default limit. I don't know why Apple has not increased it yet.

@barrysvee
Copy link
Author

Thank you again for your help.

Was there a change between 0.1.7 and 0.1.8 effecting classpath or otherwise locating resources? From terminal the out folder was not generated, although gauge_bin was. I deleted both folders before running the tests. I'm wondering if gauge cannot find its own default values for the settings it's complaining about.

xml-report is already installed.

$gauge -v
Gauge version: 0.1.8

Plugins

html-report (1.0.4)
java (0.2.2)
xml-report (0.0.2)

@barrysvee
Copy link
Author

Perhaps I should open a new defect but after upgrading to 0.2.1 and running gauge --update-all I was able to run one spec successfully and pass but no reports were generated. After running a second time I got:

1: 12 "Incorrect value for runner_request_timeout in property file. Cannot convert to time" warnings
2: 12 "Incorrect value for runner_request_timeout in property file. Cannot convert to time
[WARNING] Failed to get configuration from Gauge properties file. Error: open /usr/local/share/gauge/gauge.properties: too many open files" warnings
3: 2 "Error starting plugin html-report. Failed to get plugin.json. Plugin html-report is not installed. To install, run gauge --install html-report" warnings

Running gauge --version I get:

Error starting plugin html-report. Failed to get plugin.json. Plugin html-report is not installed. To install, run gauge --install html-report.
[WARNING] Error starting plugin xml-report. Failed to get plugin.json. Plugin xml-report is not installed. To install, run gauge --install xml-report.
^Csveebr@BGTMNML-MDCMG3Q: ~/dev/apps/AthenaBrowser/main/Gauge/DriverTest
$gauge --update-all
sveebr@BGTMNML-MDCMG3Q: ~/dev/apps/AthenaBrowser/main/Gauge/DriverTest
$gauge --version
Gauge version: 0.2.1

Plugins

java (0.3.1)
xml-report (0.1.1)
html-report (1.1.0)

And I now have four orphaned gauge processes on my Mac. This was following a reboot.

@barrysvee
Copy link
Author

After following the suggestion to increase the resources with "ulimit -S -n 2048" I was able to run a spec and generate a report.

@mahendrakariya
Copy link
Contributor

For the orphan processes, we have an open issue #166. We will be tracking it there.

Glad to know that everything else has been resolved. We have added setting the ulimit in the troubleshooting section of Gauge's documentation so that others can easily find it.

I will take the liberty of closing this issue. Feel free to reopen in case you get this error again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

No branches or pull requests

5 participants