Problems starting Juris-M on CircleCI #34
Comments
|
First encounter. Sounds like it must be a result of my crappy code formatting. Hmm. |
|
First difference spotted (how relevant it is I don't know): my system is Ubuntu 17.04, CircleCI is Ubuntu 14.04.4 LTS. |
|
Probably relevant (since I use the |
|
@fbennet, @retorquere so what would be helpful here trying to update BBL's circle config so it runs your test again, making Jurism travis build fail… ? I'm happy to throw some time at it, since this is becoming a pretty big problem for me. Just point me in the right direction. |
|
I don't know what could be changed on the BBT side; Zotero 5 launches without issue. If the xvfb issue is the problem, maybe a backport for 14.04 LTS exists (14.04 uses 2:1.15.1, 17.10 has 2:1.19.5), but since it's a minor release of xvfb, it's not immediately obvious that that would be the problem. Another option would be to move BBTs testing to the docker-based circle2.0 infrastructure, which has a lot more freedom in putting together the environment, but I don't currently have the time to dive in. Circle offers a migration guide but it's not a 10-minute job (I tried). This would be the most attractive option for me since I'd have to do this at some point anyhow. That's all assuming it's something solely on the BBT side though. Launching under xvfb is a necessity for my tests. xvfb could in principle be replaced with xtightvnc, which has a headless mode, or Xdummy, but I haven't tried. The selenium crew has docker images which come with vnc set up, but I haven't looked at them. |
I m not ruling out a bug in jurism, but given that you were able to run on local, I would like to either a) get a better description of the actual problem from circle, or b) get the tests that are there to be consistent between your local and circle.
That was my initial thought as well. I normally use travis, so this will take some learning time, but I ll see what I can do. |
|
It's not hard to get into the error environment in circle. Clone the repo, activate circle, make sure to select 4 (free) containers for your project, and you'd be ready to go. The Juris-M tests run in containers 3 and 4. You can ssh into your containers to play around with the environment. |
|
Progress TravisCI node 52: node 54 runs fine though |
|
I have no issue against going with Travis, but the reason I moved from Travis (where I ran my tests initially) to CircleCI is that CircleCI allowed me to split my tests into chunks each running in a separate container (which made total run time for my tests a lot less) but would wait for all tests to have ran before entering the release phase, and would only run the release phase when all containers ran green (I believe Circle calls this fan-in / fan-out). All sorts of workarounds exist for Travis, but CircleCI just does it out of the box. |
|
@retorquere this is from running jurism own test suite, not bbl. If you have a few minutes maybe we can chat (seems more efficient). This looks very fixable to me. Alternatively, could you run your test using node |
|
Sure, where do you want to chat? Tests on 9.4.0 are running. |
|
how about skype? |
|
Sure |
|
All tests pass on 9.4.0 (unsurprisingly, really, as I really only use it for webpack as far as the BBT code is concerned). |
|
Here is a link to the travis log for the failed jurism build, however, the same errors that BBL reports on circle are present in the standard Zotero/zotero builds?? |
|
So I ve done some more poking around, and while my travis builds are passing now, circle still isn't. The first leads seem to have all been red herrings. The same errors from the zotero.log on circle for jurism are also in the regular zotero builds, including: But they don't cause it to fail 😖 . I ll poke the debug-bridge and the add-on installation process to see if something pops up. |
|
That doesn't cause the build to fail for me either. It's the last line. I'm trying to get Xdummy or x11vnc to work, but my test harness uses a ruby gem to set up the headless env, and I haven't yet found a replacement. Assuming that xvfb is at fault at all. |
|
I've tried the circle 1.0 -> 2.0 migration service and that currently fails on "mkdir: not found". |
|
I keep running into errors related to the installation of @fbennett is there something special about how jurism interacts with zotero-addons? Could you maybe chime in on how you normally go about incorprating upstream changes? |
|
How are addons installed during tests? Selenium handles installation for me. |
|
The Circle-CI tests are against the core JM code, yes? Nothing special that I can think of. |
|
My circle tests just download the latest Juris-M (or zotero), manipulate the profile using selenium to install the plugins, and then start Juris-M (or zotero). This runs under xvfb. There's no difference between my zotero tests and my Juris-M tests other than that they download the one or the other. |
|
But I'll have to spend time on migrating my circle tests to circle 2.0 anyhow before the summer, because circle 1.0 is being phased out. Maybe that makes a difference. |
|
Only a couple of potential things come to mind. Juris-M is a bit slower on some operations, and I extended some of the timeouts on the Z tests in the JM fork. The other thing is that, in order to support the Abbreviation Filter, I've had to run the processor in a coroutine, which forced me to promisify parent functions up the chain to its root in the UI. AF is a central component of JM, so the promisification there will stay. I'm pretty sure that Dan does not intend to go there with Zotero (he hasn't responded to my mentions of the issue), so I'll be removing AF from the list of Zotero plugins shortly. |
|
Progress, of a kind. I'm a good way through porting from CircleCI 1.0 to 2.0, and Zotero also doesn't start in the 2.0 environment 🙄. Considering ditching CircleCI, looking into alternatives. |
|
Both Zotero and Juris-M fail tests intermittently in Travis-CI, although everything passes locally. Dan says it's due to race conditions that are hard to control in that environment. |
|
The problem I'm seeing is unlikely to be caused by race conditions, but race conditions are structural problems. They may be more or less likely to be triggered in some environments, but the environment is not the cause of them. |
|
It it correct that Juris-M has its |
|
(I think this is different from the 4 series, and it could be this is really all that there's too it. I can test further tonight). Zotero runs again. |
|
In the light of morning, I've gone through the Zotero PHP scripts that handle update/download negotiation, and both respond to the call with a Location (not Content-Location) header -- a external redirect to a public URL. I'm pretty sure that parking update assets on a GitHub release will be a winner. Looking forward to getting that set up. |
|
Actually, that description isn't quite correct. Download requests (to |
Ha. I wish. We're currently staggering along without automatic updates for the client in Juris-M, so it's an open field. As I ease into the thought of it, GitHub releases do look attractive. Apart from the speed advantage, I'm a short few years away from retirement, and continued access to University infrastructure isn't guaranteed. So the more stuff I can move to robust, free-of-fees external infrastructure, the better. |
|
I've set it up so it's almost entirely free
Although S3 is not free, last month I got my first-ever bill for it, a whopping $0.14, because I was experimenting with a new way to capture the logs, and I used it a lot. I have S3 set up to auto-delete logs after a week to prevent cost run-up for storage. |
|
Has |
|
Is that this commit? zotero/zotero-standalone-build@528162e |
|
Hmm, looks like it did make it. For Juris-M I patch up the More debugging. Good that we have a long weekend ahead. |
|
I've made some progress -- there were at least two issues with starting:
Still, the initialization is stuck at https://file.io/rT4FRi for some reason inside Travis. On desktop, it starts normally of course 🙄 . |
|
Got it! Travis does not make this easy to diagnose. In this case, I was trying to change too many things at once -- the deadly combo was that I have a dataDir set in my starter profile that I use for more efficient startup, and that was OK, because I overwrote it on test start anyhow. Now I started using Nice. In any case, I want to create a new starter profile (come to think of it, I may just be able to use the same starter profile for Zotero and Juris-M since Juris-M probably upgrades the profile as it starts. Anhow), but for safety (so I don't accidentally clobber my own research DB as I'm testing on the same system) I create files names |
|
Ah and Juris-M still does not seem to tolerate me removing BibTeX.js and BibLaTeX.js. |
|
But Zotero is cool with that? What's the error?
…On Fri, Apr 19, 2019, 21:41 Emiliano Heyns ***@***.***> wrote:
Ah and Juris-M still does not seem to tolerate me removing BibTeX.js and
BibLaTeX.js.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#34 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAASMSWGH5D42JK7ZRQZOG3PRG4V3ANCNFSM4DZS53NQ>
.
|
|
Starting with |
|
WRT the removed translators, Juris-M is apparently cool with this... looks to just work. Must be fallout from the botched profile I had. Sorry. I've also seen Juris-M create Sorry for all the noise. |
|
No worries, if I'm busy at this end I always have the option of quietly ignoring things. The error you posted seemed to be rooted in Myles, which is a plugin that draws on files in a git subproject/subdirectory. You're pulling with --recursive? |
|
I'm not pulling anything, BBT tests simply install the latest Juris-M (or Zotero) binaries, create a profile and install BBT in the profile, and run tests by triggering internal APIs. Is Myles bundled with Juris-M (I I think I saw ODF-Scan was bundled)? I don't install any plugins into the profile other than BBT and the debug bridge I use to drive the tests. |
|
(this is all on Linux, I should have mentioned this) So what I've just tried is this: Start with
Start normally and try to move to
|
|
I mean my tests run on Travis so I'm happy but it makes it tricky to run local tests as I rely heavily on the datadir being contained to the test; on Travis this is not an issue as I'm starting with a clean machine in any case. |
|
Myles is a bundled plugin, yes. Maybe something needs to be done to make it -datadir - aware in your setup? Is the -datadir argument a change from your previous installs? It may be looking in the wrong place for its support files. |
|
Where exactly do you use that argument? With a staging build, running ./jurism -datadir ~/MyProfileDir, I get my normal library (in a custom location). Not sure what your setup is or what it's trying to do. |
|
(ah - the odd moniker may be confusing - I'm apparently in the account of my RA, we set up an account this PM. Frank Bennett here.) |
|
It's a change from previous tests -- previously I would set Your accountant uses github? I must have a talk with mine. She needs to up her game evidently. |
|
(the Internet has not been kind to abbreviations: research assistant) |
|
Looks like Jurism is not honoring a -datadir <profile> argument. Will look at that tomorrow. |
|
Ooh, I should get me one of those! (who am I kidding. I strive to be one of those). |
|
An unusual circumstance here as well. Will check up on -datadir tomorrow. |
|
Playing with this, it looks like the
When I fire up So ... yeah. That all pretty much confirms the behavior that you outlined above. I'm not clear on what the expected behavior is, though. What setup/options are you using for Zotero, and what exactly differs between the behavior of Zotero and Jurism? |
|
Well.. Usually, I'm creating a new profile by manipulating profiles.ini, including an absolute path (outside the regular ~/. jurism / ~/. zotero directory) and moving an existing profile snapshot there. For zotero that suffices because when I use |
|
(I want to stress that my use of these things is atypical. I don't even know it anyone besides me uses datadir profile) |
In the hopes that perhaps you've seen this before: I'm starting Juris-M / Zotero on CircleCI under xvfb using
This works for Zotero (and locally on my Ubuntu system) but for Juris-M it only works locally; on CircleCI I am getting
this is the totality of the log. All the lines except the last one are also in the zotero-generated log, so I'm guessing it's that last line that does it, but a bit of googling around yields the conclusion "something's wrong"☹️ There's a lot more chatter in the Zotero log before the
Client failed to connect to the D-BUS daemon: Address does not contain a colonif that matters.I can give you access to a session on circleci by ssh if that's helpful.
The text was updated successfully, but these errors were encountered: