Do I need to build stanc in BridgeStan v1.0 after a fresh install?
#62
Comments
|
The first time you build a model with bridgestan it downloads stanc from GitHub. You can see some of that in the output you posted ( It seems this is failing in your CI, perhaps GitHub is rate limiting you?
I believe the path set in the example assumes you’re running julia from within the julia/ directory, and you will need to change the path if this is not true. In general, it is probably best to use absolute paths on your own machine, to avoid this |
|
Looking at the code you're using: I believe that the issue may be that you're using Alternatively, you can use |
|
Hi Brian. Yes, I came yesterday and this morning to a similar conclusion as in your first response. Your second message is a clear pointer to something I'm doing wrong so that's next on my list! That make construct is a leftover from earlier versions. Basically the same as I was using while I was "borrowing" stanc from cmdstan and didn't dig too much into that because it works on my Mac. The 2nd route, fiddling with paths in Julia packages, is always tricky I find. I'm also considering a 3rd option, using BridgeStan completely outside StanSample, as its own package. Or the 4th option, treating it as cmdstan. Thanks for looking into this! |
|
I'm far from an expert on make files, but I think generally they can be quite tricky to use from outside the folder they're in. In Julia/Python we use the I also tried to have all of the relative paths based off a variable called Style we use ourselves: Cmd(
`make $output_file`,
dir = abspath(bridgestan),
)Untested but should be exactly equivalent to the above: Cmd(`make -c$(abspath(bridgestan)) $output_file`)Closest to your current code, but importantly tells BridgeStan where it's own root directory is (since it is not where the command is invoked, like it is in the above two) Cmd(`make -f $(abspath(bridgestan))/Makefile BS_ROOT=$(abspath(bridgestan)) $output_file`) |
|
In the current setup of StanSample + BridgeStan, bridge_path holds the absolute path to the bridgestan directory. Setting BS_ROOT is fine on Mac but doesn't help on CI. I'll try to figure out what exactly is downloaded on CI. |
|
After inserting a couple of print statements, it seems the proper Probably because stanc is not installed, the The new optimization, not building stanc but downloading the correct binary the first time needed, do we know if that is in use in any other place using Github workflows? I wonder if the best way out for me is
With option 1 I'm a bit concerned about future maintenance work if stanc itself or flags for stanc might divert. With option 2 I'm running into: I'm just reporting these issues in the hope someone immediately sees the way out! Next up is borrowing stanc from cmdstan. |
|
“Borrowing” stanc is only a good idea if the cmdstan version exactly matches the version of Stan used by bridgestan, which might be a bit difficult to check ahead of time. We are using GitHub actions for our own CI, so I’m not sure why there would be anything specific to that environment which fails. I am not sure how deeply you have integrated bridgestan into StanSample, but anything written pre-1.0 (especially if you have any cached builds or anything like that in Actions) should get a second look. The specific error you’ve shared looks like the one you would get if you tried to load a DSO compiled before #53 with the code from after that change |
|
Thanks again Brian. In the case of StanSample.jl a user must have installed a version of cmdstan and every model compiled by that version of cmdstan would also run I'll go over the CI workflows for BridgeStan to see if I do something different. I see the curl request and the Your 3rd point is spot on! I started from a new environment and installed StanSample and BridgeStan but forget I need to |
|
Please do let us know if it turns out we've done something which unnecessarily makes life harder! If your use case is specific to models which are also used by CmdStan, you can use the |
|
Hi Brian, A lot has gotten better in v1.0 and I need to figure out how I can make this work on CI. The Github CI workflows for BridgeStan do show a correct Using BridgeStan as a package now works fine. Not sure how difficult it is to use this in Pluto as BridgeStan.jl is not registered. For now I will test the "borrowing" stanc route next. |
|
The 1st option ("borrowing STAN and STANC from cmdstan) fails with: For the 2nd option, using BridgeStan.jl, I went back to the vanilla Julia example (example.jl) and attempted to get that to work in the StanSample.jl setting using a local copy of BridgeStan (in debs/data/bridgestan). That works fine on my Mac. But I got stuck on CI and also on AutoMerge:
All in all, my current conclusion is that an individual Julia user can certainly use a locally installed BridgeStan repo but including it in StanSample.jl is tricky and certainly difficult to maintain with my limited Github workflow and make knowledge. As I want to go on with my project work, for now I'll remove BridgeStan support from StanSample.jl. But not giving up yet! |
That error definitely has the flavor of "
That seems so strange to me, since we're downloading it from stan-dev/stanc3, not this repo, so we shouldn't have any "special treatment" as it were. Is it possibly being downloaded but not in the right place? Have you tried printing out a
Very glad to hear that it is working for end users, still. I think the best way forward would likely be treating it similarly to CmdStan - up to the user to install, rather than trying to ship it as part of a Julia package. I'm not too familiar with Julia, is there some way of checking if another package is installed at run time, and adding functionality to your own if so? In Python you can do this with something like try:
import bridgestan
BRIDGESTAN_INSTALLED=True
except ImportError:
BRIDGESTAN_INSTALLED=False |
|
It seems like the Julia equivalent is available using something like Requires.jl. I think that is probably better than a submodule approach, all things considered. I'm going to mark this as |
|
Yes, on your last suggestion, there is such an option using Requires.jl. I’ll check how that works with unregistered package.I’ll get a complete list of files downloaded but on my Mac it definitely ends up in the right bin directory.Sent from my iPhoneOn Dec 13, 2022, at 11:39, Brian Ward ***@***.***> wrote:
The 1st option ("borrowing STAN and STANC from cmdstan) fails with:
That error definitely has the flavor of "make thinks it is being run somewhere it is not so it's passing a nonsense include flag". Certainly possible to debug (running make print-STAN etc and making sure that directory is really the one you want) but make definitely is frustrating for things like this
Download of stanc definitely does not seem to happen or maybe is even not allowed. Maybe that is ok when testing within the repo but not across repos?
That seems so strange to me, since we're downloading it from stan-dev/stanc3, not this repo, so we shouldn't have any "special treatment" as it were. Is it possibly being downloaded but not in the right place? Have you tried printing out a ls -R of the entire folder structure?
All in all, my current conclusion is that an individual Julia user can certainly use a locally installed BridgeStan repo but including it in StanSample.jl is tricky and certainly difficult to maintain with my limited Github workflow and make knowledge.
Very glad to hear that it is working for end users, still. I think the best way forward would likely be treating it similarly to CmdStan - up to the user to install, rather than trying to ship it as part of a Julia package. I'm not too familiar with Julia, is there some way of checking if another package is installed at run time, and adding functionality to your own if so? In Python you can do this with something like
try:
import bridgestan
BRIDGESTAN_INSTALLED=True
except ImportError:
BRIDGESTAN_INSTALLED=False
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***>
|
Experimenting with BridgeStan v1.0 I have no problem on my Mac (M2 currently). But testing on Github CI produces the error
./bin/stanc: 1: Not: not found, hence my question.StanSample.jl, in subdirectory
deps/data/bridgestan, contains a full clone of the BridgeStan repo. (git clone --recurse-submodules https://github.com/roualdes/bridgestan.git).On the Mac
./bin/stancis available indebs/data/bridgestan, but not in the repo on Github..gitignoreindebs/data/bridgestandoes include an entrybin/*.I've also tried different ways of "installing" BridgeStan the normal way in Julia, but that runs into path problems, e.g. in example.jl:
I haven't tried treating BridgeStan similar to cmdstan, i.e. installing it outside of Julia, but that is a major step back from how BridgeStan worked pre-v1.0.
I must be overlooking an option on how to include BridgeStan in another package like StanSample. Any hints would be appreciated.
This is the first part of the CI output (step 8).
The text was updated successfully, but these errors were encountered: