Use of source-repository-packages/cabal.project in CI #91
Comments
|
Long term I think what you want to do here is very much the *right* thing
to do.
I've been wrestling with trying to find a workflow to juggle several
packages at once with consistent versioning, during a stint while I'm much
more actively shuffling modules between them than usual.
The submodule with no cabal.project approach means I can just dip into each
sub-dir, spin up ghcid, fix something, and then see the impact on all the
downstream packages I'm working on at the same time in the outer
'distributive' project, committing the submodule then locks it in making it
easy to share with Aaron Vargo who has been working with me during this
bit. Unless, and I've never tested it, I can do a source-repository link
that doesn't tag the hash code and just names the branch?
Moving to vendored links as I currently use them means that I need to
commit the sub-project, go grab the version number, punch it into the
source-repository package link for each of the downstream packages' cabal
files that have their own cabal.project files and kind of blows up the work
to O(n^2).
I temporarily removed the cabal.project file because it actively gets in
the way of the above dip in, ghcid, work, dip out workflow, as then it
selects the contents of the local cabal.project file, so I have to run
ghcid from outside the project dir manually with a massive pile of flags to
tell it what library I want and to keep all of its no-code-gen
optimizations intact or risk screwing up and building with inconsistent
versions.
I'm about to do ~50 of these, so n here is going to get pretty large. I'm
inclined to slowly do this transform once I have stabilized what is moving
in/out, but knowingly break this in the interim.
…-Edward
On Wed, Apr 7, 2021 at 5:29 AM Ryan Scott ***@***.***> wrote:
I recently committed some fixes to another branch
<https://github.com/ekmett/bifunctors/tree/fix-ci> in an attempt to fix
the CI for bifunctors. I haven't pushed to master yet, however, because
it does a couple of opinionated things:
1. Instead of using submodules, like in the master branch of
distributive, it vendors in the comonad library using a
source-repository-package in the cabal.project file. See here
<e023b36#diff-b8ed06757045fac949c89f2139a862498ad2b6d1f82c61a31e7c91d6cf0eaa70R3>
.
2. Speaking of which, I'm using a cabal.project file. bifunctors used
to have one, but @ekmett <https://github.com/ekmett> appears to have
removed it in bc5d8e8
<bc5d8e8>
for reasons that aren't clear to me.
In any case, let me briefly explain why I favor (1) and (2) above:
1. Using source-repository-package allows vendored libraries to be
saved in cabal's store (using cabal-instal-3.4 or later), which means
that you don't have to constantly rebuild them. More critically, it also
means that vendored libraries can be cached in CI, so it will save quite a
bit of CI time in the long run.
2. Having a cabal.project file makes it so that I can check out the
bifunctors repo within the context of another project with a
cabal.project file and not have the latter cabal.project file clobber
it. This is important for my workflows, as I manage bifunctors within another
repo <https://github.com/RyanGlScott/ci-maintenance>.
@ekmett <https://github.com/ekmett>: I'm raising this issue because I
don't want to step on your toes when you're developing your libraries. Let
me know your preferences and I can find a way to make both of us happy.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#91>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AACKMELPDZMMGREXRRICHRTTHRFZLANCNFSM42QWHEJQ>
.
|
|
And of course right after I send that, I found that the
source-repository-package section of the cabal.project documentation is
incomplete and there _is_ a branch option.
If that works with git I'd be fine with a switch back to cabal.project
files with source-repository-package links.
…On Wed, Apr 7, 2021 at 8:27 AM Edward Kmett ***@***.***> wrote:
Long term I think what you want to do here is very much the *right* thing
to do.
I've been wrestling with trying to find a workflow to juggle several
packages at once with consistent versioning, during a stint while I'm much
more actively shuffling modules between them than usual.
The submodule with no cabal.project approach means I can just dip into
each sub-dir, spin up ghcid, fix something, and then see the impact on all
the downstream packages I'm working on at the same time in the outer
'distributive' project, committing the submodule then locks it in making it
easy to share with Aaron Vargo who has been working with me during this
bit. Unless, and I've never tested it, I can do a source-repository link
that doesn't tag the hash code and just names the branch?
Moving to vendored links as I currently use them means that I need to
commit the sub-project, go grab the version number, punch it into the
source-repository package link for each of the downstream packages' cabal
files that have their own cabal.project files and kind of blows up the work
to O(n^2).
I temporarily removed the cabal.project file because it actively gets in
the way of the above dip in, ghcid, work, dip out workflow, as then it
selects the contents of the local cabal.project file, so I have to run
ghcid from outside the project dir manually with a massive pile of flags to
tell it what library I want and to keep all of its no-code-gen
optimizations intact or risk screwing up and building with inconsistent
versions.
I'm about to do ~50 of these, so n here is going to get pretty large. I'm
inclined to slowly do this transform once I have stabilized what is moving
in/out, but knowingly break this in the interim.
-Edward
On Wed, Apr 7, 2021 at 5:29 AM Ryan Scott ***@***.***>
wrote:
> I recently committed some fixes to another branch
> <https://github.com/ekmett/bifunctors/tree/fix-ci> in an attempt to fix
> the CI for bifunctors. I haven't pushed to master yet, however, because
> it does a couple of opinionated things:
>
> 1. Instead of using submodules, like in the master branch of
> distributive, it vendors in the comonad library using a
> source-repository-package in the cabal.project file. See here
> <e023b36#diff-b8ed06757045fac949c89f2139a862498ad2b6d1f82c61a31e7c91d6cf0eaa70R3>
> .
> 2. Speaking of which, I'm using a cabal.project file. bifunctors used
> to have one, but @ekmett <https://github.com/ekmett> appears to have
> removed it in bc5d8e8
> <bc5d8e8>
> for reasons that aren't clear to me.
>
> In any case, let me briefly explain why I favor (1) and (2) above:
>
> 1. Using source-repository-package allows vendored libraries to be
> saved in cabal's store (using cabal-instal-3.4 or later), which means
> that you don't have to constantly rebuild them. More critically, it also
> means that vendored libraries can be cached in CI, so it will save quite a
> bit of CI time in the long run.
> 2. Having a cabal.project file makes it so that I can check out the
> bifunctors repo within the context of another project with a
> cabal.project file and not have the latter cabal.project file clobber
> it. This is important for my workflows, as I manage bifunctors within another
> repo <https://github.com/RyanGlScott/ci-maintenance>.
>
> @ekmett <https://github.com/ekmett>: I'm raising this issue because I
> don't want to step on your toes when you're developing your libraries. Let
> me know your preferences and I can find a way to make both of us happy.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <#91>, or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AACKMELPDZMMGREXRRICHRTTHRFZLANCNFSM42QWHEJQ>
> .
>
|
|
Trying it with cabal.project using branch based source links seems to work and should potentially fix ci without undue tag # maintenance. |
|
Thanks for the very thorough response, @ekmett! I also wish that the ergonomics of Using the |
|
I've been switching each from master to main for consistency. In theory
bifunctors should already be flipped over.
Let me know if I fudged it up.
One concern I'm noting is that this workflow seems pretty brittle. When I
go to do rebuilds after changes to the upstream branch, cabal gets
confused, and fails to pull the updated branch correctly, and because each
of these is getting a mini cabal.project file, i'm getting lots of
dist-newstyle folders, meaning lots of redundant build work in my dev
cycle. Neither of these is great for me, so I may recant using this style,
but I'll at least try it for a few days first.
If it is untenable in practice, I could maybe see about using some tooling
to handle the source-repository link management, by doing something like
generating `cabal.project.local`s via tooling or something. There should be
something in this corner of the design space that works without breaking CI
at any rate.
…-Edward
-Edward
On Wed, Apr 7, 2021 at 3:44 PM Ryan Scott ***@***.***> wrote:
Thanks for the very thorough response, @ekmett <https://github.com/ekmett>!
I also wish that the ergonomics of cabal.project allowed switching
between source-repository-packages and submodules more easily. That being
said, if using tag: <branch-name> works for you, then perhaps that's the
best approach for now.
Using the bifunctors repo as a particular example, should I vendor in
comonad's master branch as its tag?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#91 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AACKMEOU724SZ43TY4NWORLTHTN53ANCNFSM42QWHEJQ>
.
|
Good to know. If it is untenable, I'm not opposed to using submodules. The only catch is that I don't think
That is an interesting idea. We could certainly check in two |
|
Ah, I didn't notice that you already switched to using a |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I recently committed some fixes to another branch in an attempt to fix the CI for
bifunctors. I haven't pushed tomasteryet, however, because it does a couple of opinionated things:masterbranch ofdistributive, it vendors in thecomonadlibrary using asource-repository-packagein thecabal.projectfile. See here.cabal.projectfile.bifunctorsused to have one, but @ekmett appears to have removed it in bc5d8e8 for reasons that aren't clear to me.In any case, let me briefly explain why I favor (1) and (2) above:
source-repository-packageallows vendored libraries to be saved incabal's store (usingcabal-instal-3.4or later), which means that you don't have to constantly rebuild them. More critically, it also means that vendored libraries can be cached in CI, so it will save quite a bit of CI time in the long run.cabal.projectfile makes it so that I can check out thebifunctorsrepo within the context of another project with acabal.projectfile and not have the lattercabal.projectfile clobber it. This is important for my workflows, as I managebifunctorswithin another repo.@ekmett: I'm raising this issue because I don't want to step on your toes when you're developing your libraries. Let me know your preferences and I can find a way to make both of us happy.
The text was updated successfully, but these errors were encountered: