Automatically find cabal installed imports

[bergmark]

For fay-dom and fay-jquery it would be nice if the files could be automatically included on-demand, it should be possible with Paths modules and getDataFileName. Perhaps there's a better solution that doesn't create a dependency on specific packages?

Another solution would be to just use cabal for this, abstract the cabal hook in fay-server, that way deps can be listed in the .cabal.

[chrisdone]

Yeah, I think that approach is pretty workable. fay-jquery.cabal:

package: fay-jquery
data-files: Language/Fay/JQuery.hs
library
 exposed-modules: Language.Fay.JQuery
 depends: fay

then

$ cabal install fay-jquery

and then when we use it,

fay -package fay-jquery foo.hs

we can use the Cabal API in Fay to get the .hs source files. That's what you meant right? I did plan this but forgot to make a ticket.

[chrisdone]

Also --make if we're feeling clever with our Cabal API skills.

[chrisdone]

Ah, right, yeah. You're also talking about the Setup.hs file.

We can also use the dependenices passed to the hook to tell fay which packages to use. This would use the aforementioned feature. I suppose the only thing we don't have is support from Cabal to use the Fay compiler as opposed to GHC, UHC, etc. So our Setup.hs emulates that well enough. If we put the code inside Fay and then just import Language.Fay.Setup it will be much more concise and feel less finicky.

We're talking about the same stuff right?

[bergmark]

yup! sounds good!

[techtangents]

Some interesting cabal install options:

-w --with-compiler=PATH give the path to a particular compiler
--with-hc-pkg=PATH give the path to the package tool

So, via command line options, we can tell cabal to use fay, and, perhaps, a fay package manager. A fay package manager would be cool - this gives us a repository of cabal packages that are compiled with fay to javascript.

I assume cabal uses the hc-pkg program as input to dependency resolution.

Then, for convenience, we might want a wrapper around cabal install - say "fabal" - that fills in these two options.

[techtangents]

Note: to be clear, I want to be able to make use of the packages on Hackage, not just ones that are built specifically for fay.

We might want some kind of database of what packages are known good in Fay (whitelist or blacklist).

[chrisdone]

Note: to be clear, I want to be able to make use of the packages on Hackage, not just ones that are built specifically for fay.

Hm, let's be realistic about that. Most won't build.

The command line options are interesting but can we specify them in the cabal file?

[techtangents]

Really? I think that's rather pessimistic. There is a huge wealth of code on hackage and it would be a shame to give up on it. Libraries based on type classes or data structures would be very useful. I don't want to have to reimplement things like trees, zippers, state. As long as there's no IO, it should just be a matter of supporting enough language features.

[chrisdone]

It's not giving up, it's merely a technical fact. Most libraries use type-classes and GHC extensions, we don't have the former and few of the latter. The most positive advice I can give you is to try GHCJS or Haste, as you will be disappointed with Fay. That's not what the project is about. If you need so many libraries for your client side, probably you're not utilizing enough of the JS ecosystem.

[meteficha]

Just for the record: using Cabal would be great! =)

[cdsmith]

@techtangents I agree there might be a lot of Fay-compatible code on Haskell (though perhaps less than you'd think...). The problem is that there's no good way to know if it is or not, and just rewriting the dependencies behind the library's back is likely to put you in VERY fragile territory. The current plan does include a way to build pure libraries that can be easily used with both GHC and Fay. However, they need to be declared as such, by depending on a different restricted base package.

If you want to do the social work, then once cabal configs are in place, it might be worthwhile to pick the few packages with the most compelling case to work in both, and see if you can get the authors to add Cabal configuration to build them with fay. It wouldn't be too hard to write something with haskell-src-exts that scans through a downloaded hackage database and identifies candidates that only use whitelisted stuff from Prelude and avoid other dependencies and language extensions.

[techtangents]

I think that requiring a nugget in the cabal file to build with fay is silly. I would really like to avoid that, if possible

Getting a haskell package working in fay should be a matter of testing and contributing, or, worst case, forking. A manually maintained whitelist of fay-compatible packages would be useful.

Ultimately, fay is just a haskell compiler. Other haskell compilers have the same issues as fay in not supporting all of haskell + ghc exts. Do we have cabal hooks in every package that's compatible with uhc? Hugs?

Cabal supports using arbitrary haskell compilers - lets make use of that.

[cdsmith]

@techtangents There are two issues. One is that Fay isn't all of Haskell and doesn't intend to be, so many of those packages aren't guaranteed to build for language support reasons. This problem is pretty much endemic to wanting to build Haskell code with Fay... so the only real option is to just try and let the build fail sometimes. So suppose we decide to do that. If the build fails, you get to decide whether to try to convince Fay to support the language feature, or try to convince the library to not use it.

The MUCH bigger problem is that all the libraries you want to use depend on base, and base defines over 150 modules, including a C FFI, for example. So what are the options for solving this?

1. Try to get base to build with Fay. This is a LOT of work. It will require patching base, in a MUCH more intrusive way than the existing conditional compilation for GHC, Hugs, NHC, etc. You'd basically have to stub out the whole IO monad, which would be likely to touch a large percentage of the code. Not to mention JavaScript lacks efficient implementations for even basic data types like the various sized Int and Word types.
2. Transparently replace base with a replacement package that defines a subset of its features and is maintained separately as part of Fay. This has more promise... I'm not sure how it would fit with the Cabal stuff Chris has planned, but it's IMO a lot closer to realism than actually building base in JavaScript.

In any case, if option 2 is implemented, it should be optional, because you'll still never get around the problem that many packages will resolve but fail to build, and that library authors who don't intend to support Fay will frequently break your packages without a good fix. IMO, the default should be that Fay only builds packages depending on fay-base, and the package substitution stuff can perhaps be added as an optional "try it if you dare" sort of thing.

[techtangents]

I'm starting to think that the idea of a "standard library" is a bad idea in general. :)

Ok, so I'll concede that most things currently on hackage won't work. We can still:

• contribute
• fork
• write new modules

Either way, we're going to end up with a set of packages that Fay can support. So, we need to look at how we handle packages.

So, there's a few things to consider from here:

1. Hackage

   Do we want to use hackage as a repository for Fay packages? Hackage is open source and deployable elsewhere - my company runs their own internally, for instance. We could set up another Hackage instance specifically for Fay modules. Say we call it "fackage".

2. Cabal

   I really like the idea of having a wrapper around cabal, specifically for Fay. Say we call it "fabal".

• Fabal would have it's own config file that point to fackage for its packages.
• Fabal would install to its own repository of packages. It could have its own *hc-pkg program, or maybe use ghc-pkg in a different location.
• Fabal (obviously) would configure cabal to use fay as its compiler.
• Fabal could have extra options specifically for handling issues related to js code output - e.g. concatenating scripts, "linking", minification.

Then, if we find places where we want to diverge from how cabal works, we have a few 'hooks' in place where we can change things.

Basically, I think we should think of what Fay's toolchain would look like, and we can probably reuse a lot of haskell infrastructure to implement it.

[techtangents]

So, it looks like Haste takes the approach I'm talking about.
https://github.com/valderman/haste-compiler

They have a haste-pkg program that wraps ghc-pkg. There's a folder in the user's home dir set aside for haste packages. It looks like it uses a config file (like .cabal), rather than lots of command-line options. I think their bootstrap script creates this.

There's some weird stuff in there (unsafePerformIO for the lose!), but I think this approach could work for us. We could probably even reuse some of their code.

[techtangents]

Actually, from haste-inst.hs:

    ["--with-compiler=" ++ hasteBinary,
     "--with-hc-pkg=haste-pkg",
     "--with-hsc2hs=hsc2hs",
     "--prefix=" ++ hasteDir </> "haste-install"]

[chrisdone]

Precedes #140.

[chrisdone]

Added to Better Cabal support https://github.com/faylang/fay/issues/milestones

[nomeata]

Cabal supports (or once supported) hugs, which also needs source files around at compilation time. See http://cvs.haskell.org/Hugs/pages/users_guide/packages.html for what hugs expect. I’d say that Fay has the same requirements.

So it might be possible to teach Cabal to also support Fay in the sense that you can use cabal-install to install a package from hackage for fay, and instead of compiling it with ghc it will copy the sources to a location where Fay finds it. This would allow for installation of Fay specific packages via "cabal install --fay fay-jquery" as long as they don’t depend on base. And packages that should work with either can use the Cabal features to make the build-depends on base dependent on the current compiler.

[bergmark]

I'll close this now. We have working cabal packages. We can revisit if interesting Cabal patches come out or we start supporting type classes.