Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Juvix package lockfile support #2334

Closed
paulcadman opened this issue Sep 1, 2023 · 1 comment
Closed

Add Juvix package lockfile support #2334

paulcadman opened this issue Sep 1, 2023 · 1 comment
Assignees
Milestone

Comments

@paulcadman
Copy link
Collaborator

paulcadman commented Sep 1, 2023

After support for external dependencies #2272 we need to add support for a package dependency lockfile

A lockfile is generated from the package file during the compile pipeline. It contains all the information required to reproduce the full dependency source tree.

In other words is walks through the dependency tree and records the immutable version associated with each dependency spec.

If it exists, the juvix compiler uses the lockfile to resolve dependencies instead of using the dependencies specs in the juvix package file.

Example:

package A

name: A
dependencies:
  - .juvix-build/stdlib
  - git:
      url: https://my.dep1.b
      name: B
      ref: v1.0.0
version: 0.1.0

package B

name: B
dependencies:
  - .juvix-build/stdlib
  - git:
      url: https://my.dep1.c
      name: C
      ref: v2,0,0
version: 1.0.0

package C

name: C
version: 2.0.0

A lockfile for package A (format to be determined)

dependencies:
    - .juvix-build/stdlib
    -  git:
          url: https://my.dep1.c
          ref: commit-hash-b
          name: B
          dependencies:
              - .juvix-build/stdlib
              -  git:
                    url: https://my.dep1.c
                    ref: commit-hash-c
                    name: C
                    dependencies: []

NB: The tag refs (v1.0.0) are resolved to immutable git hashes in the lockfile.

@paulcadman paulcadman added this to the 0.5 milestone Sep 1, 2023
@lukaszcz lukaszcz modified the milestones: 0.5.0, 0.5.2 Sep 15, 2023
@paulcadman paulcadman self-assigned this Sep 22, 2023
jonaprieto pushed a commit that referenced this issue Oct 2, 2023
This PR adds lock file support to the compiler pipeline. The lock file
is generated whenever a compiler pipeline command (`juvix {compile,
typecheck, repl}`) is run.

The lock file contains all the information necessary to reproduce the
whole dependency source tree. In particular for git dependencies,
branch/tag references are resolved to git hash references.

## Lock file format

The lock file is a YAML `juvix.lock.yaml` file written by the compiler
alongside the package's `juvix.yaml` file.

```
LOCKFILE_SPEC: { dependencies: { DEPENDENCY_SPEC, dependencies: LOCKFILE_SPEC }
DEPENDENCY_SPEC: PATH_SPEC | GIT_SPEC
PATH_SPEC: { path: String }
GIT_SPEC: { git: {url: String, ref: String, name: String } }
```

## Example

Consider a project containing the following `juvix.yaml`:

```yaml
dependencies:
- .juvix-build/stdlib/
- git:
   url: https://github.com/anoma/juvix-containers
   ref: v0.7.1
   name: containers
name: example
version: 1.0.0
```

After running `juvix compile` the following lockfile `juvix.lock.yaml`
is generated.

```yaml
# This file was autogenerated by Juvix version 0.5.1.
# Do not edit this file manually.

dependencies:
- path: .juvix-build/stdlib/
  dependencies: []
- git:
    name: containers
    ref: 3debbc7f5776924eb9652731b3c1982a2ee0ff24
    url: https://github.com/anoma/juvix-containers
  dependencies:
  - git:
      name: stdlib
      ref: 4facf14d9b2d06b81ce1be1882aa9050f768cb45
      url: https://github.com/anoma/juvix-stdlib
    dependencies: []
  - git:
      name: test
      ref: a7ac74cac0db92e0b5e349f279d797c3788cdfdd
      url: https://github.com/anoma/juvix-test
    dependencies:
    - git:
        name: stdlib
        ref: 4facf14d9b2d06b81ce1be1882aa9050f768cb45
        url: https://github.com/anoma/juvix-stdlib
      dependencies: []
```

For subsequent runs of the juvix compile pipeline, the lock file
dependency information is used.

 ## Behaviour when package file and lock file are out of sync

If a dependency is specified in `juvix.yaml` that is not present in the
lock file, an error is raised.

Continuing the example above, say we add an additional dependency:

```
dependencies:
- .juvix-build/stdlib/
- git:
     url: https://github.com/anoma/juvix-containers
     ref: v0.7.1
     name: containers
- git:
     url: https://github.com/anoma/juvix-test
     ref: v0.6.1
     name: test
name: example
version: 1.0.0
```

`juvix compile` will throw an error:

```
/Users/paul/tmp/lockfile/dep/juvix.yaml:1:1: error:
The dependency test is declared in the package's juvix.yaml but is not declared in the lockfile: /Users/paul/tmp/lockfile/dep/juvix.lock.json
Try removing /Users/paul/tmp/lockfile/dep/juvix.lock.yaml and then run Juvix again.
```

Closes:
* #2334
@jonaprieto jonaprieto modified the milestones: 0.5.2, 0.5.3 Oct 2, 2023
@paulcadman paulcadman modified the milestones: 0.5.3, 0.5.2 Oct 4, 2023
@paulcadman
Copy link
Collaborator Author

Closed by #2388

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants