Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Variants #1

Open
wants to merge 29 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
30a5e75
wip: Variants
Mike-Dax Jul 12, 2021
b0b0b10
bug: Fixed scheduling incorrect descriptor resolution
Mike-Dax Jul 12, 2021
6c476af
wip: Replace packages on variants found
Mike-Dax Jul 12, 2021
ec03740
chore: Added variants to all resolvers
Mike-Dax Jul 12, 2021
0c7b9b5
test: Variant Utility tests
Mike-Dax Jul 12, 2021
b95e473
wip: Custom comparators for variant parameters
Mike-Dax Jul 12, 2021
4732b34
refactor: Allow dependenciesMeta to change variant parameters
Mike-Dax Jul 13, 2021
3f1c0bb
feat: PackageExtensions support Variant information
Mike-Dax Jul 13, 2021
5752541
wip: Added untyped configuration for Variants
Mike-Dax Jul 13, 2021
6d00881
chore: Don't overwrite Manifest if variants is null
Mike-Dax Jul 13, 2021
bad1289
bug: Compare the correct locator when replacing packages
Mike-Dax Jul 13, 2021
ca8089f
bug: Register the old package
Mike-Dax Jul 13, 2021
9d933da
refactor: Removed candidates key
Mike-Dax Jul 14, 2021
8b9da9a
feat: Cache parameters for variants
Mike-Dax Jul 14, 2021
eb3c338
feat: Explicit includes for variants
Mike-Dax Jul 14, 2021
c4086a9
feat: Reach into the parent package and delete the old dependency
Mike-Dax Jul 14, 2021
d476c04
refactor: Replace the descriptor hashes with our new dependency
Mike-Dax Jul 14, 2021
f7b71af
test: Added variant fixtures
Mike-Dax Aug 23, 2021
d68e502
test: Initial variant integration tests
Mike-Dax Aug 23, 2021
d694886
feat: Workspace getDependencyMeta method
Mike-Dax Aug 23, 2021
11019ec
test: Fixed variantUtil unit tests
Mike-Dax Aug 23, 2021
9e45dfe
feat: Download variants
Mike-Dax Aug 23, 2021
b8e2f6e
chore: Minor cleanup
Mike-Dax Aug 23, 2021
2eb91d1
chore: Bind descriptor
Mike-Dax Aug 24, 2021
b761a9d
chore: Cleanup of log spam
Mike-Dax Aug 24, 2021
df6530c
chore: Added temporary test case into the repo
Mike-Dax Aug 24, 2021
0967f6d
chore: Use variant-fallback test package
Mike-Dax Aug 24, 2021
f8f0961
refactor: Use VariantRemapResolver instead of npm one
Mike-Dax Aug 24, 2021
5446f9e
chore: Fixed ordering of debug log
Mike-Dax Aug 24, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .yarnrc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,3 +44,8 @@ preferInteractive: true
telemetryUserId: yarnpkg/berry

yarnPath: scripts/run-yarn.js

cacheParameters:
matrix:
platform: [darwin, linux, windows, freebsd, openbsd]
arch: [32, 64, arm, arm64, mips64le, ppc64le]
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
/* @flow */

module.exports = require(`./package.json`);

for (const key of [`dependencies`, `devDependencies`, `peerDependencies`]) {
for (const dep of Object.keys(module.exports[key] || {})) {
// $FlowFixMe The whole point of this file is to be dynamic
module.exports[key][dep] = require(dep);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
{
"name": "variants",
"version": "1.0.0",
"variants": [
{
"pattern": "variants-%par1-%par2@%version",
"matrix": {
"par1": [
"a",
"b",
"c"
],
"par2": [
1,
2
]
},
"exclude": [
{
"par1": "c",
"par2": 1
}
]
},
{
"pattern": "variants-%par1@%version",
"matrix": {
"par1": [
"d"
]
}
},
{
"pattern": "variants-fallback@%version"
}
]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
/* @flow */

module.exports = require(`./package.json`);

for (const key of [`dependencies`, `devDependencies`, `peerDependencies`]) {
for (const dep of Object.keys(module.exports[key] || {})) {
// $FlowFixMe The whole point of this file is to be dynamic
module.exports[key][dep] = require(dep);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"name": "variants-a-1",
"version": "1.0.0"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
/* @flow */

module.exports = require(`./package.json`);

for (const key of [`dependencies`, `devDependencies`, `peerDependencies`]) {
for (const dep of Object.keys(module.exports[key] || {})) {
// $FlowFixMe The whole point of this file is to be dynamic
module.exports[key][dep] = require(dep);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"name": "variants-a-2",
"version": "1.0.0"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
/* @flow */

module.exports = require(`./package.json`);

for (const key of [`dependencies`, `devDependencies`, `peerDependencies`]) {
for (const dep of Object.keys(module.exports[key] || {})) {
// $FlowFixMe The whole point of this file is to be dynamic
module.exports[key][dep] = require(dep);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"name": "variants-b-1",
"version": "1.0.0"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
/* @flow */

module.exports = require(`./package.json`);

for (const key of [`dependencies`, `devDependencies`, `peerDependencies`]) {
for (const dep of Object.keys(module.exports[key] || {})) {
// $FlowFixMe The whole point of this file is to be dynamic
module.exports[key][dep] = require(dep);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"name": "variants-b-2",
"version": "1.0.0"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
/* @flow */

module.exports = require(`./package.json`);

for (const key of [`dependencies`, `devDependencies`, `peerDependencies`]) {
for (const dep of Object.keys(module.exports[key] || {})) {
// $FlowFixMe The whole point of this file is to be dynamic
module.exports[key][dep] = require(dep);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"name": "variants-c-2",
"version": "1.0.0"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
/* @flow */

module.exports = require(`./package.json`);

for (const key of [`dependencies`, `devDependencies`, `peerDependencies`]) {
for (const dep of Object.keys(module.exports[key] || {})) {
// $FlowFixMe The whole point of this file is to be dynamic
module.exports[key][dep] = require(dep);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"name": "variants-d",
"version": "1.0.0"
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
/* @flow */

module.exports = require(`./package.json`);

for (const key of [`dependencies`, `devDependencies`, `peerDependencies`]) {
for (const dep of Object.keys(module.exports[key] || {})) {
// $FlowFixMe The whole point of this file is to be dynamic
module.exports[key][dep] = require(dep);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"name": "variants-fallback",
"version": "1.0.0"
}
88 changes: 88 additions & 0 deletions packages/acceptance-tests/pkg-tests-specs/sources/variants.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
import {PortablePath, xfs} from '@yarnpkg/fslib';
import {yarn} from 'pkg-tests-core';

const {writeConfiguration} = yarn;

describe(`Variant tests`, () => {
test(
`it should install the fallback dependency when no matching pattern can be found`,
makeTemporaryEnv(
{
dependencies: {[`variants`]: `1.0.0`},
},
async ({run, source}) => {
await run(`install`);

await expect(source(`require('variants')`)).resolves.toMatchObject({
name: `variants-fallback`,
version: `1.0.0`,
});
},
),
);

test(
`it should correctly install a dependency matching parameters set in the dependenciesMeta`,
makeTemporaryEnv(
{
dependencies: {[`variants`]: `1.0.0`},
dependenciesMeta: {
[`variants`]: {
parameters: {
par1: `a`,
par2: 1,
},
},
},
},
async ({run, source}) => {
await run(`install`);

await expect(source(`require('variants')`)).resolves.toMatchObject({
name: `variants-a-1`,
version: `1.0.0`,
});
},
),
);

test(
`it should fetch the matrix of cached variants set in the yarnrc`,
makeTemporaryEnv(
{
dependencies: {[`variants`]: `1.0.0`},
dependenciesMeta: {
[`variants`]: {
parameters: {
par1: `a`,
par2: 1,
},
},
},
},
async ({path, run}) => {
await writeConfiguration(path, {
cacheParameters: {
matrix: {
par1: [`a`, `b`, `c`],
par2: [1],
},
},
});

await run(`install`);


const cache = await xfs.readdirPromise(`${path}/.yarn/cache` as PortablePath);

const variantA1 = cache.find(file => file.startsWith(`variants-a-1-npm-1.0.0`));
const variantB1 = cache.find(file => file.startsWith(`variants-b-1-npm-1.0.0`));
const variantC1 = cache.find(file => file.startsWith(`variants-c-1-npm-1.0.0`));

expect(variantA1).toBeDefined();
expect(variantB1).toBeDefined();
expect(variantC1).not.toBeDefined(); // C1 is excluded by the variants package
},
),
);
});
1 change: 1 addition & 0 deletions packages/plugin-exec/sources/ExecResolver.ts
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@ export class ExecResolver implements Resolver {
peerDependenciesMeta: manifest.peerDependenciesMeta,

bin: manifest.bin,
variants: manifest.variants,
};
}
}
1 change: 1 addition & 0 deletions packages/plugin-file/sources/FileResolver.ts
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,7 @@ export class FileResolver implements Resolver {
peerDependenciesMeta: manifest.peerDependenciesMeta,

bin: manifest.bin,
variants: manifest.variants,
};
}
}
1 change: 1 addition & 0 deletions packages/plugin-file/sources/TarballFileResolver.ts
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,7 @@ export class TarballFileResolver implements Resolver {
peerDependenciesMeta: manifest.peerDependenciesMeta,

bin: manifest.bin,
variants: manifest.variants,
};
}
}
1 change: 1 addition & 0 deletions packages/plugin-git/sources/GitResolver.ts
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ export class GitResolver implements Resolver {
peerDependenciesMeta: manifest.peerDependenciesMeta,

bin: manifest.bin,
variants: manifest.variants,
};
}
}
1 change: 1 addition & 0 deletions packages/plugin-http/sources/TarballHttpResolver.ts
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ export class TarballHttpResolver implements Resolver {
peerDependenciesMeta: manifest.peerDependenciesMeta,

bin: manifest.bin,
variants: manifest.variants,
};
}
}
1 change: 1 addition & 0 deletions packages/plugin-link/sources/LinkResolver.ts
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ export class LinkResolver implements Resolver {
peerDependenciesMeta: manifest.peerDependenciesMeta,

bin: manifest.bin,
variants: manifest.variants,
};
}
}
1 change: 1 addition & 0 deletions packages/plugin-link/sources/RawLinkResolver.ts
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ export class RawLinkResolver implements Resolver {
peerDependenciesMeta: new Map(),

bin: new Map(),
variants: null,
};
}
}
3 changes: 3 additions & 0 deletions packages/plugin-npm/sources/NpmSemverResolver.ts
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,7 @@ export class NpmSemverResolver implements Resolver {
if (typeof manifest.raw.deprecated === `string`)
opts.report.reportWarningOnce(MessageName.DEPRECATED_PACKAGE, `${structUtils.prettyLocator(opts.project.configuration, locator)} is deprecated: ${manifest.raw.deprecated}`);


return {
...locator,

Expand All @@ -165,6 +166,8 @@ export class NpmSemverResolver implements Resolver {
peerDependenciesMeta: manifest.peerDependenciesMeta,

bin: manifest.bin,

variants: manifest.variants,
};
}
}
10 changes: 10 additions & 0 deletions packages/variant-fallback/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
/* @flow */

module.exports = require(`./package.json`);

for (const key of [`dependencies`, `devDependencies`, `peerDependencies`]) {
for (const dep of Object.keys(module.exports[key] || {})) {
// $FlowFixMe The whole point of this file is to be dynamic
module.exports[key][dep] = require(dep);
}
}
4 changes: 4 additions & 0 deletions packages/variant-fallback/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"name": "variant-fallback",
"version": "0.12.22"
}
Comment on lines +1 to +4
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you accidentally put this package in the wrong folder (it should be in the acceptance tests folder)

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was testing just 'in the yarn repo' since the iteration speed was faster, but I'll clean this up before the actual PR.

10 changes: 10 additions & 0 deletions packages/variant-test/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
/* @flow */

module.exports = require(`./package.json`);

for (const key of [`dependencies`, `devDependencies`, `peerDependencies`]) {
for (const dep of Object.keys(module.exports[key] || {})) {
// $FlowFixMe The whole point of this file is to be dynamic
module.exports[key][dep] = require(dep);
}
}
Loading