Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(core): re-enable running plugins in isolation #22527

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/generated/devkit/CreateNodes.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Type alias: CreateNodes\<T\>

Ƭ **CreateNodes**\<`T`\>: readonly [configFilePattern: string, createNodesFunction: CreateNodesFunction\<T\>]
Ƭ **CreateNodes**\<`T`\>: readonly [projectFilePattern: string, createNodesFunction: CreateNodesFunction\<T\>]

A pair of file patterns and [CreateNodesFunction](../../devkit/documents/CreateNodesFunction)

Expand Down
12 changes: 12 additions & 0 deletions docs/generated/devkit/ExpandedPluginConfiguration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Type alias: ExpandedPluginConfiguration

Ƭ **ExpandedPluginConfiguration**: `Object`

#### Type declaration

| Name | Type |
| :--------- | :--------- |
| `exclude?` | `string`[] |
| `include?` | `string`[] |
| `options?` | `unknown` |
| `plugin` | `string` |
2 changes: 1 addition & 1 deletion docs/generated/devkit/NxPluginV2.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,5 @@ A plugin for Nx which creates nodes and dependencies for the [ProjectGraph](../.
| Name | Type | Description |
| :-------------------- | :------------------------------------------------------------------------------ | :-------------------------------------------------------------------------------------------------------------------------------------------- |
| `createDependencies?` | [`CreateDependencies`](../../devkit/documents/CreateDependencies)\<`TOptions`\> | Provides a function to analyze files to create dependencies for the [ProjectGraph](../../devkit/documents/ProjectGraph) |
| `createNodes?` | [`CreateNodes`](../../devkit/documents/CreateNodes) | Provides a file pattern and function that retrieves configuration info from those files. e.g. { '\*_/_.csproj': buildProjectsFromCsProjFile } |
| `createNodes?` | [`CreateNodes`](../../devkit/documents/CreateNodes)\<`TOptions`\> | Provides a file pattern and function that retrieves configuration info from those files. e.g. { '\*_/_.csproj': buildProjectsFromCsProjFile } |
| `name` | `string` | - |
2 changes: 1 addition & 1 deletion docs/generated/devkit/PluginConfiguration.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
# Type alias: PluginConfiguration

Ƭ **PluginConfiguration**: `string` \| \{ `exclude?`: `string`[] ; `include?`: `string`[] ; `options?`: `unknown` ; `plugin`: `string` }
Ƭ **PluginConfiguration**: `string` \| [`ExpandedPluginConfiguration`](../../devkit/documents/ExpandedPluginConfiguration)
1 change: 1 addition & 0 deletions docs/generated/devkit/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@ It only uses language primitives and immutable objects
- [CustomHasher](../../devkit/documents/CustomHasher)
- [DynamicDependency](../../devkit/documents/DynamicDependency)
- [Executor](../../devkit/documents/Executor)
- [ExpandedPluginConfiguration](../../devkit/documents/ExpandedPluginConfiguration)
- [Generator](../../devkit/documents/Generator)
- [GeneratorCallback](../../devkit/documents/GeneratorCallback)
- [Hasher](../../devkit/documents/Hasher)
Expand Down
17 changes: 9 additions & 8 deletions docs/generated/devkit/logger.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,12 @@

#### Type declaration

| Name | Type |
| :------ | :-------------------------- |
| `debug` | (...`s`: `any`[]) => `void` |
| `error` | (`s`: `any`) => `void` |
| `fatal` | (...`s`: `any`[]) => `void` |
| `info` | (`s`: `any`) => `void` |
| `log` | (...`s`: `any`[]) => `void` |
| `warn` | (`s`: `any`) => `void` |
| Name | Type |
| :-------- | :-------------------------- |
| `debug` | (...`s`: `any`[]) => `void` |
| `error` | (`s`: `any`) => `void` |
| `fatal` | (...`s`: `any`[]) => `void` |
| `info` | (`s`: `any`) => `void` |
| `log` | (...`s`: `any`[]) => `void` |
| `verbose` | (...`s`: `any`[]) => `void` |
| `warn` | (`s`: `any`) => `void` |
1 change: 1 addition & 0 deletions docs/generated/packages/devkit/documents/nx_devkit.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@ It only uses language primitives and immutable objects
- [CustomHasher](../../devkit/documents/CustomHasher)
- [DynamicDependency](../../devkit/documents/DynamicDependency)
- [Executor](../../devkit/documents/Executor)
- [ExpandedPluginConfiguration](../../devkit/documents/ExpandedPluginConfiguration)
- [Generator](../../devkit/documents/Generator)
- [GeneratorCallback](../../devkit/documents/GeneratorCallback)
- [Hasher](../../devkit/documents/Hasher)
Expand Down
26 changes: 19 additions & 7 deletions packages/devkit/src/utils/convert-nx-executor.spec.ts
Original file line number Diff line number Diff line change
@@ -1,23 +1,35 @@
// When plugins from root nx.json load through ts-jest, they can cause transpile errors such as `@nx/playwright/plugin.d.ts` containing an unexpected "export" keyword.
// Mock `loadNxPlugins` function to prevent them from loading.
jest.mock('nx/src/utils/nx-plugin', () => ({
loadNxPlugins: () => Promise.resolve([]),
}));

import { TempFs } from '../../internal-testing-utils';
import { convertNxExecutor } from './convert-nx-executor';

describe('Convert Nx Executor', () => {
let fs: TempFs;

beforeAll(async () => {
fs = new TempFs('convert-nx-executor');
// The tests in this file don't actually care about the files in the temp dir.
// The converted executor reads project configuration from the workspace root,
// which is set to the temp dir in the tests. If there are no files in the temp
// dir, the glob search currently hangs. So we create a dummy file to prevent that.
await fs.createFile('blah.json', JSON.stringify({}));
});

afterAll(() => {
fs.cleanup();
});

it('should convertNxExecutor to builder correctly and produce the same output', async () => {
// ARRANGE
const { schema } = require('@angular-devkit/core');
const {
TestingArchitectHost,
} = require('@angular-devkit/architect/testing');
// nx-ignore-next-line
} = require('@angular-devkit/architect/testing') as typeof import('@angular-devkit/architect/testing');
const { Architect } = require('@angular-devkit/architect');

const registry = new schema.CoreSchemaRegistry();
registry.addPostTransform(schema.transforms.addUndefinedDefaults);
const testArchitectHost = new TestingArchitectHost();
testArchitectHost.workspaceRoot = fs.tempDir;
const architect = new Architect(testArchitectHost, registry);

const convertedExecutor = convertNxExecutor(echoExecutor);
Expand Down
13 changes: 8 additions & 5 deletions packages/nx/bin/post-install.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,10 @@ import { readNxJson } from '../src/config/nx-json';
import { setupWorkspaceContext } from '../src/utils/workspace-context';

(async () => {
const start = new Date();
try {
setupWorkspaceContext(workspaceRoot);
if (isMainNxPackage() && fileExists(join(workspaceRoot, 'nx.json'))) {
const b = new Date();
assertSupportedPlatform();

try {
Expand All @@ -35,15 +35,18 @@ import { setupWorkspaceContext } from '../src/utils/workspace-context';
});
})
);
if (process.env.NX_VERBOSE_LOGGING === 'true') {
const a = new Date();
console.log(`Nx postinstall steps took ${a.getTime() - b.getTime()}ms`);
}
}
} catch (e) {
if (process.env.NX_VERBOSE_LOGGING === 'true') {
console.log(e);
}
} finally {
if (process.env.NX_VERBOSE_LOGGING === 'true') {
const end = new Date();
console.log(
`Nx postinstall steps took ${end.getTime() - start.getTime()}ms`
);
}
}
})();

Expand Down
2 changes: 1 addition & 1 deletion packages/nx/plugins/package-json.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import type { NxPluginV2 } from '../src/utils/nx-plugin';
import type { NxPluginV2 } from '../src/project-graph/plugins';
import { workspaceRoot } from '../src/utils/workspace-root';
import { createNodeFromPackageJson } from '../src/plugins/package-json-workspaces';

Expand Down
4 changes: 3 additions & 1 deletion packages/nx/src/adapter/angular-json.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ import { existsSync } from 'fs';
import * as path from 'path';
import { readJsonFile } from '../utils/fileutils';
import { ProjectsConfigurations } from '../config/workspace-json-project-json';
import { NxPluginV2 } from '../utils/nx-plugin';
import { NxPluginV2 } from '../project-graph/plugins';

export const NX_ANGULAR_JSON_PLUGIN_NAME = 'nx-angular-json-plugin';

Expand All @@ -16,6 +16,8 @@ export const NxAngularJsonPlugin: NxPluginV2 = {
],
};

export default NxAngularJsonPlugin;

export function shouldMergeAngularProjects(
root: string,
includeProjectsFromAngularJson: boolean
Expand Down
2 changes: 1 addition & 1 deletion packages/nx/src/adapter/ngcli-adapter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ import {
ExecutorsJson,
TaskGraphExecutor,
} from '../config/misc-interfaces';
import { readPluginPackageJson } from '../utils/nx-plugin';
import { readPluginPackageJson } from '../project-graph/plugins';
import {
getImplementationFactory,
resolveImplementation,
Expand Down
2 changes: 1 addition & 1 deletion packages/nx/src/command-line/generate/generator-utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import {
resolveSchema,
} from '../../config/schema-utils';
import { readJsonFile } from '../../utils/fileutils';
import { readPluginPackageJson } from '../../utils/nx-plugin';
import { readPluginPackageJson } from '../../project-graph/plugins';

export function getGeneratorInformation(
collectionName: string,
Expand Down
2 changes: 1 addition & 1 deletion packages/nx/src/command-line/run/executor-utils.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { dirname, join } from 'path';

import { readPluginPackageJson } from '../../utils/nx-plugin';
import { readPluginPackageJson } from '../../project-graph/plugins';
import {
CustomHasher,
Executor,
Expand Down
16 changes: 8 additions & 8 deletions packages/nx/src/config/nx-json.ts
Original file line number Diff line number Diff line change
Expand Up @@ -436,14 +436,14 @@ export interface NxJsonConfiguration<T = '*' | string[]> {
useInferencePlugins?: boolean;
}

export type PluginConfiguration =
| string
| {
plugin: string;
options?: unknown;
include?: string[];
exclude?: string[];
};
export type PluginConfiguration = string | ExpandedPluginConfiguration;

export type ExpandedPluginConfiguration = {
plugin: string;
options?: unknown;
include?: string[];
exclude?: string[];
};

export function readNxJson(root: string = workspaceRoot): NxJsonConfiguration {
const nxJson = join(root, 'nx.json');
Expand Down
2 changes: 1 addition & 1 deletion packages/nx/src/config/schema-utils.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { existsSync } from 'fs';
import { extname, join } from 'path';
import { registerPluginTSTranspiler } from '../utils/nx-plugin';
import { registerPluginTSTranspiler } from '../project-graph/plugins';

/**
* This function is used to get the implementation factory of an executor or generator.
Expand Down
34 changes: 17 additions & 17 deletions packages/nx/src/config/workspaces.spec.ts
Original file line number Diff line number Diff line change
@@ -1,22 +1,9 @@
import { toProjectName, Workspaces } from './workspaces';
import { toProjectName } from './workspaces';
import { TempFs } from '../internal-testing-utils/temp-fs';
import { withEnvironmentVariables } from '../internal-testing-utils/with-environment';
import { retrieveProjectConfigurations } from '../project-graph/utils/retrieve-workspace-files';
import { readNxJson } from './configuration';

const libConfig = (root, name?: string) => ({
name: name ?? toProjectName(`${root}/some-file`),
projectType: 'library',
root: `libs/${root}`,
sourceRoot: `libs/${root}/src`,
targets: {
'nx-release-publish': {
dependsOn: ['^nx-release-publish'],
executor: '@nx/js:release-publish',
options: {},
},
},
});
import { loadNxPlugins } from '../project-graph/plugins/internal-api';

describe('Workspaces', () => {
let fs: TempFs;
Expand Down Expand Up @@ -48,10 +35,23 @@ describe('Workspaces', () => {

const { projects } = await withEnvironmentVariables(
{
NX_WORKSPACE_ROOT: fs.tempDir,
NX_WORKSPACE_ROOT_PATH: fs.tempDir,
},
() => retrieveProjectConfigurations(fs.tempDir, readNxJson(fs.tempDir))
async () => {
const [plugins, cleanup] = await loadNxPlugins(
readNxJson(fs.tempDir).plugins,
fs.tempDir
);
const res = retrieveProjectConfigurations(
plugins,
fs.tempDir,
readNxJson(fs.tempDir)
);
cleanup();
return res;
}
);
console.log(projects);
expect(projects['my-package']).toEqual({
name: 'my-package',
root: 'packages/my-package',
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ import { serializeResult } from '../socket-utils';
import { serverLogger } from './logger';
import { getCachedSerializedProjectGraphPromise } from './project-graph-incremental-recomputation';
import { HandlerResult } from './server';
import { getPlugins } from './plugins';
import { readNxJson } from '../../config/nx-json';

export async function handleRequestProjectGraph(): Promise<HandlerResult> {
try {
Expand Down
26 changes: 26 additions & 0 deletions packages/nx/src/daemon/server/plugins.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
import { readNxJson } from '../../config/nx-json';
import {
LoadedNxPlugin,
loadNxPlugins,
} from '../../project-graph/plugins/internal-api';
import { workspaceRoot } from '../../utils/workspace-root';

let loadedPlugins: Promise<LoadedNxPlugin[]>;
let cleanup: () => void;

export async function getPlugins() {
if (loadedPlugins) {
return loadedPlugins;
}
const pluginsConfiguration = readNxJson().plugins ?? [];
const [result, cleanupFn] = await loadNxPlugins(
pluginsConfiguration,
workspaceRoot
);
cleanup = cleanupFn;
return result;
}

export function cleanupPlugins() {
cleanup();
}
Original file line number Diff line number Diff line change
Expand Up @@ -32,11 +32,11 @@ import { workspaceRoot } from '../../utils/workspace-root';
import { notifyFileWatcherSockets } from './file-watching/file-watcher-sockets';
import { serverLogger } from './logger';
import { NxWorkspaceFilesExternals } from '../../native';
import {
ConfigurationResult,
ProjectConfigurationsError,
} from '../../project-graph/utils/project-configuration-utils';
import { ConfigurationResult } from '../../project-graph/utils/project-configuration-utils';
import { DaemonProjectGraphError } from '../daemon-project-graph-error';
import { LoadedNxPlugin } from '../../project-graph/plugins/internal-api';
import { getPlugins } from './plugins';
import { ProjectConfigurationsError } from '../../project-graph/error-types';

interface SerializedProjectGraph {
error: Error | null;
Expand Down Expand Up @@ -78,14 +78,15 @@ export async function getCachedSerializedProjectGraphPromise(): Promise<Serializ
// reset the wait time
waitPeriod = 100;
await resetInternalStateIfNxDepsMissing();
const plugins = await getPlugins();
if (collectedUpdatedFiles.size == 0 && collectedDeletedFiles.size == 0) {
if (!cachedSerializedProjectGraphPromise) {
cachedSerializedProjectGraphPromise =
processFilesAndCreateAndSerializeProjectGraph();
processFilesAndCreateAndSerializeProjectGraph(plugins);
}
} else {
cachedSerializedProjectGraphPromise =
processFilesAndCreateAndSerializeProjectGraph();
processFilesAndCreateAndSerializeProjectGraph(plugins);
}
return await cachedSerializedProjectGraphPromise;
} catch (e) {
Expand Down Expand Up @@ -133,7 +134,7 @@ export function addUpdatedAndDeletedFiles(
}

cachedSerializedProjectGraphPromise =
processFilesAndCreateAndSerializeProjectGraph();
processFilesAndCreateAndSerializeProjectGraph(await getPlugins());
await cachedSerializedProjectGraphPromise;

if (createdFiles.length > 0) {
Expand Down Expand Up @@ -209,7 +210,9 @@ async function processCollectedUpdatedAndDeletedFiles(
}
}

async function processFilesAndCreateAndSerializeProjectGraph(): Promise<SerializedProjectGraph> {
async function processFilesAndCreateAndSerializeProjectGraph(
plugins: LoadedNxPlugin[]
): Promise<SerializedProjectGraph> {
try {
performance.mark('hash-watched-changes-start');
const updatedFiles = [...collectedUpdatedFiles.values()];
Expand All @@ -227,14 +230,17 @@ async function processFilesAndCreateAndSerializeProjectGraph(): Promise<Serializ
serverLogger.requestLog([...updatedFiles.values()]);
serverLogger.requestLog([...deletedFiles]);
const nxJson = readNxJson(workspaceRoot);
// Set this globally to allow plugins to know if they are being called from the project graph creation
global.NX_GRAPH_CREATION = true;

let graphNodes: ConfigurationResult;
let projectConfigurationsError;

try {
graphNodes = await retrieveProjectConfigurations(workspaceRoot, nxJson);
graphNodes = await retrieveProjectConfigurations(
plugins,
workspaceRoot,
nxJson
);
} catch (e) {
if (e instanceof ProjectConfigurationsError) {
graphNodes = e.partialProjectConfigurationsResult;
Expand Down Expand Up @@ -335,7 +341,8 @@ async function createAndSerializeProjectGraph({
fileMap,
allWorkspaceFiles,
rustReferences,
currentProjectFileMapCache || readFileMapCache()
currentProjectFileMapCache || readFileMapCache(),
await getPlugins()
);

currentProjectFileMapCache = projectFileMapCache;
Expand Down
Loading