Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

auto install & import asset-packs for editor scenes #909

Merged
merged 3 commits into from
Mar 4, 2024
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
196 changes: 109 additions & 87 deletions packages/@dcl/inspector/package-lock.json

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion packages/@dcl/inspector/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"name": "@dcl/inspector",
"version": "0.1.0",
"dependencies": {
"@dcl/asset-packs": "^1.12.2",
"@dcl/asset-packs": "^1.12.3-20240301140405.commit-8aef784",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: make a asset-packs release and use that stable versioin

"ts-deepmerge": "^7.0.0"
},
"devDependencies": {
Expand Down
4 changes: 2 additions & 2 deletions packages/@dcl/sdk-commands/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

21 changes: 17 additions & 4 deletions packages/@dcl/sdk-commands/src/commands/build/index.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,14 @@
import path from 'path'
import { CliComponents } from '../../components'
import { declareArgs } from '../../logic/args'
import { installDependencies, needsDependencies, SceneProject, WearableProject } from '../../logic/project-validations'
import {
installAssetPack,
installDependencies,
isEditorScene,
needsDependencies,
SceneProject,
WearableProject
} from '../../logic/project-validations'
import { getBaseCoords } from '../../logic/scene-validations'
import { b64HashingFunction } from '../../logic/project-files'
import { bundleProject } from '../../logic/bundle'
Expand Down Expand Up @@ -57,10 +64,16 @@
}

export async function buildScene(options: Options, project: SceneProject | WearableProject) {
const shouldInstallDeps = await needsDependencies(options.components, project.workingDirectory)
const canInstall = !options.args['--skip-install']

if (shouldInstallDeps && !options.args['--skip-install']) {
await installDependencies(options.components, project.workingDirectory)
if (canInstall) {
if (await needsDependencies(options.components, project.workingDirectory)) {
await installDependencies(options.components, project.workingDirectory)
}

if (await isEditorScene(options.components, project.workingDirectory)) {
await installAssetPack(options.components, project.workingDirectory)

Check warning on line 75 in packages/@dcl/sdk-commands/src/commands/build/index.ts

View check run for this annotation

Codecov / codecov/patch

packages/@dcl/sdk-commands/src/commands/build/index.ts#L75

Added line #L75 was not covered by tests
}
}

const watch = !!options.args['--watch']
Expand Down
18 changes: 15 additions & 3 deletions packages/@dcl/sdk-commands/src/logic/bundle.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
import { printProgressInfo, printProgressStep, printWarning } from './beautiful-logs'
import { CliError } from './error'
import { getAllComposites } from './composite'
import { isEditorScene } from './project-validations'

export type BundleComponents = Pick<CliComponents, 'logger' | 'fs'>

Expand Down Expand Up @@ -47,7 +48,7 @@
* @returns the Typescript code
*/

function getEntrypointCode(entrypointPath: string, forceCustomExport: boolean) {
function getEntrypointCode(entrypointPath: string, forceCustomExport: boolean, isEditorScene: boolean = false) {
const unixEntrypointPath = entrypointPath.replace(/(\\)/g, '/')
if (forceCustomExport) return `;"use strict";export * from '${unixEntrypointPath}'`

Expand All @@ -59,6 +60,16 @@
import { compositeProvider } from '@dcl/sdk/composite-provider'
import { compositeFromLoader } from '~sdk/all-composites'

${
isEditorScene &&
`

Check warning on line 65 in packages/@dcl/sdk-commands/src/logic/bundle.ts

View check run for this annotation

Codecov / codecov/patch

packages/@dcl/sdk-commands/src/logic/bundle.ts#L65

Added line #L65 was not covered by tests
import { initAssetPacks } from '@dcl/asset-packs/dist/scene-entrypoint'
initAssetPacks(engine)

// TODO: do we need to do this on runtime ?
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move this to another issue

// I think we have that information at build-time and we avoid to do evaluate this on the worker.
// Read composite.json or main.crdt => If that file has a NetowrkEntity import '@dcl/@sdk/network'

// conditionally load networking code if the NetworkEntity component is being used...
for (const path in compositeFromLoader) {
const composite = compositeProvider.getCompositeOrNull(path)
Expand All @@ -69,6 +80,7 @@
import('@dcl/sdk/network')
}
}
}`
}

if ((entrypoint as any).main !== undefined) {
Expand Down Expand Up @@ -136,7 +148,7 @@

export async function bundleSingleProject(components: BundleComponents, options: SingleProjectOptions) {
printProgressStep(components.logger, `Bundling file ${colors.bold(options.entrypoint)}`, 1, MAX_STEP)

const editorScene = await isEditorScene(components, options.workingDirectory)
const context = await esbuild.context({
bundle: true,
platform: 'browser',
Expand Down Expand Up @@ -175,7 +187,7 @@
},
plugins: [compositeLoader(components, options)],
stdin: {
contents: getEntrypointCode(options.entrypoint, options.customEntryPoint),
contents: getEntrypointCode(options.entrypoint, options.customEntryPoint, editorScene),
resolveDir: path.dirname(options.entrypoint),
sourcefile: path.basename(options.entrypoint) + '.entry-point.ts',
loader: 'ts'
Expand Down
32 changes: 30 additions & 2 deletions packages/@dcl/sdk-commands/src/logic/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
import { CliComponents } from '../components'
import { readStringConfig } from '../components/config'
import { readJson } from './fs'
import { getPackageJson } from './project-files'

// eslint-disable-next-line @typescript-eslint/no-var-requires
const { version: sdkCommandsVersion } = require('../../package.json')
Expand All @@ -13,6 +14,33 @@
return sdkCommandsVersion
}

/**
* Returns the installed version of a certain package that lives inside another package in the current working directory.
* Returns undefined if the package is not installed.
*/
export async function getInstalledPackageVersionInsidePackage(
components: Pick<CliComponents, 'fs'>,
packageName: string,
packageRootName: string,
workingDirectory: string

Check warning on line 25 in packages/@dcl/sdk-commands/src/logic/config.ts

View check run for this annotation

Codecov / codecov/patch

packages/@dcl/sdk-commands/src/logic/config.ts#L25

Added line #L25 was not covered by tests
) {
try {
const packagePath = path.dirname(

Check warning on line 28 in packages/@dcl/sdk-commands/src/logic/config.ts

View check run for this annotation

Codecov / codecov/patch

packages/@dcl/sdk-commands/src/logic/config.ts#L27-L28

Added lines #L27 - L28 were not covered by tests
require.resolve(`${packageRootName}/package.json`, {
paths: [workingDirectory]
})
)
const packageJson = await getPackageJson(components, packagePath)

Check warning on line 33 in packages/@dcl/sdk-commands/src/logic/config.ts

View check run for this annotation

Codecov / codecov/patch

packages/@dcl/sdk-commands/src/logic/config.ts#L33

Added line #L33 was not covered by tests

const version =
(packageJson.dependencies && packageJson.dependencies[packageName]) ||
(packageJson.devDependencies && packageJson.devDependencies[packageName])

Check warning on line 37 in packages/@dcl/sdk-commands/src/logic/config.ts

View check run for this annotation

Codecov / codecov/patch

packages/@dcl/sdk-commands/src/logic/config.ts#L37

Added line #L37 was not covered by tests

return version ?? undefined
} catch (e) {
return 'unknown'

Check warning on line 41 in packages/@dcl/sdk-commands/src/logic/config.ts

View check run for this annotation

Codecov / codecov/patch

packages/@dcl/sdk-commands/src/logic/config.ts#L41

Added line #L41 was not covered by tests
}
}
/**
* Returns the installed version of a certain package in the current working directory.
* Returns "unknown" if the package is not installed.
Expand All @@ -23,12 +51,12 @@
workingDirectory: string
) {
try {
const sdkPath = path.dirname(
const packagePath = path.dirname(
require.resolve(`${packageName}/package.json`, {
paths: [workingDirectory]
})
)
const packageJson = await readJson<{ version: string }>(components, path.resolve(sdkPath, 'package.json'))
const packageJson = await readJson<{ version: string }>(components, path.resolve(packagePath, 'package.json'))

return packageJson.version ?? /* istanbul ignore next */ 'unknown'
} catch (e) {
Expand Down
52 changes: 45 additions & 7 deletions packages/@dcl/sdk-commands/src/logic/project-validations.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@
import path from 'path'
import { CliComponents } from '../components'
import { colors } from '../components/log'
import { printProgressInfo } from './beautiful-logs'
import { printProgressInfo, printSuccess, printWarning } from './beautiful-logs'
import { CliError } from './error'
import { getSceneFilePath, getValidSceneJson } from './scene-validations'
import { getInstalledPackageVersion } from './config'
import { getInstalledPackageVersion, getInstalledPackageVersionInsidePackage } from './config'
import { getSmartWearableFile, getValidWearableJson } from './portable-experience-sw-validations'
import { getPackageJson } from './project-files'

Expand Down Expand Up @@ -74,6 +74,29 @@
printProgressInfo(components.logger, colors.white('✅ Installing dependencies...'))
}

/*
* Runs "npm install" for desired project
*/
export async function installAssetPack(
components: Pick<CliComponents, 'logger' | 'spawner' | 'fs'>,
workingDirectory: string

Check warning on line 82 in packages/@dcl/sdk-commands/src/logic/project-validations.ts

View check run for this annotation

Codecov / codecov/patch

packages/@dcl/sdk-commands/src/logic/project-validations.ts#L82

Added line #L82 was not covered by tests
): Promise<void> {
const assetPack = '@dcl/asset-packs' as const
let assetPackVersion: string = ''
try {
assetPackVersion = await getInstalledPackageVersionInsidePackage(

Check warning on line 87 in packages/@dcl/sdk-commands/src/logic/project-validations.ts

View check run for this annotation

Codecov / codecov/patch

packages/@dcl/sdk-commands/src/logic/project-validations.ts#L84-L87

Added lines #L84 - L87 were not covered by tests
components,
assetPack,
'@dcl/inspector',
workingDirectory
)
await components.spawner.exec(workingDirectory, npmBin, ['install', `${assetPack}@${assetPackVersion}`, '-D'])
printSuccess(components.logger, `${assetPack}@${assetPackVersion} installed`, '')

Check warning on line 94 in packages/@dcl/sdk-commands/src/logic/project-validations.ts

View check run for this annotation

Codecov / codecov/patch

packages/@dcl/sdk-commands/src/logic/project-validations.ts#L93-L94

Added lines #L93 - L94 were not covered by tests
} catch (e: any) {
printWarning(components.logger, `Failed to install ${assetPack}@${assetPackVersion}' \n ${e.message}`)

Check warning on line 96 in packages/@dcl/sdk-commands/src/logic/project-validations.ts

View check run for this annotation

Codecov / codecov/patch

packages/@dcl/sdk-commands/src/logic/project-validations.ts#L96

Added line #L96 was not covered by tests
}
}

/**
* Run NPM commands
*/
Expand Down Expand Up @@ -106,15 +129,25 @@
export async function startValidations(components: Pick<CliComponents, 'spawner' | 'fs' | 'logger'>, cwd: string) {
try {
const sdkVersion = await getInstalledPackageVersion(components, '@dcl/sdk', cwd)
// Ignore relative or s3 versions
if (sdkVersion.startsWith('https://') || sdkVersion.startsWith('file://') || sdkVersion === '7.0.0') return
const packageJsonPath = path.resolve(cwd, 'package.json')

if (!(await components.fs.fileExists(packageJsonPath))) {
return
}

const packageJson = await getPackageJson(components, cwd)

// Ignore relative or s3 versions
const sdkPackageVersion =
(packageJson.dependencies && packageJson.dependencies['@dcl/sdk']) ||
(packageJson.devDependencies && packageJson.devDependencies['@dcl/sdk'])

Check warning on line 142 in packages/@dcl/sdk-commands/src/logic/project-validations.ts

View check run for this annotation

Codecov / codecov/patch

packages/@dcl/sdk-commands/src/logic/project-validations.ts#L142

Added line #L142 was not covered by tests
if (
sdkPackageVersion?.startsWith('https://') ||
sdkPackageVersion?.startsWith('file://') ||
sdkPackageVersion === '7.0.0'

Check warning on line 146 in packages/@dcl/sdk-commands/src/logic/project-validations.ts

View check run for this annotation

Codecov / codecov/patch

packages/@dcl/sdk-commands/src/logic/project-validations.ts#L145-L146

Added lines #L145 - L146 were not covered by tests
) {
return

Check warning on line 148 in packages/@dcl/sdk-commands/src/logic/project-validations.ts

View check run for this annotation

Codecov / codecov/patch

packages/@dcl/sdk-commands/src/logic/project-validations.ts#L148

Added line #L148 was not covered by tests
}

if (
packageJson.dependencies &&
(packageJson.dependencies['@dcl/js-runtime'] || packageJson.dependencies['@dcl/sdk'])
Expand All @@ -128,7 +161,12 @@
packageJson.devDependencies['@dcl/js-runtime'] = sdkVersion
}
await components.fs.writeFile(packageJsonPath, JSON.stringify(packageJson, null, 2), 'utf-8')
} catch (e) {
components.logger.error('Failed to run scene validations')
} catch (e: any) {
components.logger.error('Failed to run scene validations', e.message)

Check warning on line 165 in packages/@dcl/sdk-commands/src/logic/project-validations.ts

View check run for this annotation

Codecov / codecov/patch

packages/@dcl/sdk-commands/src/logic/project-validations.ts#L165

Added line #L165 was not covered by tests
}
}

// If there is a main.crdt file, its an editor scene. We need asset-packs package.
export function isEditorScene(components: Pick<CliComponents, 'fs'>, workingDirectory: string) {
return components.fs.fileExists(path.resolve(workingDirectory, 'main.crdt'))
}
2 changes: 1 addition & 1 deletion test/sdk-commands/commands/build/index.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ describe('build command', () => {
try {
await build.main({ args: { _: [], '--skip-install': true }, components })
} catch (_) {}
expect(needsDependenciesSpy).toBeCalledWith(components, process.cwd())
expect(needsDependenciesSpy).not.toBeCalled()
expect(installDependenciesSpy).not.toBeCalled()
})

Expand Down
Loading