[FEATURE] Add JSDoc build functionalities #42

codeworrior · 2018-06-28T12:22:53Z

Note: the code is full of console.log just to track down an unexpected exit of the Node.js process.

RandomByte · 2018-06-28T13:45:49Z

By writing a resource, the FileSystem adapter drains its content stream. Therefore the .clone workaround makes sense. However, I'll look into a way to have the FS adapter to provide a new stream.

There is this undocumented contract that whoever reads from a Resources stream has to provide a new readable stream via setStream

RandomByte · 2018-06-28T22:27:06Z

Basically, the issue is as follows:

Writing resource A to the file system requires to drain the stream. Currently, after that operation the Resource has irreversibly lost any content, making it impossible to further process this resource (content is somewhat essential).

I see two possible solutions here:

Before the write operation, load the content of A into a buffer.
=> After the write operation, content A₂ is still available
Only downside is the potentially high memory consumption caused by all those buffers.
The resource gets a new readable stream of the file that has been written to FS (tmp)
=> After the write operation, content A₂ is still available
However, any "physical" changes done to that file will be applied to resource A too. This might be unexpected.

RandomByte · 2018-07-05T14:52:21Z

I think 1. is closer to the expected behavior, I'll try to prepare a ui5-fs PR for that soon.

RandomByte · 2018-07-10T14:02:52Z

Started working on the discussed ui5-fs fix in SAP/ui5-fs#22

RandomByte · 2019-01-31T17:55:34Z

Blocker should be resolved with SAP/ui5-fs#22

CLAassistant · 2019-02-11T14:01:40Z

All committers have signed the CLA.

RandomByte · 2019-02-11T14:05:28Z

Rebased

RandomByte · 2019-02-26T14:09:28Z

Implementation Concept

JSDoc generation UI5 Tooling integration.pdf

Additional Implementation Details

Preferably add a "ui5 build jsdoc” command defining all JSDoc related tasks as enabled by default
- It is still possible to activate single JSDoc related tasks in any other build using the corresponding CLI parameter
- Done with [INTERNAL] Build: Add 'jsdoc' command ui5-cli#162
Add ui5.yaml configuration for JSDoc excludes (like thirdparty- or very big, not relevant resources). Grunt JSDoc build currently does not have any exclude handling. Maven JSDoc build reads this information from .library
- Basic implementation with 7c98831 - ~~TODO~~ (done) + ~~Pending clarification~~ (clarified)
Enhance Custom Task implementation for defining "final" tasks (lower part of the chart) and positioning as “first/last” task as there might not be any active "final" task that could be used for relative positioning OBSOLETE - Tasks that where meant to be executed in this "final" stage shall be moved into an "SDK" project that, when being built, will be executed last anyways
Processor returns array of resources but also writes to fs? (yes, at least for jsdoc config, but maybe different location?) (maybe different dest location?) => Only writes to tmp fs. Currently easiest solution to have easy reuse of UI5 JSDoc libs and proper separation of concerns

CC: @tommyvinhlam

test/lib/tasks/generateJsdoc.js

RandomByte · 2019-03-05T09:39:44Z

Not sure anymore whether the project name should be included in the temporary path directly. At least we should remove all alpha-num characters to prevent directory traversal

Edit: Done with 0d4a77f

RandomByte · 2019-03-07T16:47:45Z

b93d4bd depends on SAP/ui5-fs#112

RandomByte · 2019-03-07T16:55:39Z

TODO: Document new JSDoc exclude configuration in UI5 Project - Done: SAP/ui5-project#152

Example from sap.ui.core:

---
specVersion: "0.1"
type: library
metadata:
  name: sap.ui.core
  copyright: |-
   OpenUI5
    * (c) Copyright 2009-${currentYear} SAP SE or an SAP affiliate company.
    * Licensed under the Apache License, Version 2.0 - see LICENSE.txt.
builder:
  jsdoc:
    excludes:
    - "jquery-*"
    - "sap-ui-*"
    - "sap/ui/debug/**"
    - "sap/ui/qunit/**"
    - "sap/ui/thirdparty/**"

RandomByte · 2019-03-07T21:10:56Z

Still not sure whether JSDoc exclude patterns should include the virtual base path (/resources / /test-resources) to allow differentiation between the two (or more). Currently they are designed to only work within /resources (just like the JSDoc build itself).

This came to my mind when thinking about the definition of project-wide exclude patterns as requested in SAP/ui5-tooling#124. They should probably include the virtual paths to allow excluding resources of /test-resources for example.

So even though the JSDoc build itself is only working on /resources, having the same flavor of include/exclude patterns (that include the virtual base path) in the ui5.yaml might make sense.

matz3 · 2019-03-08T09:58:42Z

@RandomByte I would align this with the bundler, which also doesn't contain resources or test-resources in the pattern/filters but implicitly uses resources.

RandomByte · 2019-03-08T15:30:20Z

We had another discussion and tend to keep JSDoc exclude patterns aligned with the bundler patterns while use absolute virtual paths (/resources/pony/**) for more general excludes.

…on or .library resources are found Globs may return resources in random order, therefore choosing the first of many matches can lead to erratic results.

Prominent example: theme libraries

This prevents JSDoc errors in case no resources have written to the source path.

…located Prominent example: theme libraries

RandomByte · 2019-03-18T15:10:17Z

Rebased

devtomtom

Looks good to me!

codeworrior · 2019-03-21T12:55:20Z

I've tried it out on OpenUI5 (together with the pending change there) and it worked quite well!

As css files are required for a working SDK

They are not necessary in preview/dev scenarios

devtomtom

👍

codeworrior requested a review from RandomByte June 28, 2018 12:22

RandomByte mentioned this pull request Jul 10, 2018

[FIX] Prevent FS write from draining Resources content SAP/ui5-fs#22

Merged

RandomByte added the enhancement New feature or request label Oct 18, 2018

RandomByte force-pushed the create-jsdoc-for-libraries branch from 26f25fd to 996c711 Compare February 11, 2019 14:01

RandomByte force-pushed the create-jsdoc-for-libraries branch from 996c711 to 92e32ef Compare February 26, 2019 13:16

RandomByte force-pushed the create-jsdoc-for-libraries branch from 92e32ef to e7680f3 Compare February 26, 2019 14:19

RandomByte mentioned this pull request Feb 26, 2019

[Roadmap] UI5 Tooling SAP/ui5-tooling#101

Closed

21 tasks

RandomByte force-pushed the create-jsdoc-for-libraries branch 2 times, most recently from 2489edd to 7ac62c7 Compare February 28, 2019 15:16

devtomtom self-requested a review March 1, 2019 07:52

RandomByte force-pushed the create-jsdoc-for-libraries branch from 7ac62c7 to 399af06 Compare March 1, 2019 11:48

RandomByte reviewed Mar 5, 2019

View reviewed changes

test/lib/tasks/generateJsdoc.js Outdated Show resolved Hide resolved

RandomByte force-pushed the create-jsdoc-for-libraries branch 2 times, most recently from 2707f80 to 5718c87 Compare March 11, 2019 13:20

RandomByte added 9 commits March 18, 2019 16:03

JSDoc config JSON: Fix paths on Windows

68df5e7

Builder: Add JSDoc build option

0b17227

Add SDK Transformation task and processor

c63372c

executeJsdocSdkTransformation: Throw error if too many project api.js…

f085fad

…on or .library resources are found Globs may return resources in random order, therefore choosing the first of many matches can lead to erratic results.

executeJsdocSdkTransformation: Add tests

89f7792

sdkTransformer: Add tests

b8f9768

executeJsdocSdkTransformation: Skip projects with missing api.json

b3bdf80

Prominent example: theme libraries

generateJsdoc: Always create tmp dirs

88eda89

This prevents JSDoc errors in case no resources have written to the source path.

generateJsdoc: Skip JSDoc processing if no input resources have been …

8272acb

…located Prominent example: theme libraries

RandomByte force-pushed the create-jsdoc-for-libraries branch from d91ea6e to 8272acb Compare March 18, 2019 15:10

RandomByte added 2 commits March 19, 2019 13:20

Update JSDoc lib files from OpenUI5 change

769f201

Add generateApiIndex task

652d0cd

RandomByte force-pushed the create-jsdoc-for-libraries branch from ed3cb45 to 652d0cd Compare March 19, 2019 14:48

RandomByte added 2 commits March 20, 2019 10:49

apiIndexGenerator: Rename variables

413ccf0

Add tests for generateApiIndex task and processor

5905a2e

devtomtom previously approved these changes Mar 20, 2019

View reviewed changes

Make theme build default task for JSDoc build

b82ac65

As css files are required for a working SDK

RandomByte dismissed devtomtom’s stale review via b82ac65 March 21, 2019 13:12

Also disable replaceCopyright and replaceVersion tasks in JSDoc build

b61c7d0

They are not necessary in preview/dev scenarios

RandomByte requested a review from devtomtom March 21, 2019 14:56

devtomtom approved these changes Mar 21, 2019

View reviewed changes

RandomByte merged commit 293a4b0 into master Mar 21, 2019

RandomByte deleted the create-jsdoc-for-libraries branch March 21, 2019 14:59

matz3 mentioned this pull request Jun 16, 2021

[INTERNAL] JSDoc: Add check for copyright notice #617

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FEATURE] Add JSDoc build functionalities #42

[FEATURE] Add JSDoc build functionalities #42

codeworrior commented Jun 28, 2018

RandomByte commented Jun 28, 2018 •

edited

Loading

RandomByte commented Jun 28, 2018

RandomByte commented Jul 5, 2018

RandomByte commented Jul 10, 2018

RandomByte commented Jan 31, 2019 •

edited

Loading

CLAassistant commented Feb 11, 2019 •

edited

Loading

RandomByte commented Feb 11, 2019

RandomByte commented Feb 26, 2019 •

edited

Loading

RandomByte commented Mar 5, 2019 •

edited

Loading

RandomByte commented Mar 7, 2019

RandomByte commented Mar 7, 2019 •

edited

Loading

RandomByte commented Mar 7, 2019

matz3 commented Mar 8, 2019

RandomByte commented Mar 8, 2019

RandomByte commented Mar 18, 2019

devtomtom left a comment

codeworrior commented Mar 21, 2019

devtomtom left a comment

[FEATURE] Add JSDoc build functionalities #42

[FEATURE] Add JSDoc build functionalities #42

Conversation

codeworrior commented Jun 28, 2018

RandomByte commented Jun 28, 2018 • edited Loading

RandomByte commented Jun 28, 2018

RandomByte commented Jul 5, 2018

RandomByte commented Jul 10, 2018

RandomByte commented Jan 31, 2019 • edited Loading

CLAassistant commented Feb 11, 2019 • edited Loading

RandomByte commented Feb 11, 2019

RandomByte commented Feb 26, 2019 • edited Loading

Implementation Concept

Additional Implementation Details

RandomByte commented Mar 5, 2019 • edited Loading

RandomByte commented Mar 7, 2019

RandomByte commented Mar 7, 2019 • edited Loading

RandomByte commented Mar 7, 2019

matz3 commented Mar 8, 2019

RandomByte commented Mar 8, 2019

RandomByte commented Mar 18, 2019

devtomtom left a comment

Choose a reason for hiding this comment

codeworrior commented Mar 21, 2019

devtomtom left a comment

Choose a reason for hiding this comment

RandomByte commented Jun 28, 2018 •

edited

Loading

RandomByte commented Jan 31, 2019 •

edited

Loading

CLAassistant commented Feb 11, 2019 •

edited

Loading

RandomByte commented Feb 26, 2019 •

edited

Loading

RandomByte commented Mar 5, 2019 •

edited

Loading

RandomByte commented Mar 7, 2019 •

edited

Loading