Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Snippet compiler #11582

Merged
merged 28 commits into from
Jun 2, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
f7c8f52
Snippet compiler PoC
pikinier20 Mar 3, 2021
80a4014
Reporter fix. Message levels
pikinier20 Mar 8, 2021
825f6c3
Fix snippet wrapper. Integrate snippet checker with Wiki and Markdown…
pikinier20 Mar 8, 2021
2c7c92d
Further implementation of snippet compiler. Add package name and self…
pikinier20 Mar 18, 2021
9584098
Add arguments to control snippet compiler.
pikinier20 Mar 19, 2021
ecb4b21
Start work on handling hide directives in code snippets
BarkingBad Mar 19, 2021
506c538
Hide irrelevant code snippet fragments in scaladoc
KacperFKorban Mar 19, 2021
82dad66
Apply review suggestions from PR #11822
KacperFKorban Mar 23, 2021
b1cc5da
Add returning real line of error in source file of snippet for snippe…
BarkingBad Mar 23, 2021
56e8b0c
Add failing flag
pikinier20 Mar 26, 2021
b3b7d0b
Add debug flag.
pikinier20 Mar 26, 2021
78676bc
Add addition operation for snippet compiler arg
pikinier20 Mar 26, 2021
2fe91f3
Refactor snippet compiler args
pikinier20 Mar 29, 2021
e8a5df9
Move snippet compiler logic to parsing phase.
pikinier20 Mar 29, 2021
ac76ac6
Refactor debug flag
pikinier20 Mar 30, 2021
94f77cb
Display snippet compilation messages in scaladoc
KacperFKorban Mar 25, 2021
bcea268
Apply review changes
KacperFKorban Apr 8, 2021
8ddb298
Run snippet compiler on static sites
pikinier20 Apr 9, 2021
3e8995c
Further enhancements of snippet contexts
pikinier20 Apr 14, 2021
d27e377
Filter non-scala snippets. Report errors in parsing override flags
pikinier20 Apr 14, 2021
bb07b46
Fix positions in static sites. Enchance snippet wrapping
pikinier20 Apr 19, 2021
30eec69
CRs
pikinier20 Apr 20, 2021
d0bf391
Add and update tests for snippet compiler. Bugfixes found during testing
pikinier20 Apr 22, 2021
3b0e11e
Wrap compiler messages into Try block to avoid lazy exceptions
pikinier20 Apr 29, 2021
975065b
Update branch
pikinier20 May 11, 2021
7d0e477
Fix classpath problems. Add support for scala.js. Add bootclasspath t…
pikinier20 May 11, 2021
ac8531d
Temporarily turn off snippet checking for scala3 library
pikinier20 May 17, 2021
946752f
Change requests from reviews
pikinier20 May 20, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ class StoreReporter(outer: Reporter = Reporter.NoReporter) extends Reporter {
if (infos != null) try infos.toList finally infos = null
else Nil

override def pendingMessages(using Context): List[Diagnostic] = infos.toList
override def pendingMessages(using Context): List[Diagnostic] = if (infos != null) infos.toList else Nil

override def errorsReported: Boolean = hasErrors || (outer != null && outer.errorsReported)
}
1 change: 1 addition & 0 deletions dist/bin/scaladoc
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,7 @@ eval "\"$JAVACMD\"" \
${JAVA_OPTS:-$default_java_opts} \
"${java_args[@]}" \
"${jvm_cp_args-}" \
-Dscala.usejavacp=true \
"dotty.tools.scaladoc.Main" \
"${scala_args[@]}" \
"${residual_args[@]}" \
Expand Down
12 changes: 0 additions & 12 deletions docs/css/dottydoc.css
Original file line number Diff line number Diff line change
Expand Up @@ -180,18 +180,6 @@ h5:hover a.anchor:hover {
font-family: var(--font-family-monospace);
}

/* code */
pre, code {
font-variant-ligatures: none;
}
pre {
padding: 0;
font-size: 13px;
background: var(--pre-bg);
border-radius: 2px;
border: 1px solid rgba(0, 0, 0, 0.1);
}

/* admonitions */
blockquote {
padding: 0 1em;
Expand Down
27 changes: 22 additions & 5 deletions project/Build.scala
Original file line number Diff line number Diff line change
Expand Up @@ -1243,16 +1243,19 @@ object Build {
libraryDependencies += ("org.scala-js" %%% "scalajs-dom" % "1.1.0").cross(CrossVersion.for3Use2_13)
)

def generateDocumentation(targets: Seq[String], name: String, outDir: String, ref: String, params: Seq[String] = Nil) =
def generateDocumentation(targets: Seq[String], name: String, outDir: String, ref: String, params: Seq[String] = Nil, usingScript: Boolean = true) =
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be fixed later but do you know why we need usingScript in some situations and couldn't just always use Compile/run ?

Copy link
Contributor

@romanowski romanowski Jun 1, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was my idea to have some coverage for scripts. Beside that there are no blockers to switch all cases to running Compile/run

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, for consistency I suggest always using Compile/run and instead test the scaladoc binary from https://github.com/lampepfl/dotty/blob/master/project/scripts/cmdTests

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pikinier20 can we change that in follow up PR?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pikinier20 can we change that in follow up PR?

Sure

Def.taskDyn {
val distLocation = (dist / pack).value
val projectVersion = version.value
IO.createDirectory(file(outDir))
val stdLibVersion = stdlibVersion(Bootstrapped)
val scalaLib = findArtifactPath(externalCompilerClasspathTask.value, "scala-library")
val dottyLib = (`scala3-library` / Compile / classDirectory).value
// TODO add versions etc.
def srcManaged(v: String, s: String) = s"out/bootstrap/stdlib-bootstrapped/scala-$v/src_managed/main/$s-library-src"
def scalaSrcLink(v: String, s: String) = s"-source-links:$s=github://scala/scala/v$v#src/library"
def dottySrcLink(v: String, s: String) = s"-source-links:$s=github://lampepfl/dotty/$v#library/src"

val revision = Seq("-revision", ref, "-project-version", projectVersion)
val cmd = Seq(
"-d",
Expand All @@ -1264,7 +1267,20 @@ object Build {
s"-source-links:github://lampepfl/dotty/$referenceVersion",
) ++ scalacOptionsDocSettings ++ revision ++ params ++ targets
import _root_.scala.sys.process._
Def.task((s"$distLocation/bin/scaladoc" +: cmd).!)
if (usingScript)
Def.task((s"$distLocation/bin/scaladoc" +: cmd).!)
else {
val escapedCmd = cmd.map(arg => if(arg.contains(" ")) s""""$arg"""" else arg)
Def.task {
try {
(Compile / run).toTask(escapedCmd.mkString(" ", " ", "")).value
0
} catch {
case _ : Throwable => 1
}
}
}

}

val SourceLinksIntegrationTest = config("sourceLinksIntegrationTest") extend Test
Expand Down Expand Up @@ -1311,7 +1327,7 @@ object Build {
generateSelfDocumentation := Def.taskDyn {
generateDocumentation(
(Compile / classDirectory).value.getAbsolutePath :: Nil,
"scaladoc", "scaladoc/output/self", VersionUtil.gitHash,
"scaladoc", "scaladoc/output/self", VersionUtil.gitHash
)
}.value,
generateScalaDocumentation := Def.inputTaskDyn {
Expand Down Expand Up @@ -1352,7 +1368,7 @@ object Build {
s"-source-links:docs=github://lampepfl/dotty/master#docs",
"-doc-root-content", docRootFile.toString,
"-Ydocument-synthetic-types"
)
), usingScript = false
))
}.evaluated,

Expand All @@ -1361,7 +1377,8 @@ object Build {
(Test / Build.testcasesOutputDir).value,
"scaladoc testcases",
"scaladoc/output/testcases",
"master")
"master"
)
}.value,

Test / buildInfoKeys := Seq[BuildInfoKey](
Expand Down
35 changes: 34 additions & 1 deletion scaladoc-js/resources/scaladoc-searchbar.css
Original file line number Diff line number Diff line change
Expand Up @@ -100,5 +100,38 @@

.pull-right {
float: right;
margin-left: auto
margin-left: auto;
}

.snippet-comment-button {
position: absolute;
display: inline-block;
left: 50%;
width: 24px;
height: 24px;
background:
linear-gradient(#fff, #fff),
linear-gradient(#fff, #fff),
#aaa;
background-position: center;
background-size: 50% 2px, 2px 50%;
background-repeat: no-repeat;
border-radius: 12px;
box-shadow: 0 0 2px black;
}

.snippet-comment-button:hover {
background:
linear-gradient(#444, #444),
linear-gradient(#444, #444),
#ddd;
background-position: center;
background-size: 50% 2px, 2px 50%;
background-repeat: no-repeat;
}

.hide-snippet-comments-button {
-ms-transform: rotate(45deg);
transform: rotate(45deg);
}

1 change: 1 addition & 0 deletions scaladoc-js/src/Main.scala
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,5 @@ package dotty.tools.scaladoc
object Main extends App {
Searchbar()
SocialLinks()
CodeSnippets()
}
27 changes: 27 additions & 0 deletions scaladoc-js/src/searchbar/code-snippets/CodeSnippets.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
package dotty.tools.scaladoc

import org.scalajs.dom._
import org.scalajs.dom.ext._

class CodeSnippets:
def toggleHide(e: html.Element | html.Document) = e.querySelectorAll("code span.hideable").foreach {
case e: html.Element if e.style.getPropertyValue("display").isEmpty => e.style.setProperty("display", "none")
case e: html.Element => e.style.removeProperty("display")
}

toggleHide(document)

document.querySelectorAll("pre").foreach {
case e: html.Element if e.querySelectorAll("code span.hideable").nonEmpty =>
val a = document.createElement("a")
a.addEventListener("click", { (_: MouseEvent) =>
if(a.classList.contains("hide-snippet-comments-button")) {
a.classList.remove("hide-snippet-comments-button")
} else {
a.classList.add("hide-snippet-comments-button")
}
toggleHide(e)
})
a.classList.add("snippet-comment-button")
e.insertBefore(a, e.firstChild)
}
13 changes: 13 additions & 0 deletions scaladoc-testcases/docs/docs/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
---


---

```scala sc:compile
2 + List(0)
```

```scala sc:compile
new snippetCompiler.Snippet0 { }
```

31 changes: 31 additions & 0 deletions scaladoc-testcases/src/tests/snippetComments.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
package tests.snippetComments


/**
* This is my codeblock
*
* ```
* //{{
* import xd
* import xd2
* //}}
*
*
* val x = 1 // This is my valid comment
*
* /*
* multi line comment
* */
* val reallyFLongDeclaration = "1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890"
* val y = 2 // comment in the same line
* // comment in new line
* val z = 3
*
* //{{
* val hideMe = 7
* //}}
* ```
*
* The end of my codeblock
*/
class A
68 changes: 68 additions & 0 deletions scaladoc-testcases/src/tests/snippetCompilerTests.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
package tests
package snippetCompilerTests

/**
* ```scala sc:compile
* //{
* import scala.collection.Seq
* //}
*
* def a = 2
* val x = 1 + List()
* a
*
* try {
* 2+3
* }
*
* /*
* Huge comment
* */
* val xd: String = 42
*
* def a(i: Int): Boolean = i match // This is a function
* case 1 => true
*
* val b: Int = 2.3 /* Also quite a big comment */
*
* val d: Long = "asd"
* ```
*
* ```scala sc:fail
* def a = 2
* val x = 1 + List()
* a
* ```
*
* ```scala sc:fail
* def a = 2
* ```
*
* ```scala sc:nocompile
* def a = 3
* a()
* ```
*/
class A {
trait B
}

/**
* ```scala sc:compile
* val c: Int = 4.5
* ```
*/
class B { }

trait Quotes {
val reflect: reflectModule = ???
trait reflectModule { self: reflect.type =>
/**
* ```scala sc:compile
* 2 + List()
* ```
*
*/
def a = 3
}
}
12 changes: 12 additions & 0 deletions scaladoc-testcases/src/tests/snippetTestcase1.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
package tests.snippetTestcase1

class SnippetTestcase1:
/**
* SNIPPET(OUTERLINEOFFSET:8,OUTERCOLUMNOFFSET:6,INNERLINEOFFSET:3,INNERCOLUMNOFFSET:2)
* ERROR(LINE:8,COLUMN:8)
* ```scala sc:compile
* 2 + List()
* ```
*
*/
def a = 3
58 changes: 58 additions & 0 deletions scaladoc-testcases/src/tests/snippetTestcase2.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
package tests
package snippetTestcase2

trait Quotes2[A] {
val r1: r1Module[_] = ???
trait r1Module[A] {
type X
object Y {
/**
* SNIPPET(OUTERLINEOFFSET:13,OUTERCOLUMNOFFSET:10,INNERLINEOFFSET:6,INNERCOLUMNOFFSET:6)
* ERROR(LINE:13,COLUMN:12)
* ```scala sc:compile
* 2 + List()
* ```
*
*/
type YY
}
val z: zModule = ???
trait zModule {
/**
* SNIPPET(OUTERLINEOFFSET:25,OUTERCOLUMNOFFSET:10,INNERLINEOFFSET:7,INNERCOLUMNOFFSET:6)
* ERROR(LINE:25,COLUMN:12)
* ```scala sc:compile
* 2 + List()
* ```
*
*/
type ZZ
}
}
object r2 {
type X
object Y {
/**
* SNIPPET(OUTERLINEOFFSET:39,OUTERCOLUMNOFFSET:10,INNERLINEOFFSET:5,INNERCOLUMNOFFSET:6)
* ERROR(LINE:39,COLUMN:12)
* ```scala sc:compile
* 2 + List()
* ```
*
*/
type YY
}
val z: zModule = ???
trait zModule {
/**
* SNIPPET(OUTERLINEOFFSET:51,OUTERCOLUMNOFFSET:10,INNERLINEOFFSET:6,INNERCOLUMNOFFSET:6)
* ERROR(LINE:51,COLUMN:12)
* ```scala sc:compile
* 2 + List()
* ```
*
*/
type ZZ
}
}
}
Loading