Merge branch 'master' of github.com:scalableminds/webknossos into pri…

…cing * 'master' of github.com:scalableminds/webknossos: Replace babel with esbuild for tests and for webpack (#6527) Use less ram when initializing data textures (#6711) Composable sql interpolation (#6718) Release 23.01.0 (#6717) add license checker (#6715) Fix brush performance for coarse mags & avoid some unnecessary re-renders (#6708) Change backend format command (#6697) Improve layout of terms of services check (#6712)
scalableminds · Jan 5, 2023 · f1e34ac · f1e34ac
2 parents 0b50790 + 5995054
commit f1e34ac
Show file tree

Hide file tree

Showing 72 changed files with 1,419 additions and 2,035 deletions.
diff --git a/.babelrc b/.babelrc
diff --git a/CHANGELOG.released.md b/CHANGELOG.released.md
@@ -7,6 +7,41 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
 and this project adheres to [Calendar Versioning](http://calver.org/) `0Y.0M.MICRO`.
 For upgrade instructions, please check the [migration guide](MIGRATIONS.released.md).
 
+## [23.01.0](https://github.com/scalableminds/webknossos/releases/tag/23.01.0) - 2023-01-03
+[Commits](https://github.com/scalableminds/webknossos/compare/22.12.0...23.01.0)
+
+### Highlights
+- Added a new datasets tab to the dashboard which supports managing datasets in folders. Folders can be organized hierarchically and datasets can be moved into these folders. Selecting a dataset will show dataset details in a sidebar. [#6591](https://github.com/scalableminds/webknossos/pull/6591)
+- webKnossos is now able to recover from a lost webGL context. [#6663](https://github.com/scalableminds/webknossos/pull/6663)
+- Major performance improvements for brushing in coarse magnifications. [#6708](https://github.com/scalableminds/webknossos/pull/6708)
+
+### Added
+- Added sign in via OIDC. [#6534](https://github.com/scalableminds/webknossos/pull/6534)
+- Added the option to search a specific folder in the new datasets tab. [#6677](https://github.com/scalableminds/webknossos/pull/6677)
+- The new datasets tab in the dashboard allows multi-selection of datasets so that multiple datasets can be moved to a folder at once. As in typical file explorers, CTRL + left click adds individual datasets to the current selection. Shift + left click selects a range of datasets. [#6683](https://github.com/scalableminds/webknossos/pull/6683)
+
+### Changed
+- Bulk task creation now needs the taskTypeId, the task type summary will no longer be accepted. [#6640](https://github.com/scalableminds/webknossos/pull/6640)
+- Error handling and reporting is more robust now. [#6700](https://github.com/scalableminds/webknossos/pull/6700)
+- The Quick-Select settings are opened (and closed) automatically when labeling with the preview mode. That way, bulk labelings with preview mode don't require constantly opening the settings manually. [#6706](https://github.com/scalableminds/webknossos/pull/6706)
+
+### Fixed
+- Fixed import of N5 datasets. [#6668](https://github.com/scalableminds/webknossos/pull/6668)
+- Fixed a bug where it was possible to create invalid an state by deleting teams that are referenced elsewhere. [6664](https://github.com/scalableminds/webknossos/pull/6664)
+- Miscellaneous fixes for the new folder UI. [#6674](https://github.com/scalableminds/webknossos/pull/6674)
+- Fixed import of remote datasets with multiple layers and differing resolution pyramid. #[6670](https://github.com/scalableminds/webknossos/pull/6670)
+- Fixed broken Get-new-Task button in task dashboard. [#6677](https://github.com/scalableminds/webknossos/pull/6677)
+- Fixed access of remote datasets using the Amazon S3 protocol [#6679](https://github.com/scalableminds/webknossos/pull/6679)
+- Fixed a bug in line measurement that would lead to an infinite loop. [#6689](https://github.com/scalableminds/webknossos/pull/6689)
+- Fixed a bug where malformed json files could lead to uncaught exceptions.[#6691](https://github.com/scalableminds/webknossos/pull/6691)
+- Fixed rare crash in publications page. [#6700](https://github.com/scalableminds/webknossos/pull/6700)
+- Respect the config value mail.smtp.auth (used to be ignored, always using true) [#6692](https://github.com/scalableminds/webknossos/pull/6692)
+
+### Removed
+
+### Breaking Changes
+
+
 ## [22.12.0](https://github.com/scalableminds/webknossos/releases/tag/22.12.0) - 2022-11-24
 [Commits](https://github.com/scalableminds/webknossos/compare/22.11.2...22.12.0)
 

diff --git a/CHANGELOG.unreleased.md b/CHANGELOG.unreleased.md
@@ -8,13 +8,9 @@ and this project adheres to [Calendar Versioning](http://calver.org/) `0Y.0M.MIC
 For upgrade instructions, please check the [migration guide](MIGRATIONS.released.md).
 
 ## Unreleased
-[Commits](https://github.com/scalableminds/webknossos/compare/22.12.0...HEAD)
+[Commits](https://github.com/scalableminds/webknossos/compare/23.01.0...HEAD)
 
 ### Added
-- Added sign in via OIDC. [#6534](https://github.com/scalableminds/webknossos/pull/6534)
-- Added a new datasets tab to the dashboard which supports managing datasets in folders. Folders can be organized hierarchically and datasets can be moved into these folders. Selecting a dataset will show dataset details in a sidebar. [#6591](https://github.com/scalableminds/webknossos/pull/6591)
-- Added the option to search a specific folder in the new datasets tab. [#6677](https://github.com/scalableminds/webknossos/pull/6677)
-- The new datasets tab in the dashboard allows multi-selection of datasets so that multiple datasets can be moved to a folder at once. As in typical file explorers, CTRL + left click adds individual datasets to the current selection. Shift + left click selects a range of datasets. [#6683](https://github.com/scalableminds/webknossos/pull/6683)
 
 ### Changed
 - The log viewer in the Voxelytics workflow reporting now uses a virtualized list. [#6579](https://github.com/scalableminds/webknossos/pull/6579)
@@ -28,19 +24,10 @@ For upgrade instructions, please check the [migration guide](MIGRATIONS.released
 - Bulk task creation now needs the taskTypeId, the task type summary will no longer be accepted. [#6640](https://github.com/scalableminds/webknossos/pull/6640)
 - Error handling and reporting is more robust now. [#6700](https://github.com/scalableminds/webknossos/pull/6700)
 - The Quick-Select settings are opened (and closed) automatically when labeling with the preview mode. That way, bulk labelings with preview mode don't require constantly opening the settings manually. [#6706](https://github.com/scalableminds/webknossos/pull/6706)
+- Improved performance of opening a dataset or annotation. [#6711](https://github.com/scalableminds/webknossos/pull/6711)
 - Redesigned organization page to include more infos on organization users, storage, webKnossos plan and provided opportunities to upgrade. [#6602](https://github.com/scalableminds/webknossos/pull/6602)
 
 ### Fixed
-- Fixed import of N5 datasets. [#6668](https://github.com/scalableminds/webknossos/pull/6668)
-- Fixed a bug where it was possible to create invalid an state by deleting teams that are referenced elsewhere. [6664](https://github.com/scalableminds/webknossos/pull/6664)
-- Miscellaneous fixes for the new folder UI. [#6674](https://github.com/scalableminds/webknossos/pull/6674)
-- Fixed import of remote datasets with multiple layers and differing resolution pyramid. #[6670](https://github.com/scalableminds/webknossos/pull/6670)
-- Fixed broken Get-new-Task button in task dashboard. [#6677](https://github.com/scalableminds/webknossos/pull/6677)
-- Fixed access of remote datasets using the Amazon S3 protocol [#6679](https://github.com/scalableminds/webknossos/pull/6679)
-- Fixed a bug in line measurement that would lead to an infinite loop. [#6689](https://github.com/scalableminds/webknossos/pull/6689)
-- Fixed a bug where malformed json files could lead to uncaught exceptions.[#6691](https://github.com/scalableminds/webknossos/pull/6691)
-- Fixed rare crash in publications page. [#6700](https://github.com/scalableminds/webknossos/pull/6700)
-- Respect the config value mail.smtp.auth (used to be ignored, always using true) [#6692](https://github.com/scalableminds/webknossos/pull/6692)
 
 ### Removed
 

diff --git a/DEV_INSTALL.md b/DEV_INSTALL.md
@@ -139,10 +139,10 @@ For upgrades, please check the [changelog](CHANGELOG.released.md) & [migration g
 yarn run lint
 
 # Format frontend code
-yarn run pretty
+yarn format-frontend
 
 # Format backend code
-yarn pretty-backend
+yarn format-backend
 
 # Frontend type checking
 yarn flow

diff --git a/MIGRATIONS.released.md b/MIGRATIONS.released.md
@@ -5,6 +5,18 @@ See `MIGRATIONS.unreleased.md` for the changes which are not yet part of an offi
 This project adheres to [Calendar Versioning](http://calver.org/) `0Y.0M.MICRO`.
 User-facing changes are documented in the [changelog](CHANGELOG.released.md).
 
+## [23.01.0](https://github.com/scalableminds/webknossos/releases/tag/23.01.0) - 2023-01-03
+[Commits](https://github.com/scalableminds/webknossos/compare/22.12.0...23.01.0)
+
+- Bulk task creation now needs the taskTypeId, the task type summary will no longer be accepted. If you have scripts generating CSVs for bulk task creation, they should not output task type summaries. [#6640](https://github.com/scalableminds/webknossos/pull/6640)
+
+### Postgres Evolutions:
+
+- [091-folders.sql](conf/evolutions/091-folders.sql)
+- [092-oidc.sql](conf/evolutions/092-oidc.sql)
+- [093-terms-of-service.sql](conf/evolutions/093-terms-of-service.sql)
+
+
 ## [22.12.0](https://github.com/scalableminds/webknossos/releases/tag/22.12.0) - 2022-11-24
 [Commits](https://github.com/scalableminds/webknossos/compare/22.11.2...22.12.0)
 
@@ -485,4 +497,4 @@ No migrations necessary.
 
 
 ## [18.07.0](https://github.com/scalableminds/webknossos/releases/tag/18.07.0) - 2018-07-05
-First release
+First release
diff --git a/MIGRATIONS.unreleased.md b/MIGRATIONS.unreleased.md
@@ -6,12 +6,6 @@ This project adheres to [Calendar Versioning](http://calver.org/) `0Y.0M.MICRO`.
 User-facing changes are documented in the [changelog](CHANGELOG.released.md).
 
 ## Unreleased
-[Commits](https://github.com/scalableminds/webknossos/compare/22.12.0...HEAD)
-
-- Bulk task creation now needs the taskTypeId, the task type summary will no longer be accepted. If you have scripts generating CSVs for bulk task creation, they should not output task type summaries. [#6640](https://github.com/scalableminds/webknossos/pull/6640)
+[Commits](https://github.com/scalableminds/webknossos/compare/23.01.0...HEAD)
 
 ### Postgres Evolutions:
-
-- [091-folders.sql](conf/evolutions/091-folders.sql)
-- [092-oidc.sql](conf/evolutions/092-oidc.sql)
-- [093-terms-of-service.sql](conf/evolutions/093-terms-of-service.sql)
diff --git a/app/Startup.scala b/app/Startup.scala
@@ -14,7 +14,8 @@ import oxalis.mail.{Mailer, MailerConfig}
 import oxalis.security.WkSilhouetteEnvironment
 import oxalis.telemetry.SlackNotificationService
 import play.api.inject.ApplicationLifecycle
-import utils.{SQLClient, WkConf}
+import utils.WkConf
+import utils.sql.SQLClient
 
 import scala.concurrent.ExecutionContext.Implicits.global
 import scala.concurrent.Future

diff --git a/app/WebKnossosModule.scala b/app/WebKnossosModule.scala
@@ -11,7 +11,7 @@ import models.voxelytics.ElasticsearchClient
 import oxalis.files.TempFileService
 import oxalis.mail.MailchimpTicker
 import oxalis.telemetry.SlackNotificationService
-import utils.SQLClient
+import utils.sql.SQLClient
 
 class WebKnossosModule extends AbstractModule {
   override def configure(): Unit = {

diff --git a/app/controllers/Application.scala b/app/controllers/Application.scala
@@ -12,8 +12,9 @@ import oxalis.security.WkEnv
 import play.api.libs.json.{JsObject, Json}
 import play.api.mvc.{Action, AnyContent, PlayBodyParsers}
 import slick.jdbc.PostgresProfile.api._
-import utils.{SQLClient, SimpleSQLDAO, StoreModules, WkConf}
+import utils.{StoreModules, WkConf}
 import oxalis.mail.{DefaultMails, Send}
+import utils.sql.{SQLClient, SimpleSQLDAO}
 
 import javax.inject.Inject
 import scala.concurrent.ExecutionContext

diff --git a/app/controllers/MaintenanceController.scala b/app/controllers/MaintenanceController.scala
@@ -10,7 +10,7 @@ import oxalis.security.WkEnv
 import play.api.libs.json.Json
 import play.api.mvc.{Action, AnyContent}
 import slick.jdbc.PostgresProfile.api._
-import utils.{SQLClient, SimpleSQLDAO}
+import utils.sql.{SQLClient, SimpleSQLDAO}
 import scala.concurrent.duration._
 
 import scala.concurrent.ExecutionContext

diff --git a/app/controllers/ReportController.scala b/app/controllers/ReportController.scala
@@ -10,7 +10,8 @@ import oxalis.security.WkEnv
 import play.api.libs.json.{Json, OFormat}
 import play.api.mvc.{Action, AnyContent}
 import slick.jdbc.PostgresProfile.api._
-import utils.{ObjectId, SQLClient, SimpleSQLDAO}
+import utils.sql.{SQLClient, SimpleSQLDAO}
+import utils.ObjectId
 
 import scala.concurrent.ExecutionContext
 

diff --git a/app/models/annotation/Annotation.scala b/app/models/annotation/Annotation.scala
@@ -16,7 +16,8 @@ import slick.jdbc.PostgresProfile.api._
 import slick.jdbc.TransactionIsolation.Serializable
 import slick.lifted.Rep
 import slick.sql.SqlAction
-import utils.{ObjectId, SQLClient, SQLDAO, SimpleSQLDAO}
+import utils.sql.{SQLClient, SQLDAO, SimpleSQLDAO}
+import utils.ObjectId
 
 import scala.concurrent.ExecutionContext
 import scala.concurrent.duration.FiniteDuration

diff --git a/app/models/annotation/AnnotationPrivateLink.scala b/app/models/annotation/AnnotationPrivateLink.scala
@@ -3,14 +3,15 @@ package models.annotation
 import com.scalableminds.util.accesscontext.DBAccessContext
 import com.scalableminds.util.time.Instant
 import com.scalableminds.util.tools.Fox
-import com.scalableminds.webknossos.schema.Tables.{AnnotationPrivatelinks, _}
+import com.scalableminds.webknossos.schema.Tables._
 import oxalis.security.RandomIDGenerator
 
 import javax.inject.Inject
 import play.api.libs.json.{JsValue, Json, OFormat}
 import slick.jdbc.PostgresProfile.api._
 import slick.lifted.Rep
-import utils.{ObjectId, SQLClient, SQLDAO}
+import utils.sql.{SQLClient, SQLDAO}
+import utils.ObjectId
 
 import scala.concurrent.ExecutionContext
 

diff --git a/app/models/annotation/TracingStore.scala b/app/models/annotation/TracingStore.scala
@@ -12,7 +12,7 @@ import play.api.libs.json.{JsObject, Json}
 import play.api.mvc.{Result, Results}
 import slick.jdbc.PostgresProfile.api._
 import slick.lifted.Rep
-import utils.{SQLClient, SQLDAO}
+import utils.sql.{SQLClient, SQLDAO}
 
 import scala.concurrent.{ExecutionContext, Future}
 

diff --git a/app/models/binary/DataSet.scala b/app/models/binary/DataSet.scala
@@ -24,7 +24,8 @@ import slick.jdbc.PostgresProfile.api._
 import slick.jdbc.TransactionIsolation.Serializable
 import slick.lifted.Rep
 import slick.sql.SqlAction
-import utils.{ObjectId, SQLClient, SQLDAO, SimpleSQLDAO}
+import utils.sql.{SQLClient, SQLDAO, SimpleSQLDAO}
+import utils.ObjectId
 
 import scala.concurrent.ExecutionContext
 

diff --git a/app/models/binary/DataStore.scala b/app/models/binary/DataStore.scala
@@ -9,7 +9,8 @@ import play.api.libs.json.{Format, JsObject, Json}
 import play.api.mvc.{Result, Results}
 import slick.jdbc.PostgresProfile.api._
 import slick.lifted.Rep
-import utils.{ObjectId, SQLClient, SQLDAO}
+import utils.sql.{SQLClient, SQLDAO}
+import utils.ObjectId
 
 import scala.concurrent.{ExecutionContext, Future}
 

diff --git a/app/models/binary/Publication.scala b/app/models/binary/Publication.scala
@@ -12,7 +12,8 @@ import play.api.libs.json.Format.GenericFormat
 import play.api.libs.json.{JsObject, Json}
 import slick.jdbc.PostgresProfile.api._
 import slick.lifted.Rep
-import utils.{ObjectId, SQLClient, SQLDAO}
+import utils.sql.{SQLClient, SQLDAO}
+import utils.ObjectId
 
 import scala.concurrent.ExecutionContext
 

diff --git a/app/models/folder/Folder.scala b/app/models/folder/Folder.scala
@@ -2,15 +2,16 @@ package models.folder
 
 import com.scalableminds.util.accesscontext.DBAccessContext
 import com.scalableminds.util.tools.Fox
-import com.scalableminds.webknossos.schema.Tables.{Folders, _}
+import com.scalableminds.webknossos.schema.Tables._
 import models.organization.{Organization, OrganizationDAO}
 import models.team.{TeamDAO, TeamService}
 import models.user.User
 import play.api.libs.json.{JsObject, Json, OFormat}
 import slick.jdbc.PostgresProfile.api._
 import slick.lifted.Rep
 import slick.sql.SqlAction
-import utils.{ObjectId, SQLClient, SQLDAO}
+import utils.sql.{SQLClient, SQLDAO}
+import utils.ObjectId
 
 import javax.inject.Inject
 import scala.concurrent.ExecutionContext

diff --git a/app/models/job/Job.scala b/app/models/job/Job.scala
@@ -21,7 +21,8 @@ import play.api.libs.json.{JsObject, Json}
 import slick.jdbc.PostgresProfile.api._
 import slick.jdbc.TransactionIsolation.Serializable
 import slick.lifted.Rep
-import utils.{ObjectId, SQLClient, SQLDAO, WkConf}
+import utils.sql.{SQLClient, SQLDAO}
+import utils.{ObjectId, WkConf}
 
 import scala.concurrent.ExecutionContext
 import scala.concurrent.duration.FiniteDuration

diff --git a/app/models/job/Worker.scala b/app/models/job/Worker.scala
@@ -14,7 +14,8 @@ import play.api.inject.ApplicationLifecycle
 import play.api.libs.json.{JsObject, Json}
 import slick.jdbc.PostgresProfile.api._
 import slick.lifted.Rep
-import utils.{ObjectId, SQLClient, SQLDAO, WkConf}
+import utils.sql.{SQLClient, SQLDAO}
+import utils.{ObjectId, WkConf}
 
 import javax.inject.Inject
 import scala.concurrent.ExecutionContext

diff --git a/app/models/mesh/Mesh.scala b/app/models/mesh/Mesh.scala
@@ -12,7 +12,8 @@ import play.api.libs.json.Json._
 import play.api.libs.json._
 import slick.jdbc.PostgresProfile.api._
 import slick.lifted.Rep
-import utils.{ObjectId, SQLClient, SQLDAO}
+import utils.sql.{SQLClient, SQLDAO}
+import utils.ObjectId
 
 import scala.concurrent.ExecutionContext
 

diff --git a/app/models/organization/Organization.scala b/app/models/organization/Organization.scala
@@ -10,7 +10,8 @@ import models.team.PricingPlan
 import models.team.PricingPlan.PricingPlan
 import slick.jdbc.PostgresProfile.api._
 import slick.lifted.Rep
-import utils.{ObjectId, SQLClient, SQLDAO}
+import utils.sql.{SQLClient, SQLDAO}
+import utils.ObjectId
 
 import scala.concurrent.ExecutionContext
 

diff --git a/app/models/project/Project.scala b/app/models/project/Project.scala
@@ -13,10 +13,11 @@ import models.team.TeamDAO
 import models.user.{User, UserService}
 import net.liftweb.common.Full
 import play.api.libs.functional.syntax._
-import play.api.libs.json.{Json, _}
+import play.api.libs.json._
 import slick.jdbc.PostgresProfile.api._
 import slick.lifted.Rep
-import utils.{ObjectId, SQLClient, SQLDAO}
+import utils.sql.{SQLClient, SQLDAO}
+import utils.ObjectId
 
 import scala.concurrent.{ExecutionContext, Future}
 

diff --git a/app/models/shortlinks/ShortLink.scala b/app/models/shortlinks/ShortLink.scala
@@ -6,7 +6,8 @@ import com.scalableminds.webknossos.schema.Tables.{Shortlinks, ShortlinksRow}
 import play.api.libs.json.{Json, OFormat}
 import slick.jdbc.PostgresProfile.api._
 import slick.lifted.Rep
-import utils.{ObjectId, SQLClient, SQLDAO}
+import utils.sql.{SQLClient, SQLDAO}
+import utils.ObjectId
 
 import javax.inject.Inject
 import scala.concurrent.ExecutionContext