Feat(ajax responses): Improved localization handling, error reporting…

… and success messages (#363)

* feat(ajax responses): added IAjaxResponse interface

* refactor(error reporting): Nearly all errors now use ids instead of literals

* refactor: removed unnecessary dependencies of classes

* feat: Removed all error literals from the code and refactored all error handling

* fix(tests): Corrected signature of H5pEditor instanciation

* refactor(error handling): Converted error handling to Express pattern

* refactor(translation): added i18next and removed obsolete ITranslationService

* refactor(ajax endpoints): Responses now use interface return errors properly

* refactor(ajax endpoints): Added success message when installing and updating libraries

* refactor(translations): reorganized translation directory structure

* refactor(error reporting): added i18next to example

* fix: Corrected url generator for library files and fixed reference to client language strings

* feat: Updated English client language strings and fixed errors

* refactor: minor wording improvements

* refactor(ajax endpoint): Ajax endpoint now emits library installation und update message

* feat(tests): Added first tests for Express adapter.
BREAKING CHANGE: ILibraryStorage.getFileStream is now async!

* test: Added more AJAX endpoint tests

* test(express adapter): Added test for content type endpoint

* tests(AJAX endpoint): Added more tests for AJAX endpoint

* tests(ajax endpoint): Added test for getting library data and fixed related issues where current implementation doesn't conform to the H5P standard

* tests(ajax endpoints): Added tests for getting translations and validation errors when uploading packages

* refactor: Cleanup and added documentation

* docs: Added error handling to documentation

* feat(error reporting): Errors reported to the client now follow the H5P standard

* fix(error reporting): Corrected replacements

* refactor(example): Removed ineffective string interpolation

* docs: Corrected typos

README.md

@@ -58,7 +58,7 @@ const h5pEditor = H5P.fs(
 );
 ```
 
-and render the editor for content with a specific content ID with
+and render the editor for content (= the HTML user interface around the actual editor and the editor itself) with a specific content ID with
 
 ```js
 h5pEditor
@@ -120,6 +120,14 @@ The implementation needs to call several function regularly (comparable to a cro
 -   Call `H5PEditor.temporaryFileManager.cleanUp()` every 5 minutes. This checks which temporary files have expired and deletes them if necessary. It is important to do this, as temporary files are **not** automatically deleted when a piece of content is saved.
 -   Call `H5PEditor.contentTypeCache.updateIfNecessary()` every 12 hours. This will download information about the available content types from the H5P Hub. If you don't do this, users won't be shown new content types or updates to existing content types when they become available.
 
+### Handling errors
+
+If something goes wrong and a call to the library can't continue execution, it will normally throw either a `H5PError` or an `AggregateH5PError` (a collection of several errors). Both errors types represent errors that can be sent to the user to be displayed in the client (in the user's language). They don't include the English error message but an error id that you must translate yourself. Error ids and their English translations can be found in [`assets/translations`](/assets/translations). The translation strings follow the format used by (i18next)[https://i18next.com], but in theory you can use any localization library.
+
+Calls to the library might also throw regular `Error` objects. In this case the error is not caused by the business logic, but by some more basic functionality (file system, other library) or it might be an error that is addressed at the developer (i.e. because function parameters aren't correctly used).
+
+The Express adapter already deals with errors and returns proper HTTP status codes. Check out the implementation there for a guide how to deal with errors.
+
 ## Development & Testing
 
 ### Prerequisites

assets/translations/en.json → assets/translations/client/en.json

@@ -17,6 +17,9 @@
     "license": "License",
     "thumbnail": "Thumbnail",
     "noCopyrights": "No copyright information available for this content.",
+    "reuse": "Reuse",
+    "reuseContent": "Reuse Content",
+    "reuseDescription": "Reuse this content.",
     "downloadDescription": "Download this content as a H5P file.",
     "copyrightsDescription": "View copyright information for this content.",
     "embedDescription": "View the embed code for this content.",
@@ -53,5 +56,14 @@
     "licenseC": "Copyright",
     "contentType": "Content Type",
     "licenseExtras": "License Extras",
-    "changes": "Changelog"
-}
+    "changes": "Changelog",
+    "contentCopied": "Content is copied to the clipboard",
+    "connectionLost": "Connection lost. Results will be stored and sent when you regain connection.",
+    "connectionReestablished": "Connection reestablished.",
+    "resubmitScores": "Attempting to submit stored results.",
+    "offlineDialogHeader": "Your connection to the server was lost",
+    "offlineDialogBody": "We were unable to send information about your completion of this task. Please check your internet connection.",
+    "offlineDialogRetryMessage": "Retrying in :num....",
+    "offlineDialogRetryButtonLabel": "Retry now",
+    "offlineSuccessfulSubmit": "Successfully submitted results."
+}

assets/translations/server/en.json

@@ -0,0 +1,52 @@
+{
+    "missing-h5p-extension": "The file you uploaded is not a valid HTML5 Package (It does not have the .h5p file extension)",
+    "unable-to-unzip": "The file you uploaded is not a valid HTML5 Package (We are unable to unzip it)",
+    "file-size-too-large": "One of the files inside the package exceeds the maximum file size allowed. ({{file}} {{used}} > {{max}})",
+    "total-size-too-large": "The total size of the unpacked files exceeds the maximum size allowed. ({{used}} > {{max}})",
+    "invalid-content-folder": "A valid content folder is missing",
+    "invalid-h5p-json-file": "A valid main h5p.json file is missing",
+    "not-in-whitelist": "File "{{filename}}" not allowed. Only files with the following extensions are allowed: {{files-allowed}}.",
+    "unable-to-read-package-file": "Unable to read file from the package: {{fileName}}",
+    "unable-to-parse-package": "Unable to parse JSON file from the package: {{fileName}}",
+    "invalid-h5p-json-file-2": "The main h5p.json does not conform to the schema ({{reason}})",
+    "api-version-unsupported": "The system was unable to install the {{component}} component from the package, it requires a newer version of the H5P plugin. This site is currently running version {{current}}, whereas the required version is {{required}} or higher. You should consider upgrading and then try again.",
+    "invalid-library-name": "Invalid library name: {{name}}",
+    "invalid-library-json-file": "Could not find library.json file with valid json format for library {{name}}",
+    "invalid-schema-library-json-file": "The library.json file of the library {{name}} is invalid ({{reason}})",
+    "library-missing-file": "The file "{{file}}" is missing from library: "{{name}}"",
+    "invalid-semantics-json-file": "Invalid semantics.json file has been included in the library {{name}}",
+    "invalid-language-file": "Invalid language file {{file}} in library '{{library}}'",
+    "invalid-language-file-json": "The JSON of the language file {{file}} in the library {{library}} cannot be parsed",
+    "library-directory-name-mismatch": "Library directory name must match machineName or machineName-majorVersion.minorVersion (from library.json). (Directory: {{directoryName}} , machineName: {{machineName}}, majorVersion: {{majorVersion}}, minorVersion: {{minorVersion}})",
+    "error-creating-temporary-directory": "Could not create temporary directory to unpack H5P package",
+    "corrupt-file": "The file {{file}} could not be read from the H5P package. The package seems to be corrupt.",
+    "hub-install-no-content-type": "No content type was specified.",
+    "hub-install-invalid-content-type": "The chosen content type is invalid.",
+    "hub-install-denied": "You do not have permission to install content types. Contact the administrator of your site.",
+    "hub-install-download-failed": "Could not download package from the Hub to temporary file.",
+    "upload-package-failed-tmp": "Could not save uploaded package to temporary file.",
+    "content-file-missing": "File {{filename}} does not exist in {{contentId}}",
+    "error-generating-unique-content-filename": "Cannot determine a unique filename for {{filename}}",
+    "error-communicating-with-hub": "Could not fetch content type information from the H5P Hub. HTTP status {{statusCode}} ({{statusText}})",
+    "error-communicating-with-hub-no-status": "Could not fetch content type information from the H5P Hub.",
+    "error-registering-at-hub": "Could not register this site at the H5P Hub. HTTP status {{statusCode}} ({{statusText}})",
+    "error-registering-at-hub-no-status": "Could not register this site at the H5P Hub.",
+    "library-not-found": "Library {{name}} was not found.",
+    "invalid-main-library-name": "mainLibraryName is invalid: {{message}}",
+    "library-consistency-check-not-installed": "Can't check library consistency for {{name}} as the library is not installed.",
+    "library-consistency-check-library-json-unreadable": "Error in library {{name}}: library.json not readable: {{message}}",
+    "library-consistency-check-file-missing": "Missing files in library {{name}}: {{files}}",
+    "invalid-ubername-pattern": "'{{name}}' is not a valid H5P library name (\"ubername\"). You must follow this pattern: {{example}}'",
+    "download-content-not-found": "Content can't be downloaded as no content with id {{contentId}} exists.",
+    "download-content-forbidden": "You do not have permission to download content with id {{contentId}}.",
+    "download-content-unreadable-data": "Content can't be downloaded as the content data is unreadable.",
+    "download-content-unreadable-metadata": "Content can't be downloaded as the content metadata is unreadable.",
+    "import-package-no-id-assigned": "Something went wrong when storing the package: no content id was assigned.",
+    "installed-libraries": "Added {{count}} new H5P library",
+    "installed-libraries_plural": "Added {{count}} new H5P libraries.",
+    "updated-libraries": "Updated {{count}} old library.",
+    "updated-libraries_plural": "Updated {{count}} old libraries.",
+    "library-file-missing": "The requested library file {{filename}} of library {{library}} does not exist.",
+    "malformed-request": "The request sent by the client is malformed: {{error}}",
+    "package-validation-failed": "Validating h5p package failed."
+}

assets/translations/storage-file-implementations/en.json

@@ -0,0 +1,16 @@
+{
+    "error-generating-unique-temporary-filename": "Cannot determine a unique filename for {{name}}",
+    "temporary-file-not-found": "The file {{filename}} is not accessible for user {{userId}} or does not exist.",
+    "add-file-content-not-found": "Cannot add file {{filename}} to content with id {{id}}: Content with does not exist.",
+    "error-creating-content": "Could not create content.",
+    "error-generating-content-id": "Could not generate id for new content.",
+    "delete-content-not-found": "Cannot delete content: It does not exist.",
+    "delete-content-file-not-found": "Cannot delete file {{filename}}: It does not exist.",
+    "add-library-file-not-installed": "Can't add {{filename}} to library {{libraryName}} because the library metadata has not been installed.",
+    "clear-library-not-found": "Can't clear library {{libraryName}} because the library has not been installed.",
+    "install-library-already-installed": "Library {{libraryName}} has already been installed.",
+    "remove-library-library-missing": "Library {{libraryName}} is not installed on the system.",
+    "update-library-library-missing": "Library {{libraryName}} can't be updated as it hasn't been installed yet.",
+    "illegal-relative-filename": "Relative paths in filenames are not allowed: {{filename}} is illegal",
+    "illegal-absolute-filename": "Absolute paths in filenames are not allowed: {{filename}} is illegal"
+}

docs/endpoints.md

@@ -18,6 +18,20 @@ server.use(
 
 Note that the Express adapter does not include pages to create, editor, view, list or delete content!
 
+**IMPORTANT:** The adapter expects the requests object of Express to be extended like this:
+
+```ts
+{
+    user: IUser, // must be populated with information about the user (mostly id and access rights)
+    t: (errorId: string, replacements: {[key: string]: string }) => string 
+}
+```
+
+The function `t` must return the string for the errorId translated into the user's or the content's language.
+Replacements are added to the localized string with curly braces: {{replacement}}
+It is suggested you use [i18next](https://www.i18next.com/) for localization, but you can use any library, 
+as long as you make sure the function t is added to the request object.
+
 ## Handling requests yourself
 
 If you choose to do so, you can also handle requests manually. You must then follow these specifications:
@@ -177,8 +191,8 @@ const config = {
     coreUrl: '/core',           // URL of static player "core files"
     downloadUrl: '/download',   // URL to download h5p packages
     editorLibraryUrl: '/editor',// URL of static editor "core files" (not the content types!)
-    librariesUrl: '/libraries', // URL at which library files (= content types) can be retreived
-    paramsUrl: '/params'        // URL at which the paramteres (= content.json) of content can be retreived
+    librariesUrl: '/libraries', // URL at which library files (= content types) can be retrieved
+    paramsUrl: '/params'        // URL at which the parameters (= content.json) of content can be retrieved
     playUrl: '/play'            // URL at which content can be displayed
     ... // further configuration values
 }

docs/h5p-editor-constructor.md

@@ -2,7 +2,7 @@
 
 There are two ways of creating a H5PEditor object:
 
-- You can use the convenience function [`H5P.fs(...)`](/src/implementation/fs/index.ts) that use basic file system implementations for all data storage services. You can use the function if you're just getting started. Later on, you'll want to contruct the editor with custom implementations of the data storage services. Check out the JSDoc of the function for details how to use it.
+- You can use the convenience function [`H5P.fs(...)`](/src/implementation/fs/index.ts) that use basic file system implementations for all data storage services. You can use the function if you're just getting started. Later on, you'll want to construct the editor with custom implementations of the data storage services. Check out the JSDoc of the function for details how to use it.
 - You can construct it manually by calling `new H5P.H5PEditor(...)`. The constructor arguments are used to provide data storage services and settings. You can find the interfaces referenced in [`src/types.ts`](/src/types.ts).
 
 Explanation of the arguments of the constructor:
@@ -33,10 +33,6 @@ The `contentStorage` provides information about installed content and creates it
 const contentStorage = new FileContentStorage(`h5p/content`);
 ```
 
-## translationService
-
-Used to translates literals into the local language. It must implement the `ITranslationService` interface.
-
 ## temporaryStorage
 
 When the user uploads files in the H5P editor client, these files are not directly stored alongside the content, either because the content hasn't been saved before (and there is no contentId) or because this might create left-over files if these files aren't used after all. Instead the server stores these files in a temporary storage system and adds the '#tmp' tag to the files' path. When the editor client requests a file the system retrieves the file from temporary storage instead of the regular content storage.

examples/express.ts

@@ -1,6 +1,9 @@
 import bodyParser from 'body-parser';
 import express from 'express';
 import fileUpload from 'express-fileupload';
+import i18next from 'i18next';
+import i18nextExpressMiddleware from 'i18next-express-middleware';
+import i18nextNodeFsBackend from 'i18next-node-fs-backend';
 import os from 'os';
 import path from 'path';
 
@@ -33,6 +36,20 @@ function displayIps(port: string): void {
 }
 
 const start = async () => {
+    await i18next
+        .use(i18nextNodeFsBackend)
+        .use(i18nextExpressMiddleware.LanguageDetector)
+        .init({
+            backend: {
+                loadPath: 'assets/translations/{{ns}}/{{lng}}.json'
+            },
+            debug: process.env.DEBUG && process.env.DEBUG.includes('i18n'),
+            defaultNS: 'server',
+            fallbackLng: 'en',
+            ns: ['server', 'storage-file-implementations'],
+            preload: ['en']
+        });
+
     const h5pEditor = H5P.fs(
         await new H5P.EditorConfig(
             new H5P.fsImplementations.JsonStorage(
@@ -63,6 +80,8 @@ const start = async () => {
         next();
     });
 
+    server.use(i18nextExpressMiddleware.handle(i18next));
+
     server.use(
         h5pEditor.config.baseUrl,
         H5P.adapters.express(