Avoid duplicating docs in README, add developer instructions

.github/workflows/docs-deploy.yml

@@ -20,10 +20,10 @@ jobs:
         run: |
           npm ci
           cd website && npm install && cd -
-      - name: Build docs
-        run: |
-          npm run docs
-          cd website && npm run build && cd -
+      - name: Build documentation
+        run: cd website && npm run build && cd -
+      - name: Build API documentation
+        run: npm run docs:api
       - name: Zip docs
         run: |
           mv docs website/build/docs/api
@@ -32,11 +32,11 @@ jobs:
       - name: Copy docs to Toolforge
         uses: garygrossgarten/github-action-scp@dba89d858413133b6e35da91a3c989cc8cde3ad4
         with:
-          local: /home/runner/work/mwn/mwn/website/build/docs.zip
+          local: ${{github.workspace}}/website/build/docs.zip
           remote: /data/project/mwn/docs.zip
           host: login.toolforge.org
-          username: ${{ secrets.TOOLFORGE_USERNAME }}
-          privateKey: ${{ secrets.TOOLFORGE_PRIVATE_KEY }}
+          username: ${{secrets.TOOLFORGE_USERNAME}}
+          privateKey: ${{secrets.TOOLFORGE_PRIVATE_KEY}}
           rmRemote: true
       - name: Unzip
         uses: garygrossgarten/github-action-ssh@2b10f41b5a33808f6d24eafd253296766308b7c4
@@ -48,5 +48,5 @@ jobs:
             unzip /data/project/mwn/docs.zip -d .
             rm /data/project/mwn/docs.zip
           host: login.toolforge.org
-          username: ${{ secrets.TOOLFORGE_USERNAME }}
-          privateKey: ${{ secrets.TOOLFORGE_PRIVATE_KEY }}
+          username: ${{secrets.TOOLFORGE_USERNAME}}
+          privateKey: ${{secrets.TOOLFORGE_PRIVATE_KEY}}

package.json

@@ -17,7 +17,8 @@
 		"test": "nyc --reporter=lcov --reporter=text mocha tests/",
 		"coveralls": "nyc report --reporter=text-lcov | coveralls",
 		"test:ts": "ts-mocha -p tsconfig.json tests/ts/*",
-		"docs": "node scripts/generate-docs.js"
+		"docs": "cd website && npm start",
+		"docs:api": "node scripts/generate-docs.js"
 	},
 	"engines": {
 		"node": ">=10"

tests/test_wiki.js

@@ -7,11 +7,6 @@ let bot = new mwn({
 	...loginCredentials.account1,
 	userAgent: 'mwn (https://github.com/siddharthvp/mwn)',
 });
-let bot2 = new mwn({
-	silent: true,
-	userAgent: 'mwn (https://github.com/siddharthvp/mwn)',
-	...loginCredentials.account2,
-});
 
 async function setup() {
 	// Switching to BotPassword authentication due to OAuth being unreliable in CI due to
@@ -28,4 +23,4 @@ async function setup() {
 async function teardown() {}
 
 // Export everything
-module.exports = { mwn, bot, bot2, log, expect, assert, sinon, setup, teardown };
+module.exports = { mwn, bot, log, expect, assert, sinon, setup, teardown };

tsconfig.json

@@ -11,5 +11,6 @@
 	"include": ["./src/**/*"],
 	"ts-node": {
 		"logError": true
-	}
+	},
+	"compileOnSave": true
 }

website/docs/1-setting-up-mwn.md → website/docs/1-getting-started.md

@@ -2,7 +2,7 @@
 sidebar_position: 1
 ---
 
-# Setting up Mwn
+# Getting started
 
 Installation: `npm install mwn`
 
@@ -25,10 +25,16 @@ import { mwn } from 'mwn';
 
 If you're migrating from mwbot, note that:
 
--   `edit` in mwbot is different from `edit` in mwn. You want to use `save` instead.
--   If you were using the default formatversion=1 output format, set formatversion: 1 in the config options.
+- `edit` in mwbot is different from `edit` in mwn. You want to use `save` instead.
+- If you were using the default formatversion=1 output format, set formatversion: 1 in the config options.
 
-Create a new bot instance:
+#### Set up a bot password or OAuth credentials
+
+Mwn supports authentication via both [BotPasswords](https://www.mediawiki.org/wiki/Manual:Bot_passwords) and [OAuth 1.0a](https://www.mediawiki.org/wiki/OAuth/Owner-only_consumers). Use of OAuth is recommended as it does away the need for separate API requests for logging in, and is also more secure.
+
+Bot passwords, however, are a bit easier to set up. To generate one, go to the wiki's [Special:BotPasswords](https://en.wikipedia.org/wiki/Special:BotPasswords) page.
+
+#### Initializing bot object
 
 ```js
 const bot = await mwn.init({

website/docs/14-developing.md

@@ -0,0 +1,45 @@
+# Developing
+
+Obtain the latest development copy:
+
+```sh
+git clone https://github.com/siddharthvp/mwn.git
+cd mwn
+npm install		# install dependencies
+```
+
+Build: run `npm run build`. This generates the JS files in `build` from the TypeScript sources in `src`. The type errors raised can be ignored. You may want to configure your IDE to do this automatically on every code change.
+
+### Tests
+
+Mwn has a comprehensive test suite that uses [mocha](https://www.npmjs.com/package/mocha) runner and [chai](https://www.npmjs.com/package/chai) for expectations. Tests should be added for all bug fixes and feature changes. There are three types of tests:
+1. Tests that don't require a wiki: `npm run test:nowiki`
+2. Tests that are based on [testwiki](https://test.wikipedia.org/): `npm run test:testwiki`
+
+You need to setup botpassword login credentials for testwiki to run these tests. Create a `.auth.js` file in repository root, with this structure:
+
+```json
+{
+  "username": "",
+  "password": ""
+}
+```
+No edits (or any other type of write action) will be done on testwiki. The credentials are only for reading content. 
+
+3. Tests that require a local wiki: `npm run test:localwiki`
+
+For these, you need to spin up a local MediaWiki installation:
+  - Ensure that you have docker and docker-compose installed. If you're using Windows, you need to use WSL 2 and follow the steps at https://docs.docker.com/desktop/windows/wsl/ to link Docker Desktop to WSL.
+  - Run `npm run setuplocalwiki`
+
+If all goes well, you should now have a wiki up and running at [http://localhost:8080](http://localhost:8080). Sign in with username `Wikiuser` and password `wikipassword` (See [the setup.sh script](https://github.com/siddharthvp/mwn/blob/master/tests/docker/setup.sh) for other setup details.)
+
+**Troubleshooting**: If your local MW install fails with an error that a db table doesn't exist, try changing the sleep duration in https://github.com/siddharthvp/mwn/blob/master/tests/docker/main.sh#L2 to a higher value. The MariaDB database doesn't start accepting connections until it has initialised (time needed for that could be more on certain systems). 
+
+All tests except the ones that depend on testwiki will also be run automatically via GitHub Actions when you create a PR, so no worries if you're having trouble with local MW installation.
+
+### Documentation
+
+When adding new features, please also update the corresponding documentation in `website/docs`. Documentation can be previewed by running `npm run docs` which starts a server at [http://localhost:3000](http://localhost:3000) that updates the rendered docs live as they are edited.
+
+The docs are hosted on https://mwn.toolforge.org and generated via [Docusaurus](https://docusaurus.io/). Build and deployment take place via GitHub Actions whenever they're updated in the master branch.

website/docs/5-direct-api-calls.md

@@ -1,6 +1,8 @@
 # Direct API calls
 
-The `request` method is for directly querying the API. See [mw:API](https://www.mediawiki.org/wiki/API:Main_page) for options. You can create and test your queries in [Special:ApiSandbox](https://www.mediawiki.org/wiki/Special:ApiSandbox). Be sure to set formatversion: 2 in the options for format=json!
+The `request` method is for directly querying the API. See [mw:API](https://www.mediawiki.org/wiki/API:Main_page) for options. 
+
+You can create and test your queries in [Special:ApiSandbox](https://www.mediawiki.org/wiki/Special:ApiSandbox). However, note that ApiSandbox uses json formatversion=1 by default whereas Mwn defaults to formatversion=2. So you should set formatversion: 2 in the format=json options, otherwise the output can be formatted differently for certain API endpoints. Use of the legacy formatversion is not recommended.
 
 Example: get all images used on the article Foo
 
@@ -14,7 +16,7 @@ bot.request({
 });
 ```
 
-Mwn provides a great number of convenience methods so that you can avoid writing raw API calls, see the sections below.
+Mwn provides a great number of convenience methods so that you can avoid writing raw API calls, see the later sections.
 
 #### Raw web requests
 

website/src/pages/index.js

@@ -16,7 +16,7 @@ function HomepageHeader() {
 				<div className={styles.buttons}>
 					<Link
 						className="button button--secondary button--lg"
-						to="/docs/setting-up-mwn">
+						to="/docs/getting-started">
 						Mwn Tutorial - 5min ⏱️
 					</Link>
 				</div>