Create dev folder, move dev scripts and docs there (wikimedia-gadgets…

…#1142)

Allow `sync.pl` and `patch-test.js` to be run from any subdirectory, and move them into a new dev folder, along with morebits-test and DEVELOPER.md.  oldtwinkle.js has been removed as entirely unnecessary.

Also allows providing a port number to patchtest via commandline, and adds sync.pl to the package defs as `sync` and `deployall` with the `--mode=deploy --all` options (use `npm run deployall -- --lang=`, etc. to pass other flags).

Instruct `eslint` to ignore /dev folder.

Also moved select2 files into a lib/ folder.

.eslintrc.json

@@ -9,7 +9,7 @@
 	"globals": {
 		"mw": "readonly"
 	},
-	"ignorePatterns": "select2*",
+	"ignorePatterns": ["dev/", "lib/"],
 	"rules": {
 		"no-console": "error",
 		"no-extra-parens": ["error", "all", { "nestedBinaryExpressions": false }],
@@ -35,7 +35,7 @@
 		"func-call-spacing": ["error", "never"],
 		"indent": ["error", "tab", { "outerIIFEBody": 0, "SwitchCase": 1 }],
 		"key-spacing": ["error", {"singleLine": {"beforeColon": false, "afterColon": true}}],
-		"keyword-spacing": ["error", { "after": true , "before": true}],
+		"keyword-spacing": ["error", { "after": true, "before": true}],
 		"linebreak-style": ["error", "unix"],
 		"no-array-constructor": "error",
 		"no-bitwise": "error",

CONTRIBUTING.md

@@ -33,11 +33,11 @@ Once you've got a local fork up and running, commit your changes!
 
 ### Testing your code
 
-Testing Twinkle can be tricky, but the most straightforward way to test your code is to open up your [browser's console window][jserrors] and paste in your new code.  You'll have to load the new version by running the corresponding function in your console, e.g., `Twinkle.protect()` for twinkleprotect.js.
+Testing Twinkle can be tricky, but thankfully we've made it easy with a helper script, [patch-test.js](./dev/patch-test.js).  Simply run `npm run patchtest` in your terminal, which will generate the `patch-test-loader.js` file.  Then, run a localhost server (e.g. `php -S 127.0.0.1:5500` or `python -m SimpleHTTPServer 5500`) and paste the contents of `patch-test-loader.js` in your [browser's console window][jserrors].  You should be all set!  If you put `mw.loader.getScript('http://127.0.0.1:5500/dev/patch-test-loader.js');` in your common.js page, you won't even need to paste anything once you run the server!  You can also test your code by simply pasting it into the browser console, although that's not recommended.
 
 Some things to watch out for:
 - If your tests have any chance of making actual edits, consider making them in a sandbox; be aware that some things may not work properly outside the appropriate namespace.  An even better place to test is on the [test wiki](http://test.wikipedia.org)!  Some parts of Twinkle rely on specific template code or on certain wiki-preferences, so testing certain things outside of enWiki may be difficlut (e.g., pending changes).
-- The non-module scripts `morebits.js` and `twinkle.js` are usually more complicated to test.
+- The non-module scripts `morebits.js` and `twinkle.js` can be more complicated to test.
 - The `twinkleconfig` pseudo-module holds the code to save and determine user preferences, while `twinkle.js` holds the defaults.
 - There is some variety in how the individual modules are written, in particular among the `friendly` family as well as with `twinkleconfig.js`.
 

DEVELOPER.md → dev/DEVELOPER.md

@@ -2,11 +2,11 @@
 
 (WIP)
 
-Collaborators are encouraged to thoroughly review and [test](./CONTRIBUTING.md) each pull request, including their own.  Unless urgent or obvious, it can be helpful to leave PRs open for folks to opine.
+Collaborators are encouraged to thoroughly review and [test](../CONTRIBUTING.md#testing-your-code) each pull request, including their own.  Unless urgent or obvious, it can be helpful to leave PRs open for folks to opine.
 
 Things to watch out for:
 
-- Items and processes laid out in [CONTRIBUTING.md](./CONTRIBUTING.md) are followed.
+- Items and processes laid out in [CONTRIBUTING.md](../CONTRIBUTING.md) are followed.
 - Twinkle is meant to run on the latest weekly version of MediaWiki as rolled out every Thursday on the English Wikipedia.  Backwards compatibility is not guaranteed.
 - The goal is for Twinkle and Morebits to support the same [browsers that MediaWiki supports](https://www.mediawiki.org/wiki/Browser_compatibility).  In particular, collaborators should avoid [unsupported additions](https://kangax.github.io/compat-table/es6/) from ES6 (aka ES2015); `.includes` and `.find` are among the most likely to show up, although the jQuery `$.find()` is fine. Our ESLint configuration includes a [plugin](https://github.com/nkt/eslint-plugin-es5) that should catch most cases.
 - Certain positional jQuery selectors like `:first`, `:last`, and `:eq` were [deprecated in jQuery version 3.4.0](https://blog.jquery.com/2019/04/10/jquery-3-4-0-released/) and should probably not be reintroduced.  Instead, use methods like `.first()`, `.last()`, or `.eq()`.
@@ -15,7 +15,7 @@ Things to watch out for:
 
 There are two ways to upload Twinkle scripts to Wikipedia or another destination. You can do it with a [Perl script](#synchronization-using-syncpl) (recommended) or [manually](#manual-synchronization).
 
-After the files are synced, ensure that [MediaWiki:Gadgets-definition][] contains the gadget definition found in [Gadget.md](./Gadget.md) (`sync.pl` will report its status). In addition to the `Twinkle` definition, the gadget installs the `morebits` library as a hidden gadget, making it efficiently available for other tools to use. `Twinkle-pagestyles` is a hidden [peer gadget](https://www.mediawiki.org/wiki/ResourceLoader/Migration_guide_(users)#Gadget_peers) of Twinkle. Before Twinkle has loaded, it adds space where the TW menu would go in the Vector skin, so that the top bar does not "jump".
+After the files are synced, ensure that [MediaWiki:Gadgets-definition][] contains the gadget definition found in [gadget.txt](./gadget.txt) (`sync.pl` will report its status). In addition to the `Twinkle` definition, the gadget installs the `morebits` library as a hidden gadget, making it efficiently available for other tools to use. `Twinkle-pagestyles` is a hidden [peer gadget](https://www.mediawiki.org/wiki/ResourceLoader/Migration_guide_(users)#Gadget_peers) of Twinkle. Before Twinkle has loaded, it adds space where the TW menu would go in the Vector skin, so that the top bar does not "jump".
 
 [select2][] is also uploaded as a hidden gadget for better menus and to take advantage of the Resource Loader over the [Toolforge CDN](https://tools.wmflabs.org/cdnjs/); it is done so under the [MIT license](https://github.com/select2/select2/blob/develop/LICENSE.md).  Loading via the ResourceLoader causes it to register as a nodejs/commonjs environment with `module.exports`, so a slight tweak has been made, eliminating that check.  Ideally, this will be handled differently (see [external libraries](https://www.mediawiki.org/wiki/ResourceLoader/Migration_guide_for_extension_developers#Special_case_of_external_libraries) and [T108655](https://phabricator.wikimedia.org/T108655).  As such, be careful when updating select2 from upstream.
 
@@ -71,29 +71,29 @@ When running the program, you can enter your credentials on the command line usi
     url      =
     base     = User:AzaToth/
 
-`username`, `password`, and `mode` (one of `deploy`, `push`, or `pull`) are required, either through the command line or configuration file; lang and family default to `en.wikipedia`. Note that your working directory **must** be clean; if not, either `stash` or `commit` your changes. The script automatically handles the directory (e.g. `modules/`) from the file path when downloading/uploading.
+`username`, `password`, and `mode` (one of `deploy`, `push`, or `pull`) are required, either through the command line or configuration file; lang and family default to `en.wikipedia`. Note that your working directory **must** be clean; if not, either `stash` or `commit` your changes. The script automatically handles the directory (e.g. `modules/`) from the file path when downloading/uploading. It can be run from any subdirectory of the repository.
 
 Using the `deploy` mode, [interface-admins][intadmin] can deploy Twinkle files live to their MediaWiki:Gadget locations. You will need to set up a bot password at [Special:BotPasswords][special_botpass].
 
-    ./sync.pl --mode=deploy twinkle.js morebits.js ...
+    sync.pl --mode=deploy twinkle.js morebits.js ...
 
 If no files are provided, it will just report the status of the gadget. You may also `deploy` **all** files via
 
-    ./sync.pl --mode=deploy --all
+    sync.pl --mode=deploy --all
 
 Note that for syncing to a non-Enwiki project, you will also need to specify the --lang and/or --family parameters. For instance, to sync the files with `fr.wikiquote.org` you should specify `--lang=fr --family=wikiquote`, such as
 
-    ./sync.pl --mode=deploy --lang=fr --family=wikiquote --all
+    sync.pl --mode=deploy --lang=fr --family=wikiquote --all
 
 When `deploy`ing or `push`ing, the script will attempt to parse the latest on-wiki edit summary for the commit of the last update, and will use that to create an edit summary using the changes committed since then. If it cannot find anything that looks like a commit hash, it will give you the most recent commits for each file and prompt you to enter an edit summary manually.
 
 To `pull` user Foobar's changes (i.e. `User:Foobar/morebits.js`) down from the wiki, do:
 
-    ./sync.pl --base User:Foobar/ --mode=pull twinkle.js morebits.js ...
+    sync.pl --base User:Foobar/ --mode=pull twinkle.js morebits.js ...
 
 To `push` your changes to user Foobar's wiki page, do:
 
-    ./sync.pl --base User:Foobar/ --mode=push twinkle.js morebits.js ...
+    sync.pl --base User:Foobar/ --mode=push twinkle.js morebits.js ...
 
 The `--base` flag operates as a *prefix*; note the presence of the trailing `/`.
 

patch-test.js → dev/patch-test.js

@@ -13,14 +13,16 @@
  * How to use:
  * - Run `npm run patchtest` which generates patch-test-loader.js
  * - Set up a localhost server (such as by using server.js or by running
- *   php -S localhost:5500) and load patch-test-loader.js in the wiki environment,
+ *   php -S 127.0.0.1:5500) and load patch-test-loader.js in the wiki environment,
  *   using the browser console or the common.js page.
+ * - You can provide a different port by running `npm run patchtest -- 1234`
  */
 
 const fs = require('fs');
 const {execSync} = require('child_process');
 
-const server = 'http://127.0.0.1:5500';
+const port = isNaN(Number(process.argv[2])) ? '5500' : process.argv[2];
+const server = 'http://127.0.0.1:' + port;
 
 // find the last common commit between this branch and master, and get the list
 // of files changed since that commit
@@ -84,7 +86,7 @@ function createTestLoader(changedFiles) {
 		${importLine('modules/twinklexfd.js')}
 	});`.replace(/^\t/mg, '').replace(/^\s*$/mg, '');
 
-	fs.writeFileSync('./patch-test-loader.js', jsLoaderSource, console.log);
+	fs.writeFileSync('./dev/patch-test-loader.js', jsLoaderSource, console.log);
 
-	console.log(`Wrote import statements for ${importsCount} modified file${importsCount > 1 ? 's' : ''} to patch-test-loader.js`);
+	console.log(`Wrote import statements for ${importsCount} modified file${importsCount > 1 ? 's' : ''} to dev/patch-test-loader.js`);
 }

sync.pl → dev/sync.pl

@@ -6,7 +6,9 @@
 
 use English qw(-no_match_vars);
 use utf8;
+use Cwd qw(cwd abs_path);
 use FindBin qw($Bin);
+use File::Spec::Functions qw(rel2abs abs2rel);
 use Getopt::Long;
 use Term::ANSIColor;
 
@@ -15,6 +17,11 @@
 use MediaWiki::API;
 use File::Slurper qw(read_text write_text);
 
+# Save original directory, for relative paths of files
+my $origDir = cwd();
+# Move to top of repository
+# Could do with git rev-parse --show-toplevel, but this is fine
+chdir "$Bin/../" or die "$ERRNO\n";
 
 # Default config values, mode is intentionally absent
 my %conf = (
@@ -28,8 +35,8 @@
            );
 
 my $rc = '.twinklerc';
-# Check script directory and ~ (home) for config file, preferring the former
-my @dotLocales = map { $_.q{/}.$rc } ($Bin, $ENV{HOME});
+# Check repo directory and ~ (home) for config file, preferring the former
+my @dotLocales = map { $_.q{/}.$rc } (cwd(), $ENV{HOME});
 foreach my $dot (@dotLocales) {
   if (-e -f -r $dot) {
     # Preserve default options if not present in twinklerc
@@ -53,7 +60,7 @@
 }
 
 # Make sure we know what we're doing before doing it
-# Checks for required parameters, returns list of files (@ARGV or --all)
+# Checks for required parameters, returns list of files (from @ARGV or --all)
 my @files = forReal();
 
 # Open API and log in before anything else
@@ -77,7 +84,7 @@
     if ($page =~ /^twinkle/) {
       $page =~ s/^twinkle\b/Twinkle/; # twinkle.js, etc. files are capitalized on-wiki
     } else {
-      $page =~ s/\w+\///;       # Remove directories (modules/, select2/)
+      $page =~ s/\w+\///;       # Remove directories (modules/, lib/)
     }
     $page = $conf{base}.$page; # base set to MediaWiki:Gadget- for deploy in &forReal
 
@@ -156,7 +163,7 @@
     $wikiGadgetDef .= $wg[$_]."\n";
   }
 
-  my $gadgetFile = 'Gadget.md';
+  my $gadgetFile = 'gadget.txt';
   my $localGadgetDef = read_text($gadgetFile);
   if ($wikiGadgetDef eq $localGadgetDef) {
     print "Gadget up-to-date\n";
@@ -175,8 +182,6 @@
 
 ### SUBROUTINES
 # Check that everything is in order
-# Data::Dumper is simpler but the output is ugly, and this ain't worth another
-# dependency
 sub forReal {
   my @meaningful = qw (mode lang family);
   push @meaningful, 'url' if $conf{url};
@@ -209,12 +214,18 @@ sub forReal {
   } else {
     # Confirm files are valid
     my %checkFiles = map { $_ => 1 } @allFiles;
-    foreach my $arg (@ARGV) {
-      if (!$checkFiles{$arg}) {
-        print colored ['yellow'], "$arg not defined as part of the gadget, skipping\n";
+    foreach my $file (@ARGV) {
+      # Adjust path for being run from anywhere to main repo directory
+      # 1. Make path absolute, relative to the directory from which the script was run
+      # 2. Clean it up (../, etc.)
+      # 3. Make it relative again
+      $file = abs2rel(abs_path(rel2abs($file, $origDir)));
+
+      if (!$checkFiles{$file}) {
+        print colored ['yellow'], "$file not defined as part of the gadget, skipping\n";
         next;
       }
-      push @inputs, $arg;
+      push @inputs, $file;
     }
 
     if (!@inputs) {
@@ -442,8 +453,8 @@ sub usage {
   twinkle-pagestyles.css
   morebits.js
   morebits.css
-  select2/select2.min.js
-  select2/select2.min.css
+  lib/select2.min.js
+  lib/select2.min.css
   modules/twinkleconfig.js
   modules/twinklearv.js
   modules/twinklebatchdelete.js

package.json

@@ -1,15 +1,17 @@
 {
-  "title": "Twinkle",
-  "name": "twinkle",
-  "private": true,
-  "repository": "azatoth/twinkle",
-  "scripts": {
-    "lint": "eslint .",
-    "lint:fix": "eslint . --fix",
-    "patchtest": "node patch-test.js"
-  },
-  "devDependencies": {
-    "eslint": "^7.8.1",
-    "eslint-plugin-es5": "^1.5.0"
-  }
+	"title": "Twinkle",
+	"name": "twinkle",
+	"private": true,
+	"repository": "azatoth/twinkle",
+	"scripts": {
+		"lint": "eslint .",
+		"lint:fix": "eslint . --fix",
+		"patchtest": "node dev/patch-test.js",
+		"sync": "perl dev/sync.pl",
+		"deployall": "perl dev/sync.pl --mode=all --all"
+	},
+	"devDependencies": {
+		"eslint": "^7.8.1",
+		"eslint-plugin-es5": "^1.5.0"
+	}
 }