Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Laying the groundwork for the git refactor #50

Merged
merged 50 commits into from
Feb 7, 2023
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
a4af6c1
Laying the groundwork for the `git` refactor
confused-Techie Jan 11, 2023
d46a55a
Use `switch` for comparison, remove `lowercase()` external usage
confused-Techie Jan 12, 2023
0a13702
Added Codeberg VCS detection
confused-Techie Jan 12, 2023
53f3fce
Avoid semicolon auto-insertion, start specifying git class
confused-Techie Jan 12, 2023
76728aa
Framework of VCS Services, and testing them
confused-Techie Jan 12, 2023
f05b84d
Full testing suite written for GitHub `doesUserHaveRepo`
confused-Techie Jan 12, 2023
f4292dc
Updated `codeql` to ignore new testing location
confused-Techie Jan 12, 2023
1ccf4ae
Update fixture file name
confused-Techie Jan 12, 2023
67d4cbb
Move heavy lifting into `github.js` for ownership returns
confused-Techie Jan 12, 2023
abbbaa8
Fix typo `unkown` => `unknown`
confused-Techie Jan 13, 2023
60de1c4
Use new endpoint to find user ownership, new tests
confused-Techie Jan 13, 2023
33fbfab
`readme` & docs
confused-Techie Jan 13, 2023
d1140a3
Some more tests for new `readme` function
confused-Techie Jan 13, 2023
c77dfd4
Additional layout of core API structure
confused-Techie Jan 14, 2023
5ce374b
Merge branch 'main' into refactor-git
confused-Techie Jan 14, 2023
af71577
fix CodeQL issues
Digitalone1 Jan 14, 2023
872d107
github class: use const where possible
Digitalone1 Jan 14, 2023
850ea37
use an internal initializer and introduce private properties
Digitalone1 Jan 14, 2023
89e755a
camelCase properties + fix base class setters
Digitalone1 Jan 14, 2023
f3b609d
git class: remove setters and add jsdoc to methods
Digitalone1 Jan 15, 2023
fd84d66
Complete the specific components of the GitHub VCS Module
confused-Techie Jan 17, 2023
e0a4ddd
Fixed Undeclared Variable
confused-Techie Jan 17, 2023
2919648
Fix Syntax Error
confused-Techie Jan 17, 2023
d88fca7
Documentation, and refining exports and scope of responsibilites
confused-Techie Jan 17, 2023
fd69cf8
Progress on Publishing functionality
confused-Techie Jan 17, 2023
ee592e6
Some comments
confused-Techie Jan 17, 2023
d802f4b
Finish `newPackageData` begin fleshing out other functions
confused-Techie Jan 18, 2023
cfbd3a3
Fixed syntax error on ternary operator
confused-Techie Jan 18, 2023
9c92be4
Methodology to reduce repetitious package handling
confused-Techie Jan 18, 2023
d69a4c8
Initial construction of `package_json` class
confused-Techie Jan 19, 2023
99c2c8c
Continuing work on having easy interactions with package.json data
confused-Techie Jan 19, 2023
00dd86c
Merge remote-tracking branch 'upstream/refactor-git' into refactor-git
Digitalone1 Jan 19, 2023
bbddd30
fix CodeQL issues + typeof to check "function"
Digitalone1 Jan 19, 2023
080a7b7
Merge pull request #56 from Digitalone1/refactor-git
confused-Techie Jan 19, 2023
f7480f4
timestamps set by PostgreSQL by default
Digitalone1 Jan 19, 2023
f79cbf1
fix lexical scoping in switch
Digitalone1 Jan 19, 2023
d15d39f
downloads and stars count auto-initialized by PostgreSQL to 0
Digitalone1 Jan 19, 2023
de4bdc4
Merge pull request #61 from Digitalone1/refactor-git
confused-Techie Jan 22, 2023
7590a47
Fully Tested `vcs.newPackageData`
confused-Techie Jan 23, 2023
45a5543
Fully tested `newVersionData` agnostic logic after service selection
confused-Techie Jan 23, 2023
82e1ce9
Ownership tests, removed pseudo objects
confused-Techie Jan 23, 2023
48a93b5
`deletePackageName` & `postPackageVersion` using VCS + Removed Outdat…
confused-Techie Jan 24, 2023
3848b4c
Merge branch 'main' into refactor-git
confused-Techie Feb 5, 2023
408ac9e
Fix the failing test from generic test runners
confused-Techie Feb 5, 2023
ea97df1
Utilize VCS for `deletePackagesName`
confused-Techie Feb 5, 2023
ac5bf87
Update `postPackagesVersion` use VCS auth for `deletePackagesVersion`…
confused-Techie Feb 5, 2023
ab93fcb
Better docs, publishPackage auth feature added to `vcs.ownership()`
confused-Techie Feb 5, 2023
816cf70
Slight Refactor on VCS
confused-Techie Feb 5, 2023
74861ec
Remove final usage of `git` in `package_handler.js`
confused-Techie Feb 5, 2023
53cba5f
Remove unused variable
confused-Techie Feb 5, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion jest.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,12 @@ const config = {
},
{
displayName: "Unit-Tests",
testMatch: ["<rootDir>/src/tests/*.test.js"],
testMatch: ["<rootDir>/src/tests/vcs.test.js"],
},
{
displayName: "VCS-Tests",
testMatch: ["<rootDir>/src/vcs_providers_tests/**/*.test.js"],
}
],
};

Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
"test:integration": "cross-env NODE_ENV=test PULSAR_STATUS=dev jest --selectProjects Integration-Tests",
"start:dev": "cross-env PULSAR_STATUS=dev node ./src/dev_server.js",
"test": "cross-env NODE_ENV=test PULSAR_STATUS=dev jest",
"test:vcs": "cross-env NODE_ENV=test PULSAR_STATUS=dev jest --selectProjects VCS-Tests",
"api-docs": "quick-webserver-docs -i ./src/main.js -o ./docs/reference/API_Definition.md",
"lint": "prettier --check -u -w .",
"complex": "cr --newmi --config .complexrc .",
Expand Down
62 changes: 62 additions & 0 deletions src/tests/vcs.test.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
const vcs = require("../vcs.js");

describe("determineProvider Returns as expected", () => {
test("Returns null when no input is passed", () => {
const res = vcs.determineProvider();
expect(res.type).toBe("na");
expect(res.url).toBe("");
});
test("Returns null when null input is passed", () => {
const res = vcs.determineProvider(null);
expect(res.type).toBe("na");
expect(res.url).toBe("");
});
test("Returns proper object, when object is passed", () => {
const tmp = { type: "git", url: "https://github.com/confused-Techie/atom-backend" };
const res = vcs.determineProvider(tmp);
expect(res.type).toBe(tmp.type);
expect(res.url).toBe(tmp.url);
});
test("Returns unknown VCS Object, when unkown is passed", () => {
const tmp = "https://new-vcs.com/pulsar-edit/pulsar";
const res = vcs.determineProvider(tmp);
expect(res.type).toBe("unkown");
confused-Techie marked this conversation as resolved.
Show resolved Hide resolved
expect(res.url).toBe(tmp);
});
test("Returns unkown string when passed invalid data", () => {
const tmp = 123;
const res = vcs.determineProvider(tmp);
expect(res.type).toBe("unkown");
confused-Techie marked this conversation as resolved.
Show resolved Hide resolved
expect(res.url).toBe(tmp);
});
test("Returns proper GitHub Object, passed GitHub string", () => {
const tmp = "https://github.com/confused-Techie/atom-backend";
const res = vcs.determineProvider(tmp);
expect(res.type).toBe("git");
expect(res.url).toBe(tmp);
});
test("Returns proper GitLab Object, passed GitLab string", () => {
const tmp = "https://gitlab.com/clj-editors/atom-chlorine";
const res = vcs.determineProvider(tmp);
expect(res.type).toBe("lab");
expect(res.url).toBe(tmp);
});
test("Returns proper Sourceforge Object, when passed Sourceforge string", () => {
const tmp = "https://sourceforge.net/projects/jellyfin.mirror/";
const res = vcs.determineProvider(tmp);
expect(res.type).toBe("sfr");
expect(res.url).toBe(tmp);
});
test("Returns proper Bitbucket Object, when passed Bitbucket string", () => {
const tmp = "https://bitbucket.org/docker_alpine/alpine-jellyfin/src/master/";
const res = vcs.determineProvider(tmp);
expect(res.type).toBe("bit");
expect(res.url).toBe(tmp);
});
test("Returns proper Codeberg Object, when passed Codeberg string", () => {
const tmp = "https://codeberg.org/itbastian/makemkv-move-extras";
const res = vcs.determineProvider(tmp);
expect(res.type).toBe("berg");
expect(res.url).toBe(tmp);
});
});
133 changes: 133 additions & 0 deletions src/vcs.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
/**
* @module vcs
* @desc This Module is intended to be the platform agnostic tool to interaction
* with Version Control Systems of different types in the cloud.
* To collect data from them, format it accordingly ang return it to the requesting
* function.
*/

/**
* @async
* @function ownership
* @desc Allows the ability to check if a user has permissions to write to a repo.
* <b>MUST</b> be provided the full `user` and `package` objects here to account
* for possible situations. This allows any new handling that's needed to be defined
* here rather than in multiple locations throughout the codebase.
* Returns `ok: true` where content is the repo data from the service provider on
* success, returns `ok: false` if they do not have access to said repo, with
* specificity available within the `short` key.
* @param {object} userObj - The Full User Object, as returned by the backend,
* and appended to with authorization data.
* @param {object} packObj - The full Package objects data from the backend.
* @param {object} [opts] - An optional configuration object, that allows the
* definition of non-standard options to change the fucntionality of this function.
* `opts` can accept the following parameters:
* - dev_override: {boolean} - Wether to enable or disable the dev_override. Disabled
* by default, this dangerous boolean is inteded to be used during tests that
* overrides the default safe static returns, and lets the function run as intended
* in development mode.
* @returns {object} - A Server Status object containing either minor repo data on
* success or a failure.
*/
async function ownership(userObj, packObj, opts = { dev_override: false }) {
if (
process.env.PULSAR_STATUS === "dev" &&
!dev_override &&
process.env.MOCK_GH !== "false"
) {
console.log(`git.js.ownership() Is returning Dev Only Permissions for ${user.username}`);


}
// None dev return.


}

/**
* @function determineProvider
* @desc Determines the repostiry object by the given argument.
* Takes the `repository` key of a `package.json` and with very little if not no
* desctructing will attempt to locate the provider service and return an object
* with it.
* @param {string|object} repo - The `repository` of the retrieved package.
* @returns {object} The object related to the package repository type.
*/
function determineProvider(repo) {
try {
// First party packages do already have the regular package object.
// So we will need to check if it's an object or string.
if (repo === null || repo === undefined) {
return {
type: "na",
url: ""
};
}

// If not null, it's likely a first party package
// With an already valid package object that can just be returned.
if (typeof repo === "object") {
return repo;
}

if (typeof repo !== "string") {
return {
type: "unkown",
confused-Techie marked this conversation as resolved.
Show resolved Hide resolved
url: repo
};
}

// The repo is a string, and we need to determine who the provider is.
const lcRepo = repo.toLowerCase();

switch(true) {
case lcRepo.includes("github"):
return {
type: "git",
url: repo,
};

case lcRepo.includes("bitbucket"):
return {
type: "bit",
url: repo,
};

case lcRepo.includes("sourceforge"):
return {
type: "sfr",
url: repo,
};

case lcRepo.includes("gitlab"):
return {
type: "lab",
url: repo,
};

case lcRepo.includes("codeberg"):
return {
type: "berg",
url: repo
};

default:
// If no other recognized matches exist, return repo with na service provider.
return {
type: "unkown",
url: repo,
};
}

} catch(err) {
return {
type: "na",
url: ""
};
}
}

module.exports = {
determineProvider,
ownership,
};
51 changes: 51 additions & 0 deletions src/vcs_providers/git.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
/**
* @module git
* @desc This Module ideally will support a base class that all VCS support systems can inherit.
* This Base model is needing to work against being properly useful, to helpfully cut down on
* the code needed to create future support for other services. So we will need to test.
*/

const superagent = require("superagent");
const { GH_USERAGENT } = require("../config.js").getConfig();

class Git {
constructor(opts) {
this.api_url = opts.api_url;
this.acceptable_status_codes = opts.ok_status ?? [200];
}

async _webRequestAuth(url, token) {
try {

const res = await superagent
.get(`${this.api_url}${url}`)
.set({
Authorization: `Bearer ${token}`,
})
.set({ "User-Agent": GH_USERAGENT })
// This last line here, lets the class define what HTTP Status Codes
// It will not throw an error on.
// If a status code not present in this array is received, it will throw an error.
.ok((res) => this.acceptable_status_codes.includes(res.status));

if (res.status !== 200) {
// We have not received 200 code: return a failure
return { ok: false, short: "Failed Request", content: res };
}

// The Status code is a success, return the request.
return { ok: true, content: res };

} catch(err) {
return { ok: false, short: "Exception During Web Request", content: res };
}
}

}

// In order for extends classes to properly work with VCS, they will have to export certain functions:
// * ownership
// * readme
// *

module.exports = Git;
85 changes: 85 additions & 0 deletions src/vcs_providers/github.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
/**
* @module GitHub
* @desc A VCS Module to allow `vcs.js` to interact with GitHub as needed. Utilizing `git.js`
*/

const Git = require("./git.js");

class GitHub extends Git {
constructor(opts) {

super({
api_url: opts.api_url ?? "https://api.github.com",
ok_status: [200, 401]
});
}

ownership(user, repo) {

}

async doesUserHaveRepo(token, ownerRepo, page = 1) {
try {
let check = await this._webRequestAuth(`/user/repos?page=${page}`, token);

if (!check.ok) {
if (check.short === "Failed Request") {
// This means the request failed with a non 200 HTTP Status Code.
// Looking into the error could tell us if the token is expired or etc.
switch(check.content.status) {
case 401:
return {
ok: false,
short: "Bad Auth"
};
default:
return {
ok: false,
short: "Server Error"
};
}
}
// Otherwise the short is something else. Likely a server error, and
// we will want to return a server error.
return {
ok: false,
short: "Server Error"
};
}

for (let i = 0; i < check.content.body.length; i++) {
if (check.content.body[i].full_name === ownerRepo) {
return {
ok: true,
content: check.content.body[i]
};
}
}

// After going through every repo returned, we haven't found a repo
// that the user owns. Lets check if there's multiple pages of returns.
const nextPage = page + 1;
if (res.headers.link.includes(`?page=${nextPage}`)) {
// We have another page available via the page headers
// Lets call this again with the next page
return await this.doesUserHaveRepo(token, ownerRepo, nextPage);
}

// There are no additional pages. Return that we don't have access
return {
ok: false,
short: "No Access"
};

} catch(err) {
// We encounted an exception that's not related to the webrequest.
return {
ok: false,
short: "Server Error",
content: err
};
}
}
}

module.exports = GitHub;
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
const express = require("express");
const app = express();

let port = "9999";

app.get("/user/repos", async (req, res) => {
// We will return that these are invalid credentials no matter what
res
.status(401)
.set({
Authorization: req.get("Authorization"),
"User-Agent": req.get("User-Agent"),
Link: `<localhost:${port}/user/repos?page=1>; rel="first", <localhost:${port}/user/repos?page=1>; rel="last"`,
})
.json({
message: "Requires authentication",
documentation_url: "https://docs.github.com/rest/reference/repo#list-repositories-for-the-authenticated-user"
});
});

function setPort(val) {
port = val;
}

module.exports = {
app,
setPort,
};
Loading