Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Discovery service based on proto endpoint #30

Merged
merged 8 commits into from
Nov 16, 2017
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 18 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,25 @@ export const environment = {

Run `npm start` for a dev server. Navigate to `http://localhost:4200/`. The app will automatically reload if you change any of the source files.

## Proto files
## Protocol buffers

The JS proto code will be generated autocatically on the build command, if you want to control this step you can generate the JS code and TS definitions by running `npm run protodef`, which will search proto files in the `proto` directory, and will generate the code at `src/app/shared/proto` path. Also you can use the build script with `node build-scripts/gen-proto-code.js -i <proto_source_dir> -o <destination_dir>` or `npm run protodef:custom -- -i <proto_source_dir> -o <destination_dir>`.
### Proto files fetching

Right now two proto file locations are defined and specified as so on the project tasks through npm scripts, these are the `microservices` proto and `metrics` proto. They can be fetched by running `npm run protofetch:microservices` and `npm run protofetch:metrics`, running `npm run protofetch` will get you both. All these commands will get you the proto files at the `proto` path.

Also you can use the build script with `node build-scripts/fetch-proto.js -s <proto-file-source-url> -o <destination-dir>` or `npm run protofetch:custom -- -i <proto-file-source-url> -o <destination-dir>`.

`node build-scripts/fetch-proto.js --help` could help you on how to use the script.

### Proto code generation

If you want to control this step further, you can generate the JS code and TS definitions by running `npm run protodef`, which will search proto files in the `proto` directory, and will generate the code at `src/app/shared/proto` path.

This requires the protocol buffers compiler to be already installed on the system. On GNU/Linux you can probably [manage](https://launchpad.net/ubuntu/+source/protobuf) this [through](https://launchpad.net/%7Emaarten-fonville/+archive/ubuntu/protobuf) some [package manager](https://pkgs.org/download/protobuf-compiler), or equivalent in [other OS](http://brewformulas.org/Protobuf). Otherwise, you can check Google [Protocol Buffers releases](https://github.com/google/protobuf/releases), and look for the `protoc` precompiled zipped binaries, extract and include them in your `PATH`, to have available the `protoc` command.

Also you can use the build script with `node build-scripts/gen-proto-code.js -i <proto-source-dir> -o <destination-dir>` or `npm run protodef:custom -- -i <proto-source-dir> -o <destination-dir>`.

`node build-scripts/gen-proto-code.js --help` could help you on how to use the script.

## Code scaffolding

Expand Down
116 changes: 116 additions & 0 deletions build-scripts/fetch-proto.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
#! /usr/bin/env node
const package = require('../package.json');
const path = require('path');
const fs = require('fs');
const { URL } = require('url');
const axios = require('axios');
const yargs = require('yargs');
const shell = require('shelljs');
const ora = require('ora');

// Set the directory from where we will be working
const workingDir = path.join(__dirname, '..');

// Default URL path from where proto file will fetched (could be changed by params)
let originProtoPath = 'http://localhost:8080/proto/models/microservices';
// Fetched proto file destination (could be changed by params)
let destProtoPath = path.join(workingDir, 'proto');

// Reading params
const argv = yargs
.usage('fetch-proto\n\nUsage: $0 [options]')
.help('help').alias('help', 'h')
.version(package.version).alias('version', 'v')
.options({
source: {
alias: ['s'],
description: '<proto-file-source-url>',
requiresArg: true,
default: originProtoPath,
string: true,
coerce: arg => {
try {
const urlObject = new URL(arg);
return urlObject;
} catch (e) {
// throw new Error(`Error: ${arg} is not a valid URL path`);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove comment

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adressed on 59ef685

throw e;
}
}
},
output: {
alias: 'o',
description: '<output-destination-path>',
requiresArg: true,
default: destProtoPath,
string: true,
normalize: true,
coerce: arg => {
if (fs.existsSync(arg)) {
return arg;
}
else {
throw new Error(`Error: ${arg} is not a valid directory path`);
}
}
}
})
.fail((msg, err, yargs) => {
shell.echo(msg);
process.exit(1);
})
.wrap(Math.min(120, yargs.terminalWidth()))
.argv;

originProtoPath = new URL(argv.source);
destProtoPath = argv.output;


function fetchProto () {
const spinner = new ora({
spinner: 'dots3',
interval: 60,
color: 'magenta',
text: `${originProtoPath} fetching...`
}).start();
// GET request for remote proto
return axios({
method: 'GET',
url: originProtoPath.href,
headers: { 'Accept': 'text/plain'},
responseType: 'text',
timeout: 5000
})
.then((response) => {
const protoFilename = `${originProtoPath.pathname.match(/([^\/]*)\/*$/)[1]}.proto`;
fs.writeFileSync(path.join(destProtoPath, `${protoFilename}`), response.data);
const successMessage = `${protoFilename} fetched`;
spinner.succeed(successMessage);
return successMessage;
})
.catch((error) => {
spinner.fail(`Error connecting to: ${error.config.url}`);
// shell.echo(`Error connecting to: ${error.config.url}`);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same above

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adressed on 59ef685

if (error.response) {
// The request was made and the server responded with a status code out of range 2xx
shell.echo(`${error.response.status} - ${error.response.statusText}`);
shell.echo(error.response.data);
}
else if (error.request) {
// The request was made but no response was received
// `error.request` is an instance of http.ClientRequest
shell.echo(`Destination can’t be reached`);
}
else {
// Something happened in setting up the request that triggered an Error
shell.echo(error.message);
}
throw error;
});
}

try {
fetchProto();
} catch (e) {}

module.exports.fetchProto = fetchProto;
27 changes: 19 additions & 8 deletions build-scripts/gen-proto-code.js
Original file line number Diff line number Diff line change
Expand Up @@ -71,16 +71,23 @@ const funcList = shell.ls('-l', path.join(originProtoPath, '*.proto'))

const protoname = path.basename(file.name, path.extname(file.name));

const jsCodeFilePath = path.join(destCodePath, `${protoname}.js`);
const tsCodeFilePath = path.join(destCodePath, `${protoname}.d.ts`);

const execCommand = `pbjs -t static-module -o ${jsCodeFilePath} -w commonjs ${file.name} &
pbts -o ${tsCodeFilePath} ${jsCodeFilePath}`;
const execCommand = `protoc\
--plugin=protoc-gen-ts=${workingDir}/node_modules/.bin/protoc-gen-ts\
--js_out=import_style=commonjs,binary:${destCodePath}\
--ts_out=service=true:${destCodePath}\
--proto_path=${originProtoPath}\
${file.name}`;

const processProto = () => {
return new Promise((resolve, reject) => {
const spinner = new ora(`Processing ${protoname}`).start();
shell.exec(execCommand, { async: true }, code => code ? reject(spinner.fail()) : resolve(spinner.succeed()));
const spinner = new ora({
spinner: 'dots2',
interval: 60,
color: 'magenta',
text: `${protoname} processing...`
}
).start();
shell.exec(execCommand, { async: true }, code => code ? reject(spinner.fail(`${protoname} processed`)) : resolve(spinner.succeed(`${protoname} processed`)));
});
};

Expand All @@ -94,7 +101,11 @@ async function run () {
if (funcList.length > 0) {
shell.echo(`${funcList.length} proto files found`);
for (const func of funcList) {
await func();
try {
await func();
} catch (e) {
shell.echo('Code generation operation couldn\'t be completed');
}
}
shell.echo(`Generated code stored in ${path.join(destCodePath)}`);
}
Expand Down
Loading