This repository has been archived by the owner on Aug 18, 2021. It is now read-only.
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* WIP commit - Added @code.gov integrations package. - Added initial code to handle the data returned from Github Integration. - Need to finish what data is going to be indexed and how it will change the repo object. * Renamed mapping_201.json to mapping - Updated mapping require statements - Delete unused mapping files - updated mapping in test * Reverted github integration and added a new remoteVcs field to indicate what remote vcs is being used. * Fix linting errors * Changing the search properties from string to keyword and text - This makes the properties type compatible with Elasticsearch 5.6+ and 6.x+ * WIP commit * WIP commit - Added @code.gov integrations package. - Added initial code to handle the data returned from Github Integration. - Need to finish what data is going to be indexed and how it will change the repo object. * Reverted github integration and added a new remoteVcs field to indicate what remote vcs is being used. * WIP commit * WIP commit - Added @code.gov integrations package. - Added initial code to handle the data returned from Github Integration. - Need to finish what data is going to be indexed and how it will change the repo object. * Reverted github integration and added a new remoteVcs field to indicate what remote vcs is being used. * Changing the search properties from string to keyword and text - This makes the properties type compatible with Elasticsearch 5.6+ and 6.x+ * WIP commit * Added use of code-gov-adapters * Added use of code-gov-adapters * Fixed format of empty dates and added new version of code-gov-adapters * Updating this AGAIN!!!!!!!! * Swapped out us of old elasticsearch adpater for @code.gov/code-gov-adapters library * Work in progress commit - Added dataScoreNormalizer * WIP commit * Grouped issues by project instead of issue type * Tweaked styles for row by issue type * WIP refactor to use code-gov-adapter in the routes. - General code clean up - Refactored and simplyfied helper functions * WIP update of code-gov-adapters * Uncommented version and root routes * Cleaned up require statements and refactored languages endpoint * Updated dependencies * Add logger to ElasticsearchAdpater initialization * Refactor to us our new elasticsearch adapter - AliasSwapper does not extend AbstractIndexTool anymore * Added logger instance * Refactored to use new Elasticsearch adapter * New dependency version for gsa/code-gov-adapters * Refactor - Removed unused require statements - Removed ElasticSearchLogger. - IndexOptimizer is no longer extending AbstractIndexTool. - Refactored forceMerge to use async await. - Refactored init function to use async await. * Code refactor - Removed the use of the async package - Changed the elastisearch adapter to use the package @code.gov/code-gov-adapter - Removed getter function - Fixed and cleaned JS Docs - Cleaned up cleanIndicesForAlias to use async/await properly. - Refactored init to use async/await * Code refactor - Added _buildActions helper function - Cleaned up JS Docs - Fixed getIndexesForAlias bug. The function was not returning the indices array. - removed the use of the async package. * Code refactor - Removed the use of the async package - made the RepoIndexer init function use async/await * Added callback and request timeout - Added callback to repo indexer to continue with indexing process - Added request timeout param to indexDocument function in the repo indexer. * Eslint fix * WIP commit * Add size parameter to search stream - Added default size value to constructor - Added size to searechQuery - Changed pagination (from + size) to use the results count. * Refactor init function - Changed init function to use async/await - Removed promise chain code - Changed body parameter to document for _indexTerm * Cosmetic changes * Refactor init function - Changed init function to use async/await - Removed promise chain code - Removed async package from required statements * Cosmetic changes * Refactored deleteIndices - Added param destructuring - added requestTimeout with a default of 30000 * Bug fix and function refactor - Refactored getIndexesForAlias to return the results directly from the adapter. - Fixed a bug in the init fucntion that would not execute the alias swaps when no indices where found for the alias supplied * Refactor function - Refactored indexDocument to use param destructuring * function refactoring - Refactored index function to use async/await and remove the use of the async package - Removed the need to use callbacks in the index function. - Module executiong code now used promises instead of callbacks * function refactoring - Refactored index function to use async/await - Removed the need to use callbacks in the index function. - Module executiong code now used promises instead of callbacks * function refactoring - Refactored index function to use async/await - Removed the need to use callbacks in the index function. - Module executiong code now used promises instead of callbacks - Removed the use of the async package. * added score field to mappings. This fixes a sorting bug/error from Elasticsearch * Update code-gov-adapters dependency * Updated error handling for routes. - Implemented a kind of circuit breaker to bubble errors up to the error handler middleware * Fixed github http regular expression * Added github integration * Update code-gov-integratios dependency * Moved logger initialization - Logger now initializes after the cofig object - Logger now has the logger level passed to the constructor * Added logger level to logger constructor * Added logger level to logger constructor * Added logger level to logger constructor * Changed code-gov-integrations to use github url for now * Elasticsearch adapter in repos indexer is now a reference to the class not an instace * Added adapter instantiation to constructor * Added esHost to esParams * Added esHost to esParams * Added score normalizer * WIP commit - Added @code.gov integrations package. - Added initial code to handle the data returned from Github Integration. - Need to finish what data is going to be indexed and how it will change the repo object. * WIP commit - Added @code.gov integrations package. - Added initial code to handle the data returned from Github Integration. - Need to finish what data is going to be indexed and how it will change the repo object. * Reverted github integration and added a new remoteVcs field to indicate what remote vcs is being used. * Reverted github integration and added a new remoteVcs field to indicate what remote vcs is being used. * Changing the search properties from string to keyword and text - This makes the properties type compatible with Elasticsearch 5.6+ and 6.x+ * WIP commit * WIP commit * Added use of code-gov-adapters * Added use of code-gov-adapters * Fixed format of empty dates and added new version of code-gov-adapters * WIP commit - Added @code.gov integrations package. - Added initial code to handle the data returned from Github Integration. - Need to finish what data is going to be indexed and how it will change the repo object. * Renamed mapping_201.json to mapping - Updated mapping require statements - Delete unused mapping files - updated mapping in test * Reverted github integration and added a new remoteVcs field to indicate what remote vcs is being used. * Fix linting errors * WIP commit * Updating this AGAIN!!!!!!!! * Swapped out us of old elasticsearch adpater for @code.gov/code-gov-adapters library * Work in progress commit - Added dataScoreNormalizer * WIP commit * Grouped issues by project instead of issue type * Tweaked styles for row by issue type * WIP refactor to use code-gov-adapter in the routes. - General code clean up - Refactored and simplyfied helper functions * WIP update of code-gov-adapters * Uncommented version and root routes * Cleaned up require statements and refactored languages endpoint * Updated dependencies * Add logger to ElasticsearchAdpater initialization * Refactor to us our new elasticsearch adapter - AliasSwapper does not extend AbstractIndexTool anymore * Added logger instance * Refactored to use new Elasticsearch adapter * New dependency version for gsa/code-gov-adapters * Refactor - Removed unused require statements - Removed ElasticSearchLogger. - IndexOptimizer is no longer extending AbstractIndexTool. - Refactored forceMerge to use async await. - Refactored init function to use async await. * Code refactor - Removed the use of the async package - Changed the elastisearch adapter to use the package @code.gov/code-gov-adapter - Removed getter function - Fixed and cleaned JS Docs - Cleaned up cleanIndicesForAlias to use async/await properly. - Refactored init to use async/await * Code refactor - Added _buildActions helper function - Cleaned up JS Docs - Fixed getIndexesForAlias bug. The function was not returning the indices array. - removed the use of the async package. * Code refactor - Removed the use of the async package - made the RepoIndexer init function use async/await * Added callback and request timeout - Added callback to repo indexer to continue with indexing process - Added request timeout param to indexDocument function in the repo indexer. * Eslint fix * WIP commit * Add size parameter to search stream - Added default size value to constructor - Added size to searechQuery - Changed pagination (from + size) to use the results count. * Refactor init function - Changed init function to use async/await - Removed promise chain code - Changed body parameter to document for _indexTerm * Cosmetic changes * Refactor init function - Changed init function to use async/await - Removed promise chain code - Removed async package from required statements * Cosmetic changes * Refactored deleteIndices - Added param destructuring - added requestTimeout with a default of 30000 * Bug fix and function refactor - Refactored getIndexesForAlias to return the results directly from the adapter. - Fixed a bug in the init fucntion that would not execute the alias swaps when no indices where found for the alias supplied * Refactor function - Refactored indexDocument to use param destructuring * function refactoring - Refactored index function to use async/await and remove the use of the async package - Removed the need to use callbacks in the index function. - Module executiong code now used promises instead of callbacks * function refactoring - Refactored index function to use async/await - Removed the need to use callbacks in the index function. - Module executiong code now used promises instead of callbacks * function refactoring - Refactored index function to use async/await - Removed the need to use callbacks in the index function. - Module executiong code now used promises instead of callbacks - Removed the use of the async package. * added score field to mappings. This fixes a sorting bug/error from Elasticsearch * Update code-gov-adapters dependency * Updated error handling for routes. - Implemented a kind of circuit breaker to bubble errors up to the error handler middleware * Added github integration * Update code-gov-integratios dependency * Moved logger initialization - Logger now initializes after the cofig object - Logger now has the logger level passed to the constructor * Added logger level to logger constructor * Added logger level to logger constructor * Added logger level to logger constructor * Changed code-gov-integrations to use github url for now * Elasticsearch adapter in repos indexer is now a reference to the class not an instace * Added adapter instantiation to constructor * Added esHost to esParams * Added esHost to esParams * Added score normalizer * Added logger to score normalization * Added helper functions * Added getting log level from env * Removed github integration from normal havesting - Moved github integration code to a separate file. - Added a step to the indexing process. Github data is now fetched after score normalization * Work in progress commit. * Added config and proper adapter constructor code * Added esHost parameter to elasticsearch params * Added repoIndexInfo to getGithubData - This will ensure that the correct index is being used * Removed integrations and encoding - Added code to handle utf16le encoding. For the moment this was only present in DOJ code.json * Added repoIndexInfo to normalizeRepoScores - This will ensure that the correct index is being used - Also added the use of the index to update and search functions * New code.json files form agencies. These will be used as our fallback data * Added dependencies * Fix github data fetch bug - Code was using the incorrect fields from the integrations package - Uncommented getGithubData function * Changing return fields on routes to match the past
- Loading branch information