Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

elasticsearch history api #1682 #1725

Merged
merged 16 commits into from
Aug 14, 2019
Merged

elasticsearch history api #1682 #1725

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
10a382f
serve get_account_history with elasticsearch plugin
oxarbitrage Apr 20, 2019
00496a9
replace IdToString
oxarbitrage Apr 20, 2019
2a6d7b2
add is_plugin_enabled to application
oxarbitrage Jul 15, 2019
1737d45
add elasticsearch-mode command for save, query or all
oxarbitrage Jul 15, 2019
6dda9fe
execute elasticsearch account_history in a new thread
oxarbitrage Jul 16, 2019
26e9f16
simplify conditional
oxarbitrage Jul 31, 2019
4fac171
capture only what is necessary in lambda
oxarbitrage Jul 31, 2019
29bf975
wrap command line option long lines
oxarbitrage Jul 31, 2019
de76301
make mode an enum for comparison performance
oxarbitrage Jul 31, 2019
5838a38
throw when mode::all if elasticsearch-operation-string is false
oxarbitrage Jul 31, 2019
1f6ac14
change ES query
oxarbitrage Jul 31, 2019
63f7aff
remove range from second ES query
oxarbitrage Jul 31, 2019
a7fc6b0
add get_running_mode() to plugin and use it in api, fix capture
oxarbitrage Jul 31, 2019
13d8034
Merge branch 'develop' into issue1682
oxarbitrage Jul 31, 2019
07de45e
create only 1 thread to serve ES queries
oxarbitrage Aug 1, 2019
f1bedc7
Merge branch 'issue1682' of https://github.com/oxarbitrage/bitshares-…
oxarbitrage Aug 1, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 9 additions & 6 deletions libraries/plugins/elasticsearch/elasticsearch_plugin.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ class elasticsearch_plugin_impl
bool _elasticsearch_operation_object = false;
uint32_t _elasticsearch_start_es_after_block = 0;
bool _elasticsearch_operation_string = true;
std::string _elasticsearch_mode = "only_save";
mode _elasticsearch_mode = mode::only_save;
CURL *curl; // curl handler
vector <string> bulk_lines; // vector of op lines
vector<std::string> prepare;
Expand Down Expand Up @@ -446,8 +446,8 @@ void elasticsearch_plugin::plugin_set_program_options(
"Start doing ES job after block(0)")
("elasticsearch-operation-string", boost::program_options::value<bool>(),
"Save operation as string. Needed to serve history api calls(true)")
("elasticsearch-mode", boost::program_options::value<std::string>(),
"Mode of operation: only_save, only_query, all(only_save)")
("elasticsearch-mode", boost::program_options::value<uint16_t>(),
"Mode of operation: only_save(0), only_query(1), all(2) - Default: 0")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd use values 1,2,3 so the option behaves like a bitset. But that's just me.

;
cfg.add(cli);
}
Expand Down Expand Up @@ -485,11 +485,14 @@ void elasticsearch_plugin::plugin_initialize(const boost::program_options::varia
my->_elasticsearch_operation_string = options["elasticsearch-operation-string"].as<bool>();
}
if (options.count("elasticsearch-mode")) {
my->_elasticsearch_mode = options["elasticsearch-mode"].as<std::string>();
const auto option_number = options["elasticsearch-mode"].as<uint16_t>();
if(option_number > mode::all)
FC_THROW_EXCEPTION(fc::exception, "Elasticsearch mode not valid");
my->_elasticsearch_mode = static_cast<mode>(options["elasticsearch-mode"].as<uint16_t>());
}

if(my->_elasticsearch_mode != "only_query") {
if (my->_elasticsearch_mode == "all")
if(my->_elasticsearch_mode != mode::only_query) {
if (my->_elasticsearch_mode == mode::all)
my->_elasticsearch_operation_string = true;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Silently overriding this will cause confusion.
IMO add log output if user has configured this to false. Or perhaps fail to make this very explicit.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, i am now not allowing the mode to be all if operation_string is false so no silent changes are made.

5838a38


database().applied_block.connect([this](const signed_block &b) {
Expand Down
3 changes: 3 additions & 0 deletions libraries/plugins/elasticsearch/include/graphene/elasticsearch/elasticsearch_plugin.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,8 @@ class elasticsearch_plugin : public graphene::app::plugin
graphene::utilities::ES prepareHistoryQuery(string query);
};

enum mode { only_save = 0 , only_query = 1, all = 2 };

struct operation_visitor
{
typedef void result_type;
Expand Down Expand Up @@ -302,6 +304,7 @@ struct adaptor_struct {

} } //graphene::elasticsearch

FC_REFLECT_ENUM( graphene::elasticsearch::mode, (only_save)(only_query)(all) )
FC_REFLECT( graphene::elasticsearch::operation_history_struct, (trx_in_block)(op_in_trx)(operation_result)(virtual_op)(op)(op_object) )
FC_REFLECT( graphene::elasticsearch::block_struct, (block_num)(block_time)(trx_id) )
FC_REFLECT( graphene::elasticsearch::fee_struct, (asset)(asset_name)(amount)(amount_units) )
Expand Down