Skip to content

Commit

Permalink
feat: redesigned plugin action management, replacing "templates" with…
Browse files Browse the repository at this point in the history
… "batches"; resolves #15 (#51)

* progress: break a bunch of stuff for actionmanager refactor

* progress: more refactor changes

* progress: more refactoring

* progress: compiling and sort-of-functioning multi-threaded batches

* progress: make batch creation easier and more flexible in the GUI

* test: convert all tests to use new batch system

* progress: update cli options for GUI

* progress: finish replacing old template references and capabilities
  • Loading branch information
hello-adam authored Jun 26, 2020
1 parent cec3cd6 commit 78da500
Show file tree
Hide file tree
Showing 62 changed files with 2,029 additions and 1,867 deletions.
6 changes: 3 additions & 3 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,11 +115,11 @@ Importer/Exporters are the primary way to get bitstreams in and out of hobbits.
- TCP Data: import bits by listening for TCP data on a specified port, or
export bits to a TCP server at a specified address

## Templates
## Batches

When a user develops a sequence of operations that they want to perform
frequently, they can save it as a "Template" and perform it later with a single
step. Templates can also be used to process bits on the command line with the
frequently, they can save it as a "Batch" and perform it later with a single
step. Batches can also be used to process bits on the command line with the
hobbits-runner CLI.

## Technical Architecture
Expand Down
2 changes: 1 addition & 1 deletion docs/plugin-developer-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ The Analyzer and Operator interfaces have several simple methods in common:
The plugin state contains parameters for how the plugin should run. For example,
the Take Skip operator plugin will require a "take_skip_string" parameter to
guide its execution. The state that it passed in can be provided by the plugin's
own `getStateFromUi` method, or it can come from a saved template that was
own `getStateFromUi` method, or it can come from a saved batch that was
generated from a state returned by a plugin result (`OperatorResult` or
`AnalyzerResult`.) Plugin results will usually just return the state that was
passed in (e.g. `AnalyzerResult::result(bitInfo, recallablePluginState)`,) but
Expand Down
10 changes: 9 additions & 1 deletion docs/user-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,13 +104,21 @@ Importer/Exporters are the primary way to get bitstreams in and out of hobbits.
For example, if you want to open a file as bits, selecting the `File Data`
import plugin will let you select a file and then load it in as a Bit Container.

## Batch Execution

Sequences of operator, analyzer, and importer actions can be saved as "batches"
that can be executed all at once. You can create a batch by selecting either
the container that starts the sequence or the container that ends the sequence
and then navigating to `File->Save Batch...`. The saved batch can later be
executed via `File->Apply Batch...`.

## Command line execution

When running Hobbits from the command line, a variety of configuration and
data loading options are provided. Simply run it with the `--help` option to see
which options are available.

The hobbits-runner program can be used to apply a saved Hobbits template to an
The hobbits-runner program can be used to apply a saved Hobbits batch to an
input file without needing to open the GUI.

## Plugin loading
Expand Down
110 changes: 0 additions & 110 deletions src/hobbits-core/analyzeractor.cpp

This file was deleted.

13 changes: 13 additions & 0 deletions src/hobbits-core/analyzerresult.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,19 @@ const QJsonObject AnalyzerResult::getPluginState() const
return m_pluginState;
}

bool AnalyzerResult::hasEmptyState() const
{
return m_pluginState.isEmpty();
}

QString AnalyzerResult::errorString() const
{
if (m_pluginState.contains("error")) {
return m_pluginState.value("error").toString();
}
return QString();
}

QSharedPointer<const AnalyzerResult> AnalyzerResult::result(QSharedPointer<BitInfo> bitInfo, QJsonObject pluginState)
{

Expand Down
3 changes: 3 additions & 0 deletions src/hobbits-core/analyzerresult.h
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,9 @@ class HOBBITSCORESHARED_EXPORT AnalyzerResult
AnalyzerResult* setPluginState(QJsonObject pluginState);
const QJsonObject getPluginState() const;

bool hasEmptyState() const;
QString errorString() const;

static QSharedPointer<const AnalyzerResult> result(QSharedPointer<BitInfo> bitInfo, QJsonObject pluginState);
static QSharedPointer<const AnalyzerResult> error(QString error);

Expand Down
147 changes: 147 additions & 0 deletions src/hobbits-core/analyzerrunner.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
#include "analyzerrunner.h"
#include "pluginaction.h"
#include "pluginactionlineage.h"
#include "pluginactionmanager.h"
#include "settingsmanager.h"

QSharedPointer<AnalyzerRunner> AnalyzerRunner::create(
QSharedPointer<const HobbitsPluginManager> pluginManager,
QSharedPointer<const PluginAction> action)
{
if (action->getPluginType() != PluginAction::Analyzer) {
return nullptr;
}
auto plugin = pluginManager->getAnalyzer(action->getPluginName());
if (!plugin) {
return nullptr;
}

auto runner = QSharedPointer<AnalyzerRunner>(new AnalyzerRunner());

runner->m_id = QUuid::createUuid();
runner->m_analyzer = plugin;
runner->m_action = action;
runner->m_pluginFileLocation = pluginManager->getPluginLocation(plugin->getName());

return runner;
}

QUuid AnalyzerRunner::id() const
{
return m_id;
}

QSharedPointer<ActionWatcher<QSharedPointer<const AnalyzerResult>>> AnalyzerRunner::getWatcher()
{
return m_actionWatcher;
}

QSharedPointer<BitContainer> AnalyzerRunner::container() const
{
return m_container;
}

QSharedPointer<ActionWatcher<QSharedPointer<const AnalyzerResult>>> AnalyzerRunner::run(QSharedPointer<BitContainer> container)
{

if (!m_actionWatcher.isNull() && m_actionWatcher->watcher()->future().isRunning()) {
emit reportError(m_id, QString("Analyzer runner is already running"));
return QSharedPointer<ActionWatcher<QSharedPointer<const AnalyzerResult>>>();
}

auto pluginState = m_action->getPluginState();
if (pluginState.isEmpty()) {
pluginState = m_analyzer->getStateFromUi();
if (pluginState.isEmpty() || pluginState.contains("error")) {
if (pluginState.contains("error")) {
emit reportError(m_id, QString("Plugin '%1' reported an error with its current state: '%2'").arg(
m_analyzer->getName()).arg(pluginState.value("error").toString()));
}
else if (pluginState.isEmpty()) {
emit reportError(m_id, QString(
"Plugin '%1' is in an invalid state and can't be executed. Double check the input fields.").arg(
m_analyzer->getName()));
}
return QSharedPointer<ActionWatcher<QSharedPointer<const AnalyzerResult>>>();
}
}

QSharedPointer<ActionProgress> progress(new ActionProgress());

QVariant previousRunning = SettingsManager::getInstance().getPrivateSetting(SettingsData::PLUGINS_RUNNING_KEY);
QStringList runningPlugins;
if (previousRunning.isValid() && previousRunning.canConvert<QStringList>()) {
runningPlugins = previousRunning.toStringList();
}
runningPlugins.append(m_pluginFileLocation);
SettingsManager::getInstance().setPrivateSetting(SettingsData::PLUGINS_RUNNING_KEY, QVariant(runningPlugins));

auto future = QtConcurrent::run(
QThreadPool::globalInstance(),
AnalyzerRunner::analyzerCall,
m_analyzer,
container,
pluginState,
progress);

m_actionWatcher = QSharedPointer<ActionWatcher<QSharedPointer<const AnalyzerResult>>>(
new ActionWatcher<QSharedPointer<const AnalyzerResult>>(
future,
progress));

m_container = container;

connect(m_actionWatcher->watcher(), SIGNAL(finished()), this, SLOT(postProcess()));
connect(m_actionWatcher->progress().data(), &ActionProgress::progressPercentChanged, [this](int progress) {
this->progress(m_id, progress);
});

return m_actionWatcher;
}

void AnalyzerRunner::postProcess()
{
disconnect(m_actionWatcher->watcher(), SIGNAL(finished()), this, SLOT(postProcess()));
disconnect(m_actionWatcher->progress().data(), &ActionProgress::progressPercentChanged, nullptr, nullptr);

QSharedPointer<const AnalyzerResult> result = m_actionWatcher->watcher()->future().result();

if (result.isNull()) {
QString errorString = QString("Plugin '%1' failed to execute. Double check the input fields.").arg(
m_analyzer->getName());
emit reportError(m_id, errorString);
emit finished(m_id);
return;
}

if (result->getPluginState().contains("error")) {
QString errorString = QString("Plugin '%1' reported an error with its processing: %2").arg(m_analyzer->getName()).arg(
result->getPluginState().value("error").toString());
emit reportError(m_id, errorString);
emit finished(m_id);
return;
}

QSharedPointer<PluginAction> action =
QSharedPointer<PluginAction>(
new PluginAction(
PluginAction::Analyzer,
m_analyzer->getName(),
result->getPluginState()));
PluginActionLineage::recordLineage(action, {m_container}, {m_container});

if (!result->bitInfo().isNull()) {
m_container->setBitInfo(result->bitInfo());
}

emit finished(m_id);
}

QSharedPointer<const AnalyzerResult> AnalyzerRunner::analyzerCall(
QSharedPointer<AnalyzerInterface> analyzer,
QSharedPointer<const BitContainer> bits,
QJsonObject pluginState,
QSharedPointer<ActionProgress> progressTracker)
{
return analyzer->analyzeBits(bits, pluginState, progressTracker);
}
Loading

0 comments on commit 78da500

Please sign in to comment.