Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create mephisto task addons #1218

Merged
merged 4 commits into from
Jul 25, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 11 additions & 11 deletions .github/workflows/cypress-end-to-end-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ jobs:
outputs:
simple_static_task: ${{ steps.filter.outputs.simple_static_task }}
static_react_task: ${{ steps.filter.outputs.static_react_task }}
static_react_task_with_tips: ${{ steps.filter.outputs.static_react_task_with_tips }}
static_react_task_with_worker_opinion: ${{ steps.filter.outputs.static_react_task_with_worker_opinion }}
mnist: ${{ steps.filter.outputs.mnist }}
template: ${{ steps.filter.outputs.template }}
toxicity_detection: ${{ steps.filter.outputs.toxicity_detection }}
Expand Down Expand Up @@ -46,8 +46,8 @@ jobs:
- 'examples/simple_static_task/**'
static_react_task:
- 'examples/static_react_task/**'
static_react_task_with_tips:
- 'examples/static_react_task_with_tips/**'
static_react_task_with_worker_opinion:
- 'examples/static_react_task_with_worker_opinion/**'
mnist:
- 'examples/remote_procedure/mnist/**'
template:
Expand Down Expand Up @@ -305,9 +305,9 @@ jobs:
headless: true

# Learn more about this test here: https://github.com/facebookresearch/Mephisto/pull/833
static_react_task_with_tips:
static_react_task_with_worker_opinion:
needs: changes
if: ${{ (needs.changes.outputs.static_react_task_with_tips == 'true') || (needs.changes.outputs.mephisto-task == 'true') || (needs.changes.outputs.mephisto-worker-addons == 'true') || (needs.changes.outputs.abstractions == 'true') || (needs.changes.outputs.data_model == 'true') || (needs.changes.outputs.operations == 'true') || (needs.changes.outputs.tools == 'true')}}
if: ${{ (needs.changes.outputs.static_react_task_with_worker_opinion == 'true') || (needs.changes.outputs.mephisto-task == 'true') || (needs.changes.outputs.mephisto-task-addons == 'true') || (needs.changes.outputs.abstractions == 'true') || (needs.changes.outputs.data_model == 'true') || (needs.changes.outputs.operations == 'true') || (needs.changes.outputs.tools == 'true')}}
runs-on: ubuntu-latest
steps:
- name: 🔀 Checking out repo
Expand Down Expand Up @@ -353,10 +353,10 @@ jobs:
build: npm i -D cypress@11
install: false
browser: chrome
project: ./examples/static_react_task_with_tips/webapp
project: ./examples/static_react_task_with_worker_opinion/webapp
config-file: ./cypress.config.js
spec: ./examples/static_react_task_with_tips/webapp/cypress/e2e/pre_submission_tests/*
start: python examples/static_react_task_with_tips/run_task.py mephisto.task.force_rebuild=true mephisto.task.post_install_script=link_packages.sh
spec: ./examples/static_react_task_with_worker_opinion/webapp/cypress/e2e/pre_submission_tests/*
start: python examples/static_react_task_with_worker_opinion/run_task.py mephisto.task.force_rebuild=true mephisto.task.post_install_script=link_packages.sh
wait-on: "http://localhost:3000/?worker_id=x&assignment_id=1"
headless: true

Expand Down Expand Up @@ -426,10 +426,10 @@ jobs:
with:
build: npm i -D cypress@11
install: false
project: ./examples/static_react_task_with_tips/webapp
project: ./examples/static_react_task_with_worker_opinion/webapp
config-file: cypress.config.js
spec: ./examples/static_react_task_with_tips/webapp/cypress/e2e/post_submission_tests/*
start: python examples/static_react_task_with_tips/run_task.py mephisto.task.post_install_script=link_packages.sh mephisto.task.force_rebuild=true
spec: ./examples/static_react_task_with_worker_opinion/webapp/cypress/e2e/post_submission_tests/*
start: python examples/static_react_task_with_worker_opinion/run_task.py mephisto.task.post_install_script=link_packages.sh mephisto.task.force_rebuild=true
wait-on: "http://localhost:3000/?worker_id=x&assignment_id=1"
browser: chrome
headless: true
2 changes: 0 additions & 2 deletions docker/docker-compose.dev.vscode.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,6 @@
# This source code is licensed under the MIT license found in the
# LICENSE file in the root directory of this source tree.

version: "3"

services:
mephisto_dc_vscode:
container_name: mephisto_dc_vscode
Expand Down
2 changes: 0 additions & 2 deletions docker/docker-compose.dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,6 @@
# This source code is licensed under the MIT license found in the
# LICENSE file in the root directory of this source tree.

version: "3"

services:
mephisto_dc:
container_name: mephisto_dc
Expand Down
3 changes: 2 additions & 1 deletion docs/web/docs/explanations/architecture_overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,8 @@ This is a quick guide over file directories in Mephisto project. Note that some
- `webpack.config.js` - webpack config for React app containing FC
- `webpack.config.presigned_urls.js` - webpack config for React app containing FC with presigned S3 URLs feature
- `webpack.config.review.js` - webpack config for React app displayin read-only task page inside iframe (for TaskReview app)
- `static_react_task_with_tips` - deprecated example (current code uses "bonuses" and "feedback" for disambiguation instead of "tips")
- `static_react_task` - simple React application (with Onboarding and Screening)
- `static_react_task_with_worker_opinion` - simple React application with Worker Opinion widget
- `simple_static_task` - deprecated example (Mephisto is moving away from HTML templates as its tooling mostly supports React-based tasks)
- `other examples` - older Mephisto examples (working, but not the best code structure)
- `hydra_configs` - settings for Hydra (Mephisto's configuration tool)
Expand Down
6 changes: 6 additions & 0 deletions docs/web/docs/guides/how_to_use/review_app/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,3 +69,9 @@ _Note that a custom view of Task results is included (at the bottom) only if you
![Task statistics](./screenshots/task_stats.png)
<br/>
<br/>

### Task worker opinions

![Task statistics](./screenshots/task_worker_opinions.png)
<br/>
<br/>
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
48 changes: 47 additions & 1 deletion docs/web/docs/guides/how_to_use/review_app/server_api.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ Assemble stats with results for a Task.

---

### `GET /api/tasks/{id}/charts`
### `GET /api/tasks/{id}/timeline`

Check if Grafana server is available and redirect or return error.

Expand All @@ -128,6 +128,51 @@ Check if Grafana server is available and redirect or return error.

---

### `GET /api/tasks/{id}/worker-opinions`

Returns all Worker Opinions related to a Task.

```
{
"task_name": <str>,
"worker_opinions": [
{
"data": {
"attachments": [
{
"destination": <str>,
"encoding": <str>,
"fieldname": <str>,
"filename": <str>,
"mimetype": <str>,
"originalname": <str>,
"path": <str>,
"size": <int>
},
... // more attachments
],
"questions": [
{
"answer": <str>,
"id": <str>,
"question": <str>,
"reviewed": <bool>,
"toxicity": <str> | null
},
... // more questions
]
},
"unit_data_folder": <str>,
"unit_id": <str>,
"worker_id": <str>
},
... // more worker opinions
]
}
```

---

### `GET /api/tasks/{id}/worker-units-ids`

Get full, unpaginated list of unit IDs within a task (for subsequent client-side grouping by worker_id and `GET /task-units` pagination)
Expand Down Expand Up @@ -264,6 +309,7 @@ Get full input for specified workers results (`units_ids` parameter is mandatory
"has_task_source_review": <bool>,
"id": <int>,
"inputs": <json object>, // instructions for worker
"metadata": <json object>, // any metadata (e.g. Worker Opinion)
"outputs": <json object>, // response from worker
"prepared_inputs": <json object>, // prepared instructions from worker
"unit_data_folder": <str>}, // path to data dir in file system
Expand Down
104 changes: 0 additions & 104 deletions docs/web/docs/guides/how_to_use/worker_experience/feedback.md

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
---

# Copyright (c) Meta Platforms and its affiliates.
# This source code is licensed under the MIT license found in the
# LICENSE file in the root directory of this source tree.

sidebar_position: 1
---

# Mephisto Task Addons

## Overview

The `mephisto-task-addons` package provides:
- `WorkerOpinion` widget: collect workers' feedback for each completed unit

## Usage

1. Add `mephisto-task-addons` library to your webpack config:
```js
// Specifies location of your packages (e.g. `../../dir`)
var PATH_TO_PACKAGES = "<path>"

module.exports = {
...
resolve: {
alias: {
...
"mephisto-task-addons": path.resolve(
__dirname,
`${PATH_TO_PACKAGES}/packages/mephisto-task-addons`
),
}
}
};
```

2. Import desired widgets from `mephisto-task-addons` in your code like so:

```jsx
import { WorkerOpinion } from "mephisto-task-addons";
...
<WorkerOpinion
maxTextLength={500}
questions={[
"Was this task hard?",
"Is this a good example?",
]}
/>
```

This file was deleted.

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading