Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds a tips example #791

Merged
merged 108 commits into from
Jul 14, 2022
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
108 commits
Select commit Hold shift + click to select a range
cf68dcc
✨ Added get_tip_by_task_name & new_tip method
Etesam913 Jun 2, 2022
c9d9aff
✨ Added get_tip() method
Etesam913 Jun 2, 2022
1b0c7ea
🎨 Added Tip class to data_models folder
Etesam913 Jun 2, 2022
40f683b
✅ Added test_tips_methods() to test new db methods
Etesam913 Jun 2, 2022
7a80d98
🔥 Removed commented out code
Etesam913 Jun 3, 2022
39acef7
✨ Tips from webapp can now be read from backend
Etesam913 Jun 3, 2022
f6e9744
✨ Added update_current_agent_state_metadata()
Etesam913 Jun 4, 2022
bcef575
🚧 Tips from units of task are printed to user
Etesam913 Jun 5, 2022
586e824
✨ Created script for accepting/rejecting tips
Etesam913 Jun 5, 2022
d2d7468
✨ Added remove_accepted_tip script
Etesam913 Jun 5, 2022
8a0048e
🐛 MockAgentState now has update_metadata() function
Etesam913 Jun 5, 2022
bc515a1
✨ Now a bonus can be paid to a worker for their tip
Etesam913 Jun 5, 2022
06e47d4
♻️ Replace quit() statements with while loops
Etesam913 Jun 5, 2022
4030473
♻️ Added spacing
Etesam913 Jun 6, 2022
de85e12
Merge branch 'add-tips-to-toxicity-detection' of https://github.com/f…
Etesam913 Jun 7, 2022
47c7bca
🔨 Configured mephisto-task with npm link
Etesam913 Jun 7, 2022
5e7d46b
🏗️ Began postTip() in mephisto-task and server.py
Etesam913 Jun 7, 2022
cc25c57
🎨 Tips to persist without remote_procedure
Etesam913 Jun 7, 2022
5011a35
🐛 Tips from unsubmitted tasks do not hide in review
Etesam913 Jun 8, 2022
e45e596
🔥 Removed tips db methods & data-model
Etesam913 Jun 8, 2022
0eb5146
♻️ Tip endpoints generalized into metadata endpoints
Etesam913 Jun 8, 2022
bbec236
🚨 Removed some extra spaces
Etesam913 Jun 8, 2022
ff4d52e
🐛 Fixed unrecognized tips in remove_tip script
Etesam913 Jun 8, 2022
90a07cb
✨ Tips can now be retrieved from taskConfig property
Etesam913 Jun 8, 2022
a6f5010
⛓ Linked mephisto-worker-experience to toxic example
Etesam913 Jun 8, 2022
686f954
Merge branch 'tips-component' of https://github.com/facebookresearch/…
Etesam913 Jun 9, 2022
9dbb1a5
➕ Added react-popper-tooltip lib for tooltips
Etesam913 Jun 10, 2022
67ef982
✨ Tips are now retrieved and submitted by default
Etesam913 Jun 10, 2022
1fb9059
✨ Added success/error handling for worker-experience
Etesam913 Jun 10, 2022
702a05c
♻️ Refactored Tips.js
Etesam913 Jun 11, 2022
88f4885
📝 Removed createTip & createFeedback exports
Etesam913 Jun 11, 2022
e6e75e1
✨ List prop is now used for default tips
Etesam913 Jun 11, 2022
e5f94f4
🐛 Fixed static-react-task not starting
Etesam913 Jun 11, 2022
e6cc801
✨ Added tips example
Etesam913 Jun 12, 2022
4bf8181
✅ Pass code-style test in Tips.js
Etesam913 Jun 12, 2022
846f69b
🚨 Mocked test_deploy_shutdown input
Etesam913 Jun 12, 2022
6eae9bb
🚨 Relinted code
Etesam913 Jun 12, 2022
fecb3e3
📝 Updated readmes of a task and package
Etesam913 Jun 12, 2022
2d7a637
🔀 Merged with tips-component
Etesam913 Jun 13, 2022
4f659e5
✏️ Fixed typos and added more detail to mep-task readme
Etesam913 Jun 14, 2022
5c709ab
🧪 Trying out moving db code out of build_router
Etesam913 Jun 15, 2022
29cb77c
🚚 Moved build_router db code to frontend_args
Etesam913 Jun 15, 2022
500b0d8
🔀 Merged with tips-component branch
Etesam913 Jun 15, 2022
05bb004
🐛 Fixed bug with text str failing check
Etesam913 Jun 21, 2022
6a8f43e
🐛 Fix bug with feedback text not a string
Etesam913 Jun 22, 2022
5dee9f7
✨ Added suppport for metadata in chat task
Etesam913 Jun 22, 2022
536e789
🔥 Removed None if statement for metadata
Etesam913 Jun 22, 2022
e155acd
🔥 Removed undeeded packages from toxicity_detection
Etesam913 Jun 22, 2022
208a307
✨ Added mpackage to yarn workspaces
Etesam913 Jun 23, 2022
4602ba3
↩️ Reverted the mephisto-task version increment
Etesam913 Jun 23, 2022
b959010
✅ Passed code-style test
Etesam913 Jun 23, 2022
ba231fd
♻️ Refactored get_frontend_args to make code common
Etesam913 Jun 23, 2022
46065b5
✨ Ttextnow shown as wrapped when enter is pressed
Etesam913 Jun 23, 2022
fec8427
🎨 Added maxHeight for tips list
Etesam913 Jun 23, 2022
59dc6a7
🎨 Changed position of the tips button
Etesam913 Jun 23, 2022
1995403
✅ Reformat tips
Etesam913 Jun 23, 2022
cd73e77
🔥 Removed extra space
Etesam913 Jun 23, 2022
99c2602
🔀 Merged with add-cypress-to-static-task
Etesam913 Jun 27, 2022
df705a9
🎨 Updated the headless version of Tips
Etesam913 Jun 27, 2022
1f575b6
🎨 Prefixed tips classnames to improve readability
Etesam913 Jun 27, 2022
851cba1
✏️ Renamed worker-experience to worker-addons
Etesam913 Jun 27, 2022
6893718
🚛 Moved databrowser out of base blueprint class
Etesam913 Jun 27, 2022
a26fdb4
✏️ Fixed typo with mephisto-user-experience
Etesam913 Jun 27, 2022
237d6b3
🎨 Changed loader className to prevent clashes
Etesam913 Jun 27, 2022
a677c80
🐛 Fixed bug where button was loading forever
Etesam913 Jun 29, 2022
ad1c5d7
🎨 Split main tips file into two more components
Etesam913 Jun 29, 2022
255ac13
🚛 Moved tipsComponents into TaskTips file
Etesam913 Jun 29, 2022
8841f0c
✅ Updated formatting
Etesam913 Jun 30, 2022
5c01b86
🎨 Modified fonts & sizes for tips popup
Etesam913 Jun 30, 2022
e455f7c
🔥 Removed some unneeded testing code
Etesam913 Jun 30, 2022
155282c
✨ Added a close button for the tips popup
Etesam913 Jun 30, 2022
7dd6206
🔥 Replaced bulma.css with custom styles
Etesam913 Jul 2, 2022
194942c
✏️ Removed headless prop
Etesam913 Jul 2, 2022
2ed447e
🎨 Upped line-height and added box-sizing property
Etesam913 Jul 2, 2022
046dfe9
🎨 Made small style changes to Tips
Etesam913 Jul 3, 2022
9e8b96c
🚛 Moved tips button in example
Etesam913 Jul 3, 2022
22bd57b
🎨 Small style fixes
Etesam913 Jul 3, 2022
6d76b13
✏️ Added margin to tip_text
Etesam913 Jul 3, 2022
d2e35b3
🚛 Moved tips logic in clientIO to agent
Etesam913 Jul 4, 2022
96a9d6a
Merge branch 'add-cypress-to-static-task' of https://github.com/faceb…
Etesam913 Jul 5, 2022
e532205
♻️ Added hydra params to build_bundle in new example
Etesam913 Jul 5, 2022
7a9ebee
🔀 Merge branch 'add-cypress-to-static-task'
Etesam913 Jul 6, 2022
bbbf29a
🔀 Merged with agent-state-consolidate
Etesam913 Jul 7, 2022
49d28a5
🏗 Began working on refactor
Etesam913 Jul 7, 2022
0b02403
✨ Refactored tips works witth remote-tasks
Etesam913 Jul 8, 2022
783529d
🗑 Cleaned up most files
Etesam913 Jul 8, 2022
256aa79
👈 Reverted _update_submit()
Etesam913 Jul 8, 2022
887c3d5
☝️ Switched order of load_metadata() and load_data()
Etesam913 Jul 8, 2022
1189241
Merge branch 'main' of https://github.com/facebookresearch/Mephisto i…
Etesam913 Jul 8, 2022
048f0b3
Merge branch 'main' into add-tips-example
Etesam913 Jul 8, 2022
11a3442
Merge branch 'add-tips-example' of https://github.com/facebookresearc…
Etesam913 Jul 8, 2022
0c6cd87
🐛 Added an import that was removed previously
Etesam913 Jul 8, 2022
f56079e
🐛 Made UserSubmission inputs disabled when loading
Etesam913 Jul 10, 2022
070a7c1
🔥 Removed comment & unused import
Etesam913 Jul 11, 2022
5824274
🔧 Configured react-shadow with Tips component
Etesam913 Jul 11, 2022
b383d54
👈 Reverted tips-example styles back to default
Etesam913 Jul 11, 2022
b0a3874
✨ Tips is in shadow dom when headless=false
Etesam913 Jul 12, 2022
eed4376
✏️ Fixed typo with headless=true being default
Etesam913 Jul 12, 2022
aeee926
♻️ Refactored unit collection
Etesam913 Jul 12, 2022
1dca1d3
✨ Tips are now saved to a tips.csv file
Etesam913 Jul 13, 2022
a53de6f
♻️ Moved metadata gathering in base blueprint class
Etesam913 Jul 13, 2022
0b8c9cf
♻✅ Added required fields to tips_location field
Etesam913 Jul 13, 2022
862e9f4
✅ Set default of tips_location to MISSING
Etesam913 Jul 13, 2022
d6a63f1
🥅 Added error check for get_tips_directory()
Etesam913 Jul 13, 2022
dba55fe
♻️ Added a more sensible default for tips_location
Etesam913 Jul 13, 2022
fc37242
🚛 Moved tips_location to base BlueprintArgs
Etesam913 Jul 13, 2022
ebfde86
🐛 Fixed issue with tips_location default
Etesam913 Jul 14, 2022
cee4929
🔥 Removed self.task_run from blueprint.py
Etesam913 Jul 14, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion mephisto/abstractions/_subcomponents/agent_state.py
Original file line number Diff line number Diff line change
Expand Up @@ -179,7 +179,7 @@ def set_init_state(self, data: Any) -> bool:
return True

@abstractmethod
def get_init_state(self, get_all_state: Optional[bool] = False) -> Optional[Any]:
def get_init_state(self) -> Optional[Any]:
"""
Return the initial state for this agent,
None if no such state exists
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,17 +35,13 @@ def _set_init_state(self, data: Any):
"""Set the initial state for this agent"""
self.state["inputs"] = data

def get_init_state(
self, get_all_state: Optional[bool] = False
) -> Optional[Dict[str, Any]]:
def get_init_state(self) -> Optional[Dict[str, Any]]:

"""
Return the initial state for this agent,
None if no such state exists
"""
if get_all_state == True:
return self.state
elif self.state["inputs"] is None:
if self.state["inputs"] is None:
return None
return self.state["inputs"].copy()

Expand Down
4 changes: 1 addition & 3 deletions mephisto/abstractions/blueprints/mock/mock_agent_state.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,7 @@ def _set_init_state(self, data: Any):
"""Set the initial state for this agent"""
self.init_state = data

def get_init_state(
self, get_all_state: Optional[bool] = False
) -> Optional[Dict[str, Any]]:
def get_init_state(self) -> Optional[Dict[str, Any]]:
"""
Return the initial state for this agent,
None if no such state exists
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,7 @@ def _set_init_state(self, data: Any):
"""Set the initial state for this agent"""
self.init_data: Optional[Any] = data

def get_init_state(
self, get_all_state: Optional[bool] = False
) -> Optional[Dict[str, Any]]:
def get_init_state(self) -> Optional[Dict[str, Any]]:
"""
Return the initial state for this agent,
None if no such state exists
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,7 @@ def _set_init_state(self, data: Any):
"""Set the initial state for this agent"""
self.init_data: Optional[Dict[str, Any]] = data

def get_init_state(
self, get_all_state: Optional[bool] = False
) -> Optional[Dict[str, Any]]:
def get_init_state(self) -> Optional[Dict[str, Any]]:
"""
Return the initial state for this agent,
None if no such state exists
Expand Down
55 changes: 27 additions & 28 deletions mephisto/tools/data_browser.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,17 +4,14 @@
# This source code is licensed under the MIT license found in the
# LICENSE file in the root directory of this source tree.

from mephisto.abstractions.database import MephistoDB
from mephisto.data_model.unit import Unit
from mephisto.data_model.task_run import TaskRun
from mephisto.abstractions.blueprint import AgentState
from mephisto.data_model.agent import Agent
from mephisto.data_model.worker import Worker

from mephisto.abstractions.databases.local_database import LocalMephistoDB
from mephisto.data_model.constants.assignment_state import AssignmentState

from typing import List, Optional, Any, Dict
from typing import List, Any, Dict


class DataBrowser:
Expand All @@ -28,45 +25,47 @@ def __init__(self, db=None):
db = LocalMephistoDB()
self.db = db

def _get_units_for_task_runs(self, task_runs: List[TaskRun]) -> List[Unit]:
def collect_matching_units_from_task_runs(
self, task_runs: List[TaskRun], statuses: List[str]
) -> List[Unit]:
"""
Return a list of all Units in a terminal completed state from all
the provided TaskRuns.
Loops through task_runs to collect all units in the provided statuses list
"""
units = []
for task_run in task_runs:
assignments = task_run.get_assignments()
for assignment in assignments:
found_units = assignment.get_units()
for unit in found_units:
if unit.get_status() in [
AssignmentState.COMPLETED,
AssignmentState.ACCEPTED,
AssignmentState.REJECTED,
AssignmentState.SOFT_REJECTED,
]:
if unit.get_status() in statuses:
units.append(unit)
return units

def _get_units_for_task_runs(self, task_runs: List[TaskRun]) -> List[Unit]:
"""
Return a list of all Units in a terminal completed state from all
the provided TaskRuns.
"""
statuses = [
AssignmentState.COMPLETED,
AssignmentState.ACCEPTED,
AssignmentState.REJECTED,
AssignmentState.SOFT_REJECTED,
]
return self.collect_matching_units_from_task_runs(task_runs, statuses)
Copy link
Contributor

@pringshia pringshia Jul 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think a method where you just pass in the negation (filtering) would feel better here, e.g. get all tasks except those the ones which are the following status, e.g. EXPIRED, - but I'm just nit-picking, this is fine

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like that would just make the code harder to understand as you would have to go to assignment_state.py to figure out which AssignmentState's are being accepted.

It wouldn't really make it more concise either as the code would become

filtered_statuses=[
  AssignmentState.CREATED,
  AssignmentState.LAUNCHED,
  AssignmentState.ASSIGNED,
  AssignmentState.MIXED,
  AssignmentState.EXPIRED  
]

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally for legibility, it could also be OK to put these collections inside of the AssignmentState class as class functions. AssignmentState.completed() and AssignmentState.final_agent() come to mind as relevant for these two lists at the moment.


def _get_all_units_for_task_runs(self, task_runs: List[TaskRun]) -> List[Unit]:
"""
Does the same as _get_units_for_task_runs except that it includes the EXPIRED state
"""
units = []
for task_run in task_runs:
assignments = task_run.get_assignments()
for assignment in assignments:
found_units = assignment.get_units()
for unit in found_units:
if unit.get_status() in [
AssignmentState.COMPLETED,
AssignmentState.ACCEPTED,
AssignmentState.REJECTED,
AssignmentState.SOFT_REJECTED,
AssignmentState.EXPIRED,
]:
units.append(unit)
return units
statuses = [
AssignmentState.COMPLETED,
AssignmentState.ACCEPTED,
AssignmentState.REJECTED,
AssignmentState.SOFT_REJECTED,
AssignmentState.EXPIRED,
]
return self.collect_matching_units_from_task_runs(task_runs, statuses)

def get_task_name_list(self) -> List[str]:
return [task.task_name for task in self.db.find_tasks()]
Expand Down