Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

what attribute mean in dataset, positive and label #1

Open
Miraclemarvel55 opened this issue Apr 22, 2024 · 2 comments
Open

what attribute mean in dataset, positive and label #1

Miraclemarvel55 opened this issue Apr 22, 2024 · 2 comments

Comments

@Miraclemarvel55
Copy link

Miraclemarvel55 commented Apr 22, 2024

In the probe subset , what mean positive and label if true or false and original_human_prediction": "2",
"flipped_human_prediction" what the number mean ,please share the definition, tks.

@SiyuanWangw
Copy link
Owner

positive in the probing subset refers to the presence or absence of negation words in the conclusion of a rule. Specifically, positive=True indicates that there are no negation words (such as CanDrive) whereas positive=True signifies the inclusion of negation words (such as CanNotDrive).

label means whether this rule is verified by human expert as original_human_prediction="2"(True/Right/Yes) and flipped_human_prediction="1"(False/Wrong/No). The definition of original probing and flipped probing can be found in our paper (the second paragraph of Section 3.1).

@Miraclemarvel55
Copy link
Author

Really appreciate your work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants