Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Negated antonym #255

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ checklist==0.0.11
spacy==3.0.0
numpy


# for back_translation
torch

Expand Down
30 changes: 30 additions & 0 deletions transformations/negated_antonym_perturbation/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Negated Antonym Perturbation
This perturbation rephrases the adjectives and adverbs to their negated antonym

Authors: Mayukh Das (Technical University of Braunschweig / [email protected])

## What type of a transformation is this?
This transformation detects all type of adjectives and adverb and converts them to their negated antonym.
Therfore the transformation retains the semantics of the original text as positives gets converted to negated negatives and negatives gets converted to negated positives.
Example: I think you are prepared for the test --> I think you are not unprepared for the test
## What tasks does it intend to benefit?
This can act as a valid perturbation that retains the semantics.
This will benefit for robustness of text classification like sentiment analysis, etc.


## What are the limitations of this transformation?
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This transformation is unfortunately the same as #222 as it too uses negated antonyms. @Mayukhga83 is there anything different?

Copy link
Author

@Mayukhga83 Mayukhga83 Oct 13, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kaustubhdhole unfortunately yes except for the 'not' word as @marco-digio pointed. Secondly, this always does double negation for both even and odd number of adjective unlike #222 which only does it for even number of adjective words.
As the addition of 'not' gives a better robustness score than #222 (I have only checked for roberta-base-imdb) , It could be merged as a special case of the other PR if suited.

It is limited to only adjectives and adverbs (comparative and superlative)

## Robustness Evaluation

original accuracy: 96.0
dataset_name: 'imdb'
model_name: 'aychang/roberta-base-imdb'
no_of_examples: 250
accuracy after perturbation: 91.0

The accuracy drops by 5% when tested on imdb dataset for roberta

## KeyWords
tokenizer-required
highly-meaning-preserving
1 change: 1 addition & 0 deletions transformations/negated_antonym_perturbation/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
from .transformation import *
68 changes: 68 additions & 0 deletions transformations/negated_antonym_perturbation/test.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
{
"type": "negated_antonym_transformation",
"test_cases": [
{
"class": "NegatedAntonym",
"inputs": {
"sentence": "I think you are successful."
},
"outputs": [{
"sentence": "I think you are not unsuccessful ."
}]
},
{
"class": "NegatedAntonym",
"inputs": {
"sentence": "I think you are prepared for the test."
},
"outputs": [{
"sentence": "I think you are not unprepared for the test ."
}]
},
{
"class": "NegatedAntonym",
"inputs": {
"sentence": "He ran quickly."
},
"outputs": [{
"sentence": "He ran not slowly ."
}]
},
{
"class": "NegatedAntonym",
"inputs": {
"sentence": "He plays the flute beautifully."
},
"outputs": [{
"sentence": "He plays the flute not unattractively ."
}]
},
{
"class": "NegatedAntonym",
"inputs": {
"sentence": "I think you are joyful."
},
"outputs": [{
"sentence": "I think you are not sorrowful ."
}]
},
{
"class": "NegatedAntonym",
"inputs": {
"sentence": "I think you are smarter than me."
},
"outputs": [{
"sentence": "I think you are not stupid than me ."
}]
},
{
"class": "NegatedAntonym",
"inputs": {
"sentence": "The movie was bad."
},
"outputs": [{
"sentence": "The movie was not good ."
}]
}
]
}
40 changes: 40 additions & 0 deletions transformations/negated_antonym_perturbation/transformation.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
import nltk
from nltk.corpus import wordnet
import nltk.tokenize as nt

from interfaces.SentenceOperation import SentenceOperation
from tasks.TaskTypes import TaskType

''' Class that converts adjectives and adverbs to its negated antonym'''


class NegatedAntonym(SentenceOperation):
tasks = [TaskType.TEXT_CLASSIFICATION, TaskType.TEXT_TO_TEXT_GENERATION]
Mayukhga83 marked this conversation as resolved.
Show resolved Hide resolved
languages = ["en"]

Mayukhga83 marked this conversation as resolved.
Show resolved Hide resolved
def __init__(self, seed=0):
super().__init__(seed)

def generate(self, sentence):
return [self.Neg_Antonym(sentence)]

def Neg_Antonym(self, sentence):

tokenized_sent = nt.word_tokenize(sentence)
pos_sentences = nltk.pos_tag(tokenized_sent)

for i in range(len(pos_sentences)):
antonyms = []
if pos_sentences[i][1] == 'JJ' or pos_sentences[i][1] == 'JJR' or pos_sentences[i][1] == 'JJS' or \
pos_sentences[i][1] == 'RB' or pos_sentences[i][1] == 'RBR' or pos_sentences[i][1] == 'RBS':
for syn in wordnet.synsets(tokenized_sent[i]):

for lm in syn.lemmas():

if lm.antonyms():
antonyms.append(lm.antonyms()[0].name())

if len(antonyms) != 0:
tokenized_sent[i] = 'not ' + antonyms[0]

return ' '.join(tokenized_sent)