Skip to content

Commit

Permalink
Fixed Typo and increased performance in analyze_sentence (#2070)
Browse files Browse the repository at this point in the history
* Type

Type on  line 147. Sentence is "...happy ending has it won't..." where "has" should be "as"

* Swapped + operator for append when adding to list

As @piskvorky pointed out, using `s + [None]`on line 149 creates a new list, an expensive operation considering the goal is just to append `None` to an already existing list. Using `append` increases performance by 5 micro seconds per function call on my system
  • Loading branch information
JonathanHourany authored and piskvorky committed Jun 10, 2018
1 parent 700a1be commit e50a057
Showing 1 changed file with 4 additions and 3 deletions.
7 changes: 4 additions & 3 deletions gensim/models/phrases.py
Original file line number Diff line number Diff line change
Expand Up @@ -141,11 +141,12 @@ def analyze_sentence(self, sentence, threshold, common_terms, scorer):
"""
s = [utils.any2utf8(w) for w in sentence]
# adding None is a trick that helps getting an automatic happy ending
# as it won't be a common_word, nor score
s.append(None)
last_uncommon = None
in_between = []
# adding None is a trick that helps getting an automatic happy ending
# has it won't be a common_word, nor score
for word in s + [None]:
for word in s:
is_common = word in common_terms
if not is_common and last_uncommon:
chain = [last_uncommon] + in_between + [word]
Expand Down

0 comments on commit e50a057

Please sign in to comment.