alihan/textattack-nlp-transformer
1
0
Fork 0
mirror of https://github.com/QData/TextAttack.git synced 2021-10-13 00:05:06 +03:00
Code Issues Projects Releases Wiki Activity

have flair use textattack tokenization method

Browse Source
This commit is contained in:
Jin Yong Yoo
2020-10-06 01:16:14 -04:00
parent a07deef834
commit 224d788d3e
2 changed files with 6 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
textattack/shared/attacked_text.py
View File

@@ -125,7 +125,9 @@ class AttackedText:
Uses FLAIR part-of-speech tagger.
"""
if not self._pos_tags:
sentence = Sentence(self.text)
sentence = Sentence(
self.text, use_tokenizer=textattack.shared.utils.words_from_text
)
textattack.shared.utils.flair_tag(sentence)
self._pos_tags = sentence
flair_word_list, flair_pos_list = textattack.shared.utils.zip_flair_result(