{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# The TextAttack ecosystem: search, transformations, and constraints\n", "\n", "An attack in TextAttack consists of four parts.\n", "\n", "### Goal function\n", "\n", "The **goal function** determines if the attack is successful or not. One common goal function is **untargeted classification**, where the attack tries to perturb an input to change its classification. \n", "\n", "### Search method\n", "The **search method** explores the space of potential transformations and tries to locate a successful perturbation. Greedy search, beam search, and brute-force search are all examples of search methods.\n", "\n", "### Transformation\n", "A **transformation** takes a text input and transforms it, for example replacing words or phrases with similar ones, while trying not to change the meaning. Paraphrase and synonym substitution are two broad classes of transformations.\n", "\n", "### Constraints\n", "Finally, **constraints** determine whether or not a given transformation is valid. Transformations don't perfectly preserve syntax or semantics, so additional constraints can increase the probability that these qualities are preserved from the source to adversarial example. There are many types of constraints: overlap constraints that measure edit distance, syntactical constraints check part-of-speech and grammar errors, and semantic constraints like language models and sentence encoders." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### A custom transformation\n", "\n", "This lesson explains how to create a custom transformation. In TextAttack, many transformations involve *word swaps*: they take a word and try and find suitable substitutes. Some attacks focus on replacing characters with neighboring characters to create \"typos\" (these don't intend to preserve the grammaticality of inputs). Other attacks rely on semantics: they take a word and try to replace it with semantic equivalents.\n", "\n", "\n", "### Banana word swap \n", "\n", "As an introduction to writing transformations for TextAttack, we're going to try a very simple transformation: one that replaces any given word with the word 'banana'. In TextAttack, there's an abstract `WordSwap` class that handles the heavy lifting of breaking sentences into words and avoiding replacement of stopwords. We can extend `WordSwap` and implement a single method, `_get_replacement_words`, to indicate to replace each word with 'banana'. 🍌" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": true }, "outputs": [], "source": [ "from textattack.transformations import WordSwap\n", "\n", "class BananaWordSwap(WordSwap):\n", " \"\"\" Transforms an input by replacing any word with 'banana'.\n", " \"\"\"\n", " \n", " # We don't need a constructor, since our class doesn't require any parameters.\n", "\n", " def _get_replacement_words(self, word):\n", " \"\"\" Returns 'banana', no matter what 'word' was originally.\n", " \n", " Returns a list with one item, since `_get_replacement_words` is intended to\n", " return a list of candidate replacement words.\n", " \"\"\"\n", " return ['banana']" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "### Using our transformation\n", "\n", "Now we have the transformation chosen, but we're missing a few other things. To complete the attack, we need to choose the **search method** and **constraints**. And to use the attack, we need a **goal function**, a **model** and a **dataset**. (The goal function indicates the task our model performs – in this case, classification – and the type of attack – in this case, we'll perform an untargeted attack.)\n", "\n", "### Creating the goal function, model, and dataset\n", "We are performing an untargeted attack on a classification model, so we'll use the `UntargetedClassification` class. For the model, let's use an LSTM trained for news classification on the AG News dataset. Luckily, TextAttack comes with 1000 text samples from some popular datasets, as well as pretrained models for those datasets. So we don't have to train our own model, or procure someone else's. We can just use the built-in datasets and models for this." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "\u001b[34;1mtextattack\u001b[0m: Goal function matches model LSTMForAGNewsClassification.\n" ] } ], "source": [ "# Import the dataset.\n", "from textattack.datasets.classification import AGNews\n", "# Create the model.\n", "from textattack.models.classification.lstm import LSTMForAGNewsClassification\n", "model = LSTMForAGNewsClassification()\n", "# Create the goal function using the model.\n", "from textattack.goal_functions import UntargetedClassification\n", "goal_function = UntargetedClassification(model)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Creating the attack\n", "Let's keep it simple: let's use a greedy search method, and let's not use any constraints for now. " ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": true }, "outputs": [], "source": [ "from textattack.search_methods import GreedySearch\n", "from textattack.constraints.pre_transformation import RepeatModification, StopwordModification\n", "from textattack.shared import Attack\n", "\n", "# We're going to use our Banana word swap class as the attack transformation.\n", "transformation = BananaWordSwap() \n", "# We'll constrain modification of already modified indices and stopwords\n", "constraints = [RepeatModification(),\n", " StopwordModification()]\n", "# We'll use the Greedy search method\n", "search_method = GreedySearch()\n", "# Now, let's make the attack from the 4 components:\n", "attack = Attack(goal_function, constraints, transformation, search_method)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's print our attack to see all the parameters:" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Attack(\n", " (search_method): GreedySearch\n", " (goal_function): UntargetedClassification\n", " (transformation): BananaWordSwap\n", " (constraints): \n", " (0): RepeatModification\n", " (1): StopwordModification\n", " (is_black_box): True\n", ")\n" ] } ], "source": [ "print(attack)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Using the attack\n", "\n", "Let's use our attack to attack 10 samples (by setting `num_examples` to 10). Additionally, we set `attack_n` to `True`, which indicates that we should attack 10 samples, no matter what. If the model mispredicts a sample already, it isn't attacked; since `attack_n` is `True`, if a sample is mispredicted, we'll try the next example in the dataset, and continue until `num_examples` attacks have been completed." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "12it [00:00, 19.61it/s] \n" ] } ], "source": [ "from tqdm import tqdm # tqdm provides us a nice progress bar.\n", "from textattack.loggers import CSVLogger # tracks a dataframe for us.\n", "\n", "results_iterable = attack.attack_dataset(AGNews(), num_examples=10, attack_n=True)\n", "results = []\n", "\n", "logger = CSVLogger(color_method='html')\n", "\n", "for result in tqdm(results_iterable, total=10):\n", " logger.log_attack_result(result)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Visualizing attack results\n", "\n", "We are logging `AttackResult` objects using a `CSVLogger`. This logger stores all attack results in a dataframe, which we can easily access and display. Since we set `color_method` to `'html'`, the attack results will display their differences, in color, in HTML. Using `IPython` utilities and `pandas`" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
original_textperturbed_text
0Thirst, Fear and Bribes on Desert Escape from Africa AGADEZ, Niger (Reuters) - Customs officers in this dusty Saharan town turned a blind eye as yet another creaking truck piled with grain, smuggled cigarettes and dozens of migrants heading for Europe rumbled off into the desert.Thirst, Fear and Bribes on banana Escape from Africa banana, banana (Reuters) - Customs officers in this dusty Saharan town turned a blind eye as yet another creaking truck piled with grain, smuggled cigarettes and dozens of migrants heading for Europe rumbled off into the desert.
1Toshiba 20 TV freaks out, sends distress signal See what happens when your warranty runs out?. In this case, a 20 Toshiba owned by Chris van Rossman started sending out the international distress signal at 121.Toshiba 20 TV freaks out, sends distress signal See what happens when your warranty runs out?. In this case, a 20 Toshiba owned by Chris van Rossman started sending out the international distress signal at 121.
2British hostage tried fleeing before death: report The portrait of Ken Bigley, who was murdered in Iraq October 7, stands in front of the congregation during a service at Liverpool #39;s Roman Catholic Cathedral on October 10.British hostage tried fleeing before death: report The portrait of Ken Bigley, who was murdered in Iraq October 7, stands in front of the congregation during a service at Liverpool #39;s Roman Catholic Cathedral on October 10.
3Keychain clicker kills TVs Discrete device turns off televisions, creating a little peace and quiet. Until the yelling starts.banana banana kills TVs Discrete device turns off televisions, creating a little peace and quiet. Until the yelling starts.
4Cleric returns to Iraq, orders march on Najaf Powerful Shiite leader says he plans to lead a mass demonstration today to end fighting. Iraqi hostage: Militants said Wednesday they had kidnapped the brother-in-law of Iraqi Defense Minister Hazem Shaalanbanana returns to banana, orders march on Najaf banana Shiite leader says he plans to banana a mass demonstration today to end fighting. banana banana: banana banana Wednesday they had banana the brother-in-law of banana Defense banana Hazem Shaalan
5Hewitt Beats Roddick to Reach Masters Final HOUSTON (Reuters) - A fired-up Lleyton Hewitt defused hard-hitting Andy Roddick 6-3, 6-2 on Saturday, scurrying into the final of the Masters Cup for the third time in four years.banana Beats Roddick to banana Masters Final HOUSTON (Reuters) - banana fired-up Lleyton Hewitt defused hard-hitting Andy Roddick 6-3, 6-2 on Saturday, scurrying into the final of the banana Cup for the third time in four years.
6Despite booming economy, no concrete move on debt relief (AFP) AFP - Senior finance officials have hailed a robust global economic recovery, albeit one threatened by surging oil prices, but made little headway pushing China toward currency reform and took no firm steps to ease the debt of the world's poorest nations.banana banana economy, no concrete move on banana relief (AFP) AFP - Senior finance officials have hailed a robust global banana recovery, albeit one threatened by surging banana prices, but made little headway pushing China toward currency reform and took no firm banana to ease the debt of the world's poorest nations.
7Ethiopian court sentences 3 former rebels to death for mass murders (Canadian Press) Canadian Press - ADDIS ABABA, Ethiopia (AP) - A court has sentenced three former rebels to death for killing dozens of people while rebel factions jockeyed for power more than a decade ago, a government spokesman said Thursday.banana court sentences 3 former banana to death for mass murders (Canadian banana) Canadian banana - ADDIS ABABA, banana (AP) - A court has sentenced three former banana to death for banana dozens of people while rebel factions jockeyed for power more than a decade ago, a government spokesman said Thursday.
8Just a close call for closer No need to check with your primary care provider about the origin of that nervous tic you woke up with this morning, on the first full day of autumn. Ninth-inning home runs allowed in three consecutive games by Sox closer Keith Foulke, who also was tagged with blown saves in each of the last two games, were enough to leave ...Just a close call for closer No need to check with your primary care banana about the origin of that banana banana you woke up with this morning, on the first full day of autumn. Ninth-inning home runs allowed in three consecutive banana by Sox closer banana Foulke, who also was tagged with blown saves in each of the last two banana, were enough to leave ...
9Study: Wild Monkeys Resort to Use of Tools WASHINGTON - Wild South American monkeys routinely use fist-sized rocks to crack open seeds and to dig in dry Brazilian soil for grubs and edible tubers, researchers report in the journal Science.banana: Wild banana Resort to Use of Tools WASHINGTON - Wild South American monkeys routinely use fist-sized rocks to crack open seeds and to dig in dry Brazilian soil for grubs and edible tubers, researchers report in the journal Science.
10Bryant's Request to Seal Evidence Opposed (AP) AP - The prosecutor who charged Kobe Bryant with felony sexual assault has joined news organizations in opposing an attempt by the NBA star's attorney to permanently seal evidence and documents in the case.Bryant's Request to Seal Evidence Opposed (AP) AP - banana prosecutor who charged Kobe Bryant with felony sexual assault has joined news organizations in opposing an attempt by the banana star's attorney to permanently seal evidence and documents in the case.
11Eagles have lift off Crystal Palace were triumphant last night (Oct 4) over 10-man Fulham at Selhurst Park, lifting themselves off the bottom of the Premiership.Eagles have lift off Crystal Palace were triumphant last night (Oct 4) over 10-banana Fulham at Selhurst Park, lifting themselves off the banana of the banana.
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import pandas as pd\n", "pd.options.display.max_colwidth = 480 # increase colum width so we can actually read the examples\n", "\n", "from IPython.core.display import display, HTML\n", "display(HTML(logger.df[['original_text', 'perturbed_text']].to_html(escape=False)))" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "### Conclusion n", "\n", "We can examine these examples for a good idea of how many words had to be changed to \"banana\" to change the prediction score from the correct class to another class. The examples without perturbed words were originally misclassified, so they were skipped by the attack. Looks like some examples needed only a couple \"banana\"s, while others needed up to 17 \"banana\" substitutions to change the class score. Wow! 🍌" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.3" } }, "nbformat": 4, "nbformat_minor": 2 }