mirror of
https://github.com/Textualize/textual.git
synced 2025-10-17 02:38:12 +03:00
Snapshot testing guide (#3357)
* Snapshot testing guide * Typo fixes * Some more typo fixes * Typo fixes * Update docs/guide/testing.md Co-authored-by: Rodrigo Girão Serrão <5621605+rodrigogiraoserrao@users.noreply.github.com> * Add clarifications, PR feedback * Add clarifications, PR feedback --------- Co-authored-by: Rodrigo Girão Serrão <5621605+rodrigogiraoserrao@users.noreply.github.com>
This commit is contained in:
@@ -134,7 +134,7 @@ await pilot.click(Button, offset(0, -1))
|
||||
### Modifier keys
|
||||
|
||||
You can simulate clicks in combination with modifier keys, by setting the `shift`, `meta`, or `control` parameters.
|
||||
Here's how you could simulate ctrl-clicking a widget with an id of "slider":
|
||||
Here's how you could simulate ctrl-clicking a widget with an ID of "slider":
|
||||
|
||||
```python
|
||||
await pilot.click("#slider", control=True)
|
||||
@@ -162,7 +162,149 @@ You can generally solve this by calling [`pause()`][textual.pilot.Pilot.pause] w
|
||||
You can also supply a `delay` parameter, which will insert a delay prior to waiting for pending messages.
|
||||
|
||||
|
||||
## Textual's test
|
||||
## Textual's tests
|
||||
|
||||
Textual itself has a large battery of tests.
|
||||
If you are interested in how we write tests, see the [tests/](https://github.com/Textualize/textual/tree/main/tests) directory in the Textual repository.
|
||||
|
||||
## Snapshot testing
|
||||
|
||||
A _snapshot_ is a record of what an application looked like at a given point in time.
|
||||
|
||||
_Snapshot testing_ is the process of creating a snapshot of an application while a test runs, and comparing it to a historical version.
|
||||
If there's a mismatch, the snapshot testing framework flags it for review.
|
||||
|
||||
This offers a simple, automated way of checking our application displays like we expect.
|
||||
|
||||
### pytest-textual-snapshot
|
||||
|
||||
You can use [`pytest-textual-snapshot`](https://github.com/Textualize/pytest-textual-snapshot) to snapshot test your Textual app.
|
||||
This is a plugin for pytest which adds support for snapshot testing Textual apps, and it's maintained by the developers of Textual.
|
||||
|
||||
A test using this package saves a snapshot (in this case, an SVG screenshot) of a running Textual app to disk.
|
||||
The next time the test runs, it takes another snapshot and compares it to the previously saved one.
|
||||
If the snapshots differ, the test fails, and you can view a side-by-side diff showing the visual change.
|
||||
|
||||
#### Installation
|
||||
|
||||
You can install `pytest-textual-snapshot` using your favorite package manager (`pip`, `poetry`, etc.).
|
||||
|
||||
```
|
||||
pip install pytest-textual-snapshot
|
||||
```
|
||||
|
||||
#### Creating a snapshot test
|
||||
|
||||
With the package installed, you now have access to the `snap_compare` pytest fixture.
|
||||
|
||||
Let's look at an example of how we'd create a snapshot test for the [calculator app](https://github.com/Textualize/textual/blob/main/examples/calculator.py) below.
|
||||
|
||||
```{.textual path="examples/calculator.py" columns=100 lines=41 press="3,.,1,4,5,9,2,wait:400"}
|
||||
```
|
||||
|
||||
First, we need to create a new test and specify the path to the Python file containing the app.
|
||||
This path should be relative to the location of the test.
|
||||
|
||||
```python
|
||||
def test_calculator(snap_compare):
|
||||
assert snap_compare("path/to/calculator.py")
|
||||
```
|
||||
|
||||
Let's run the test as normal using `pytest`.
|
||||
|
||||
```
|
||||
pytest
|
||||
```
|
||||
|
||||
When this test runs for the first time, an SVG screenshot of the calculator app is generated, and the test will fail.
|
||||
Snapshot tests always fail on the first run, since there's no previous version to compare the snapshot to.
|
||||
|
||||

|
||||
|
||||
If you open the snapshot report in your browser, you'll see something like this:
|
||||
|
||||

|
||||
|
||||
!!! tip
|
||||
|
||||
You can usually open the link directly from the terminal, but some terminal emulators may
|
||||
require you to hold ++ctrl++ or ++command++ while clicking for links to work.
|
||||
|
||||
The report explains that there's "No history for this test".
|
||||
It's our job to validate that the initial snapshot looks correct before proceeding.
|
||||
Our calculator is rendering as we expect, so we'll save this snapshot:
|
||||
|
||||
```
|
||||
pytest --snapshot-update
|
||||
```
|
||||
|
||||
!!! warning
|
||||
|
||||
Only ever run pytest with `--snapshot-update` if you're happy with how the output looks
|
||||
on the left hand side of the snapshot report. When using `--snapshot-update`, you're saying "I'm happy with all of the
|
||||
screenshots in the snapshot test report, and they will now represent the ground truth which all future runs will be compared
|
||||
against". As such, you should only run `pytest --snapshot-update` _after_ running `pytest` and confirming the output looks good.
|
||||
|
||||
Now that our snapshot is saved, if we run `pytest` (with no arguments) again, the test will pass.
|
||||
This is because the screenshot taken during this test run matches the one we saved earlier.
|
||||
|
||||
#### Catching a bug
|
||||
|
||||
The real power of snapshot testing comes from its ability to catch visual regressions which could otherwise easily be missed.
|
||||
|
||||
Imagine a new developer joins your team, and tries to make a few changes to the calculator.
|
||||
While making this change they accidentally break some styling which removes the orange coloring from the buttons on the right of the app.
|
||||
When they run `pytest`, they're presented with a report which reveals the damage:
|
||||
|
||||

|
||||
|
||||
On the right, we can see our "historical" snapshot - this is the one we saved earlier.
|
||||
On the left is how our app is currently rendering - clearly not how we intended!
|
||||
|
||||
We can click the "Show difference" toggle at the top right of the diff to overlay the two versions:
|
||||
|
||||

|
||||
|
||||
This reveals another problem, which could easily be missed in a quick visual inspection -
|
||||
our new developer has also deleted the number 4!
|
||||
|
||||
!!! tip
|
||||
|
||||
Snapshot tests work well in CI on all supported operating systems, and the snapshot
|
||||
report is just an HTML file which can be exported as a build artifact.
|
||||
|
||||
|
||||
#### Pressing keys
|
||||
|
||||
You can simulate pressing keys before the snapshot is captured using the `press` parameter.
|
||||
|
||||
```python
|
||||
def test_calculator_pressing_numbers(snap_compare):
|
||||
assert snap_compare("path/to/calculator.py", press=["1", "2", "3"])
|
||||
```
|
||||
|
||||
#### Changing the terminal size
|
||||
|
||||
To capture the snapshot with a different terminal size, pass a tuple `(width, height)` as the `terminal_size` parameter.
|
||||
|
||||
```python
|
||||
def test_calculator(snap_compare):
|
||||
assert snap_compare("path/to/calculator.py", terminal_size=(50, 100))
|
||||
```
|
||||
|
||||
#### Running setup code
|
||||
|
||||
You can also run arbitrary code before the snapshot is captured using the `run_before` parameter.
|
||||
|
||||
In this example, we use `run_before` to hover the mouse cursor over the widget with ID `number-5`
|
||||
before taking the snapshot.
|
||||
|
||||
```python
|
||||
def test_calculator_hover_number(snap_compare):
|
||||
async def run_before(pilot) -> None:
|
||||
await pilot.hover("#number-5")
|
||||
|
||||
assert snap_compare("path/to/calculator.py", run_before=run_before)
|
||||
```
|
||||
|
||||
For more information, visit the [`pytest-textual-snapshot` repo on GitHub](https://github.com/Textualize/pytest-textual-snapshot).
|
||||
|
||||
BIN
docs/images/testing/snapshot_report_console_output.png
Normal file
BIN
docs/images/testing/snapshot_report_console_output.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 32 KiB |
BIN
docs/images/testing/snapshot_report_diff_after.png
Normal file
BIN
docs/images/testing/snapshot_report_diff_after.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 59 KiB |
BIN
docs/images/testing/snapshot_report_diff_before.png
Normal file
BIN
docs/images/testing/snapshot_report_diff_before.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 73 KiB |
BIN
docs/images/testing/snapshot_report_example.png
Normal file
BIN
docs/images/testing/snapshot_report_example.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 151 KiB |
Reference in New Issue
Block a user