Sybil Documentation Testing

Sybil validates code examples embedded in documentation and docstrings by parsing and executing them as part of normal test runs.

Official Documentation: https://sybil.readthedocs.io/en/latest/

Installation

pip install sybil[pytest]

Pytest Integration

Configure in conftest.py. See pytest integration docs.

from sybil import Sybil
from sybil.parsers.markdown.codeblock import PythonCodeBlockParser
from sybil.parsers.markdown.skip import SkipParser

pytest_collect_file = Sybil(
    parsers=[
        SkipParser(),
        PythonCodeBlockParser(),
    ],
    patterns=["*.md", "**/*.py"],
).pytest()

Sybil Parameters

See API reference.

Document Types

See API reference.

• Default: Parse entire file
• PythonDocument: Import .py file as module, names available in namespace
• PythonDocStringDocument: Parse only docstrings from .py files

from sybil.document import PythonDocStringDocument

pytest_collect_file = Sybil(
    parsers=[...],
    patterns=["src/**/*.py"],
    document_types={".py": PythonDocStringDocument},
).pytest()

Fixtures

import pytest
from sybil import Sybil


@pytest.fixture
def my_fixture():
    return {"key": "value"}


pytest_collect_file = Sybil(
    parsers=[...],
    patterns=["*.md"],
    fixtures=["my_fixture"],  # Available in document namespace
).pytest()

Setup/Teardown

def sybil_setup(namespace):
    namespace["helper"] = lambda x: x * 2


def sybil_teardown(namespace):
    pass  # Cleanup if needed


pytest_collect_file = Sybil(
    parsers=[...],
    setup=sybil_setup,
    teardown=sybil_teardown,
).pytest()

Disable pytest's Built-in Doctest

Add to pyproject.toml to prevent conflicts:

[tool.pytest.ini_options]
addopts = "-p no:doctest"

Markdown Parsers

See Markdown parsers docs.

from sybil.parsers.markdown.codeblock import PythonCodeBlockParser, CodeBlockParser
from sybil.parsers.markdown.skip import SkipParser
from sybil.parsers.markdown.clear import ClearNamespaceParser

Skip Directives

See skip directive docs.

SkipParser must come before other parsers to handle skip directives.

<!-- skip: next -->

```python
# This example is skipped
```

# Skipped and reported as skipped test with reason

# Multiple examples

# All skipped

# Conditionally skipped based on namespace variable

## Invisible Code Blocks

Setup code that doesn't render in documentation. See [invisible code blocks docs](https://sybil.readthedocs.io/en/latest/markdown.html#invisible-code-blocks).

```markdown
<!-- invisible-code-block: python
setup_var = "hidden setup"
-->

Clear Namespace

Reset the document namespace for isolation. See clear namespace docs.

<!-- clear-namespace -->

Custom Evaluators

See evaluators API.

from sybil.parsers.markdown.codeblock import CodeBlockParser
from sybil.evaluators.python import PythonEvaluator

# Custom evaluator with future imports
evaluator = PythonEvaluator(future_imports=["annotations"])

parser = CodeBlockParser(language="python", evaluator=evaluator)

Running Sybil Tests

Sybil tests are collected like regular pytest tests. To run only Sybil tests:

# Run tests from specific directory containing documented code
pytest src/mypackage/ -v

# Exclude regular tests, only run documentation examples
pytest docs/ -v

To exclude Sybil tests from regular test runs, use pytest's --ignore flag or configure addopts in pyproject.toml.

Example: Complete conftest.py

"""Sybil configuration for docstring testing."""

from sybil import Sybil
from sybil.document import PythonDocStringDocument
from sybil.evaluators.python import PythonEvaluator
from sybil.parsers.markdown.codeblock import CodeBlockParser
from sybil.parsers.markdown.skip import SkipParser

import mypackage


def sybil_setup(namespace):
    """Pre-populate namespace for all examples."""
    namespace["pkg"] = mypackage


def sybil_teardown(namespace):
    """Cleanup after document."""
    pass


pytest_collect_file = Sybil(
    parsers=[
        SkipParser(),
        CodeBlockParser(language="python", evaluator=PythonEvaluator()),
        CodeBlockParser(language="py", evaluator=PythonEvaluator()),
    ],
    patterns=["src/mypackage/**/*.py"],
    document_types={".py": PythonDocStringDocument},
    setup=sybil_setup,
    teardown=sybil_teardown,
    excludes=[
        "**/tests/**",
        "**/_internal/**",
    ],
).pytest()