CLI

sphinx-performance

sphinx-performance is used via its command line interface.

To get a first help, type sphinx-performance --help

$ sphinx-performance --help
Usage: sphinx-performance [OPTIONS]

  CLI performance handling.

Options:
  --project TEXT      Defines the project to build
  --profile TEXT      Activates profiling for given area
  --parallel INTEGER  Number of parallel processes to use. Same as -j for
                      sphinx-build
  --builder TEXT      Define the builder to use
  --keep              Keeps the temporary src and build folders
  --browser           Opens the project in your browser
  --snakeviz          Opens snakeviz view for measured profiles in browser
  --debug             console.prints more information, incl. sphinx build
                      output
  --temp TEXT         Base folder path to use for temp folders. Must exist.
  --csv TEXT          CSV file path, which shall store the results.
  -h, --help          Show this message and exit.

Use sphinx-command to compare different Sphinx project setups.

For deeper analysis, use sphinx-analysis.

--project

--project allows to select a specific test project.

This can be an integrated project or a path to a self defined project:

sphinx-performance --project needs
sphinx-performance --project my/sphinx/project

Can be used multiple times, so that for each project a specific test run gets executed:

sphinx-performance --project basic --project needs

Default: basic

--parallel

Defines the amount of cores (Sphinx workers) to use for the build.

Uses internally the Sphinx option -j.

Parallel execution can bring huge benefits, if the documentation is based on multiple pages.

$ sphinx-performance --parallel 1 --parallel 4 --pages 30

Running 2 test configurations.


Running 2 test configurations.

─────────────────────────────────── Run 1/2 ────────────────────────────────────
Project:         basic
Core/s:          1
Builder:         html
Config:          pages: 30, sphinx: 5.1, dummies: 10, folders: 0, depth: 1
Info:            #dummies: 300, #pages: 30, #indexes: 1, #folders: 0

Docs path:       /tmp/tmpu2_lkk7f
Docs files:      31 rst files with 26.57 kB
Docs setup:      0.08 s

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
sphinx-performance 0.1.8 requires sphinx<6.0.0,>=5.3.0, but you have sphinx 5.1.0 which is incompatible.
Deps setup:      1.89 s
WARNING: html_static_path entry '_static' does not exist

Build files:     115 files with 1198.91 kB
File max️:        281.82 kB by /tmp/tmpu2_lkk7f/_build/_static/jquery-3.6.0.js
File min:        0.04 kB by /tmp/tmpu2_lkk7f/_build/_static/custom.css
File Ø:          10.43 kB (0.02 s)
Reading time:    0.59 s
Writing time:    0.58 s
Build Duration:  1.95 s
─────────────────────────────────── Run 2/2 ────────────────────────────────────
Project:         basic
Core/s:          4
Builder:         html
Config:          pages: 30, sphinx: 5.1, dummies: 10, folders: 0, depth: 1
Info:            #dummies: 300, #pages: 30, #indexes: 1, #folders: 0

Docs path:       /tmp/tmpex9rmd5_
Docs files:      31 rst files with 26.57 kB
Docs setup:      0.08 s

Deps setup:      0.84 s
WARNING: html_static_path entry '_static' does not exist

Build files:     115 files with 1199.77 kB
File max️:        281.82 kB by /tmp/tmpex9rmd5_/_build/_static/jquery-3.6.0.js
File min:        0.04 kB by /tmp/tmpex9rmd5_/_build/_static/custom.css
File Ø:          10.43 kB (0.02 s)
Reading time:    0.13 s
Writing time:    0.16 s
Build Duration:  2.06 s
─────────────────────────────────── RESULTS ────────────────────────────────────
╭───────────────┬────────────┬────────────╮
│       #       │   Run 1    │   Run 2    │
├───────────────┼────────────┼────────────┤
│    runtime    │    1.95    │    2.06    │
│    project    │   basic    │   basic    │
│               │            │            │
│     depth     │     1      │     1      │
│    sphinx     │    5.1     │    5.1     │
│    dummies    │     10     │     10     │
│    builder    │    html    │    html    │
│    folders    │     0      │     0      │
│   parallel    │     1      │     4      │
│     pages     │     30     │     30     │
│               │            │            │
│   #folders    │     0      │     0      │
│    #pages     │     30     │     30     │
│   #dummies    │    300     │    300     │
│   #indexes    │     1      │     1      │
│               │            │            │
│ max file size │ 281.82 kB  │ 281.82 kB  │
│ min file size │  0.04 kB   │  0.04 kB   │
│    # files    │    115     │    115     │
│  folder size  │ 1198.91 kB │ 1199.77 kB │
│ reading time  │   0.59 s   │   0.13 s   │
│ avg file size │  10.43 kB  │  10.43 kB  │
│ avg file time │   0.02 s   │   0.02 s   │
│ writing time  │   0.58 s   │   0.16 s   │
│               │            │            │
╰───────────────┴────────────┴────────────╯

Overall runtime: 4.01 seconds.

--builder

Defines the builder to use, for instance html, text, json, xml.

Type make in a sphinx project folder so see all available builders.

Can be used multiple times, so that for each builder a specific test run gets executed.

$ sphinx-performance --pages 2 --builder html --builder text

Running 2 test configurations.


Running 2 test configurations.

─────────────────────────────────── Run 1/2 ────────────────────────────────────
Project:         basic
Core/s:          1
Builder:         html
Config:          pages: 2, sphinx: 5.1, dummies: 10, folders: 0, depth: 1
Info:            #dummies: 20, #pages: 2, #indexes: 1, #folders: 0

Docs path:       /tmp/tmpaykliwp4
Docs files:      3 rst files with 1.93 kB
Docs setup:      0.02 s

Deps setup:      0.87 s
WARNING: html_static_path entry '_static' does not exist

Build files:     31 files with 586.48 kB
File max️:        281.82 kB by /tmp/tmpaykliwp4/_build/_static/jquery-3.6.0.js
File min:        0.04 kB by /tmp/tmpaykliwp4/_build/_static/custom.css
File Ø:          18.92 kB (0.03 s)
Reading time:    0.03 s
Writing time:    0.08 s
Build Duration:  0.80 s
─────────────────────────────────── Run 2/2 ────────────────────────────────────
Project:         basic
Core/s:          1
Builder:         text
Config:          pages: 2, sphinx: 5.1, dummies: 10, folders: 0, depth: 1
Info:            #dummies: 20, #pages: 2, #indexes: 1, #folders: 0

Docs path:       /tmp/tmp3pi63ss8
Docs files:      3 rst files with 1.93 kB
Docs setup:      0.02 s

Deps setup:      0.84 s
WARNING: html_static_path entry '_static' does not exist

Build files:     7 files with 41.91 kB
File max️:        14.43 kB by 
/tmp/tmp3pi63ss8/_build/.doctrees/environment.pickle
File min:        0.32 kB by /tmp/tmp3pi63ss8/_build/index.txt
File Ø:          5.99 kB (0.09 s)
Reading time:    0.01 s
Writing time:    0.01 s
Build Duration:  0.61 s
─────────────────────────────────── RESULTS ────────────────────────────────────
╭───────────────┬───────────┬──────────╮
│       #       │   Run 1   │  Run 2   │
├───────────────┼───────────┼──────────┤
│    runtime    │   0.80    │   0.61   │
│    project    │   basic   │  basic   │
│               │           │          │
│     pages     │     2     │    2     │
│   parallel    │     1     │    1     │
│    builder    │   html    │   text   │
│    dummies    │    10     │    10    │
│    sphinx     │    5.1    │   5.1    │
│     depth     │     1     │    1     │
│    folders    │     0     │    0     │
│               │           │          │
│    #pages     │     2     │    2     │
│   #indexes    │     1     │    1     │
│   #folders    │     0     │    0     │
│   #dummies    │    20     │    20    │
│               │           │          │
│ max file size │ 281.82 kB │ 14.43 kB │
│ avg file size │ 18.92 kB  │ 5.99 kB  │
│ min file size │  0.04 kB  │ 0.32 kB  │
│ reading time  │  0.03 s   │  0.01 s  │
│    # files    │    31     │    7     │
│ writing time  │  0.08 s   │  0.01 s  │
│  folder size  │ 586.48 kB │ 41.91 kB │
│ avg file time │  0.03 s   │  0.09 s  │
│               │           │          │
╰───────────────┴───────────┴──────────╯

Overall runtime: 1.42 seconds.

Project parameters

sphinx-performance takes every project parameters and provides it to the test project.

Which parameters are need and how their defaults looks like are documented in the test project specific documentation.

$ sphinx-performance --pages 2 --dummies 5

Running 1 test configurations.


Running 1 test configurations.

─────────────────────────────────── Run 1/1 ────────────────────────────────────
Project:         basic
Core/s:          1
Builder:         html
Config:          pages: 2, dummies: 5, sphinx: 5.1, folders: 0, depth: 1
Info:            #dummies: 10, #pages: 2, #indexes: 1, #folders: 0

Docs path:       /tmp/tmp0lwxwlrn
Docs files:      3 rst files with 1.14 kB
Docs setup:      0.02 s

Deps setup:      0.84 s
WARNING: html_static_path entry '_static' does not exist

Build files:     31 files with 576.04 kB
File max️:        281.82 kB by /tmp/tmp0lwxwlrn/_build/_static/jquery-3.6.0.js
File min:        0.04 kB by /tmp/tmp0lwxwlrn/_build/_static/custom.css
File Ø:          18.58 kB (0.03 s)
Reading time:    0.02 s
Writing time:    0.08 s
Build Duration:  0.79 s
─────────────────────────────────── RESULTS ────────────────────────────────────
╭───────────────┬───────────╮
│       #       │   Run 1   │
├───────────────┼───────────┤
│    runtime    │   0.79    │
│    project    │   basic   │
│               │           │
│    sphinx     │    5.1    │
│    folders    │     0     │
│   parallel    │     1     │
│     depth     │     1     │
│    builder    │   html    │
│    dummies    │     5     │
│     pages     │     2     │
│               │           │
│   #indexes    │     1     │
│    #pages     │     2     │
│   #folders    │     0     │
│   #dummies    │    10     │
│               │           │
│ reading time  │  0.02 s   │
│ min file size │  0.04 kB  │
│ avg file size │ 18.58 kB  │
│ max file size │ 281.82 kB │
│  folder size  │ 576.04 kB │
│ avg file time │  0.03 s   │
│ writing time  │  0.08 s   │
│    # files    │    31     │
│               │           │
╰───────────────┴───────────╯

Overall runtime: 0.79 seconds.

Commonly used parameters

This is a list of parameters, which are supported by most test projects.

--pages

Amount of pages per folder:

sphinx-performance --pages 20

--folders

Amount of folders per folder depth:

sphinx-performance --folders 10 --pages 10

This will create 10 sub-folders, each containing 10 pages.

--depth

Folder depth.

1 = Create folders once on root level

2 = Create also folder again in folders of root level

And so one.

This means the amount of folders and files raises exponential:

sphinx-performance --pages 10 --folders 10 --depth 0  # 10 pages, 0 folders
sphinx-performance --pages 10 --folders 10 --depth 1  # 110 pages, 10 folders
sphinx-performance --pages 10 --folders 10 --depth 2  # 1110 pages, 100 folders

--ref

Allows to select a preconfigured testproject setup by its reference name.

Each test project can provide its own set of configurations.

There is no common naming rule, how such references shall be named in Sphinx-Performance. For details take a look at the variable references of performance.py file of each test project. For embedded projects:

• Basic config file

• Needs config file

• Theme config file

Example:

sphinx-performance --project theme --ref alabaster --ref rtd --ref pydata --ref furo --browser --depth 2

Config parameters set by a reference configuration can be easily overwritten by just using the parameter in the command line call:

sphinx-performance --ref small --folders 50

Parameter matrix

All project parameters can be set multiple times, so that tests gets executed for each given parameter.

sphinx-performance creates also a configuration matrix, if multiple parameters are given multiple times.

This --pages 1 --pages 5 --dummies 1 --dummies 20 would run 4 tests with:

1. pages = 1 and dummies = 1

2. pages = 1 and dummies = 20

3. pages = 5 and dummies = 1

4. pages = 5 and dummies = 20

$ sphinx-performance --pages 1 --pages 5 --dummies 1 --dummies 20

Running 4 test configurations.


Running 4 test configurations.

─────────────────────────────────── Run 1/4 ────────────────────────────────────
Project:         basic
Core/s:          1
Builder:         html
Config:          pages: 1, dummies: 1, sphinx: 5.1, folders: 0, depth: 1
Info:            #dummies: 1, #pages: 1, #indexes: 1, #folders: 0

Docs path:       /tmp/tmpo4sukwi4
Docs files:      2 rst files with 0.34 kB
Docs setup:      0.02 s

Deps setup:      0.84 s
WARNING: html_static_path entry '_static' does not exist

Build files:     28 files with 557.71 kB
File max️:        281.82 kB by /tmp/tmpo4sukwi4/_build/_static/jquery-3.6.0.js
File min:        0.04 kB by /tmp/tmpo4sukwi4/_build/_static/custom.css
File Ø:          19.92 kB (0.03 s)
Reading time:    0.01 s
Writing time:    0.07 s
Build Duration:  0.75 s
─────────────────────────────────── Run 2/4 ────────────────────────────────────
Project:         basic
Core/s:          1
Builder:         html
Config:          pages: 1, dummies: 20, sphinx: 5.1, folders: 0, depth: 1
Info:            #dummies: 20, #pages: 1, #indexes: 1, #folders: 0

Docs path:       /tmp/tmp0sva3267
Docs files:      2 rst files with 1.88 kB
Docs setup:      0.01 s

Deps setup:      0.83 s
WARNING: html_static_path entry '_static' does not exist

Build files:     28 files with 577.71 kB
File max️:        281.82 kB by /tmp/tmp0sva3267/_build/_static/jquery-3.6.0.js
File min:        0.04 kB by /tmp/tmp0sva3267/_build/_static/custom.css
File Ø:          20.63 kB (0.03 s)
Reading time:    0.01 s
Writing time:    0.07 s
Build Duration:  0.79 s
─────────────────────────────────── Run 3/4 ────────────────────────────────────
Project:         basic
Core/s:          1
Builder:         html
Config:          pages: 5, dummies: 1, sphinx: 5.1, folders: 0, depth: 1
Info:            #dummies: 5, #pages: 5, #indexes: 1, #folders: 0

Docs path:       /tmp/tmpkl140raf
Docs files:      6 rst files with 1.00 kB
Docs setup:      0.02 s

Deps setup:      0.84 s
WARNING: html_static_path entry '_static' does not exist

Build files:     40 files with 598.47 kB
File max️:        281.82 kB by /tmp/tmpkl140raf/_build/_static/jquery-3.6.0.js
File min:        0.04 kB by /tmp/tmpkl140raf/_build/_static/custom.css
File Ø:          14.96 kB (0.02 s)
Reading time:    0.03 s
Writing time:    0.10 s
Build Duration:  0.81 s
─────────────────────────────────── Run 4/4 ────────────────────────────────────
Project:         basic
Core/s:          1
Builder:         html
Config:          pages: 5, dummies: 20, sphinx: 5.1, folders: 0, depth: 1
Info:            #dummies: 100, #pages: 5, #indexes: 1, #folders: 0

Docs path:       /tmp/tmpecxwcg7g
Docs files:      6 rst files with 8.67 kB
Docs setup:      0.02 s

Deps setup:      0.84 s
WARNING: html_static_path entry '_static' does not exist

Build files:     40 files with 698.00 kB
File max️:        281.82 kB by /tmp/tmpecxwcg7g/_build/_static/jquery-3.6.0.js
File min:        0.04 kB by /tmp/tmpecxwcg7g/_build/_static/custom.css
File Ø:          17.45 kB (0.03 s)
Reading time:    0.15 s
Writing time:    0.13 s
Build Duration:  1.02 s
─────────────────────────────────── RESULTS ────────────────────────────────────
╭───────────────┬───────────┬───────────┬───────────┬───────────╮
│       #       │   Run 1   │   Run 2   │   Run 3   │   Run 4   │
├───────────────┼───────────┼───────────┼───────────┼───────────┤
│    runtime    │   0.75    │   0.79    │   0.81    │   1.02    │
│    project    │   basic   │   basic   │   basic   │   basic   │
│               │           │           │           │           │
│     depth     │     1     │     1     │     1     │     1     │
│    sphinx     │    5.1    │    5.1    │    5.1    │    5.1    │
│     pages     │     1     │     1     │     5     │     5     │
│    folders    │     0     │     0     │     0     │     0     │
│   parallel    │     1     │     1     │     1     │     1     │
│    builder    │   html    │   html    │   html    │   html    │
│    dummies    │     1     │    20     │     1     │    20     │
│               │           │           │           │           │
│    #pages     │     1     │     1     │     5     │     5     │
│   #indexes    │     1     │     1     │     1     │     1     │
│   #dummies    │     1     │    20     │     5     │    100    │
│   #folders    │     0     │     0     │     0     │     0     │
│               │           │           │           │           │
│ max file size │ 281.82 kB │ 281.82 kB │ 281.82 kB │ 281.82 kB │
│    # files    │    28     │    28     │    40     │    40     │
│ min file size │  0.04 kB  │  0.04 kB  │  0.04 kB  │  0.04 kB  │
│ avg file size │ 19.92 kB  │ 20.63 kB  │ 14.96 kB  │ 17.45 kB  │
│ reading time  │  0.01 s   │  0.01 s   │  0.03 s   │  0.15 s   │
│ writing time  │  0.07 s   │  0.07 s   │  0.10 s   │  0.13 s   │
│ avg file time │  0.03 s   │  0.03 s   │  0.02 s   │  0.03 s   │
│  folder size  │ 557.71 kB │ 577.71 kB │ 598.47 kB │ 698.00 kB │
│               │           │           │           │           │
╰───────────────┴───────────┴───────────┴───────────┴───────────╯

Overall runtime: 3.38 seconds.

--temp

Defines the location of the folder to use for creating the temporary test project folders.

By default a operating system specific is chosen, on Linux this is /tmp.

--temp can also be a relative path.

So a sphinx-performance --temp . will create a test-folder like tmp0zmq3js2 in the current working directory.

Use --temp together with --keep, to keep the test-folder at an easy accessible location.

--debug

Shows the out put of Sphinx build and Python dependency installation step:

$ sphinx-performance --debug

Running 1 test configurations.


Running 1 test configurations.

─────────────────────────────────── Run 1/1 ────────────────────────────────────
Project:         basic
Core/s:          1
Builder:         html
Config:          sphinx: 5.1, dummies: 10, pages: 10, folders: 0, depth: 1
Info:            #dummies: 100, #pages: 10, #indexes: 1, #folders: 0

Docs path:       /tmp/tmp88nbzcaa
Docs files:      11 rst files with 8.96 kB
Docs setup:      0.04 s

──────────────────────── Installing dependencies START ─────────────────────────
Requirement already satisfied: sphinx==5.1 in /home/docs/checkouts/readthedocs.org/user_builds/sphinx-performance/envs/latest/lib/python3.8/site-packages (from -r /tmp/tmp88nbzcaa/requirements.txt (line 1)) (5.1.0)
Requirement already satisfied: sphinxcontrib-applehelp in /home/docs/checkouts/readthedocs.org/user_builds/sphinx-performance/envs/latest/lib/python3.8/site-packages (from sphinx==5.1->-r /tmp/tmp88nbzcaa/requirements.txt (line 1)) (1.0.4)
Requirement already satisfied: sphinxcontrib-devhelp in /home/docs/checkouts/readthedocs.org/user_builds/sphinx-performance/envs/latest/lib/python3.8/site-packages (from sphinx==5.1->-r /tmp/tmp88nbzcaa/requirements.txt (line 1)) (1.0.2)
Requirement already satisfied: sphinxcontrib-jsmath in /home/docs/checkouts/readthedocs.org/user_builds/sphinx-performance/envs/latest/lib/python3.8/site-packages (from sphinx==5.1->-r /tmp/tmp88nbzcaa/requirements.txt (line 1)) (1.0.1)
Requirement already satisfied: sphinxcontrib-htmlhelp>=2.0.0 in /home/docs/checkouts/readthedocs.org/user_builds/sphinx-performance/envs/latest/lib/python3.8/site-packages (from sphinx==5.1->-r /tmp/tmp88nbzcaa/requirements.txt (line 1)) (2.0.1)
Requirement already satisfied: sphinxcontrib-serializinghtml>=1.1.5 in /home/docs/checkouts/readthedocs.org/user_builds/sphinx-performance/envs/latest/lib/python3.8/site-packages (from sphinx==5.1->-r /tmp/tmp88nbzcaa/requirements.txt (line 1)) (1.1.5)
Requirement already satisfied: sphinxcontrib-qthelp in /home/docs/checkouts/readthedocs.org/user_builds/sphinx-performance/envs/latest/lib/python3.8/site-packages (from sphinx==5.1->-r /tmp/tmp88nbzcaa/requirements.txt (line 1)) (1.0.3)
Requirement already satisfied: Jinja2>=2.3 in /home/docs/checkouts/readthedocs.org/user_builds/sphinx-performance/envs/latest/lib/python3.8/site-packages (from sphinx==5.1->-r /tmp/tmp88nbzcaa/requirements.txt (line 1)) (3.1.2)
Requirement already satisfied: Pygments>=2.0 in /home/docs/checkouts/readthedocs.org/user_builds/sphinx-performance/envs/latest/lib/python3.8/site-packages (from sphinx==5.1->-r /tmp/tmp88nbzcaa/requirements.txt (line 1)) (2.16.1)
Requirement already satisfied: docutils<0.20,>=0.14 in /home/docs/checkouts/readthedocs.org/user_builds/sphinx-performance/envs/latest/lib/python3.8/site-packages (from sphinx==5.1->-r /tmp/tmp88nbzcaa/requirements.txt (line 1)) (0.18.1)
Requirement already satisfied: snowballstemmer>=1.1 in /home/docs/checkouts/readthedocs.org/user_builds/sphinx-performance/envs/latest/lib/python3.8/site-packages (from sphinx==5.1->-r /tmp/tmp88nbzcaa/requirements.txt (line 1)) (2.2.0)
Requirement already satisfied: babel>=1.3 in /home/docs/checkouts/readthedocs.org/user_builds/sphinx-performance/envs/latest/lib/python3.8/site-packages (from sphinx==5.1->-r /tmp/tmp88nbzcaa/requirements.txt (line 1)) (2.12.1)
Requirement already satisfied: alabaster<0.8,>=0.7 in /home/docs/checkouts/readthedocs.org/user_builds/sphinx-performance/envs/latest/lib/python3.8/site-packages (from sphinx==5.1->-r /tmp/tmp88nbzcaa/requirements.txt (line 1)) (0.7.13)
Requirement already satisfied: imagesize in /home/docs/checkouts/readthedocs.org/user_builds/sphinx-performance/envs/latest/lib/python3.8/site-packages (from sphinx==5.1->-r /tmp/tmp88nbzcaa/requirements.txt (line 1)) (1.4.1)
Requirement already satisfied: requests>=2.5.0 in /home/docs/checkouts/readthedocs.org/user_builds/sphinx-performance/envs/latest/lib/python3.8/site-packages (from sphinx==5.1->-r /tmp/tmp88nbzcaa/requirements.txt (line 1)) (2.31.0)
Requirement already satisfied: packaging in /home/docs/checkouts/readthedocs.org/user_builds/sphinx-performance/envs/latest/lib/python3.8/site-packages (from sphinx==5.1->-r /tmp/tmp88nbzcaa/requirements.txt (line 1)) (23.1)
Requirement already satisfied: importlib-metadata>=4.4 in /home/docs/checkouts/readthedocs.org/user_builds/sphinx-performance/envs/latest/lib/python3.8/site-packages (from sphinx==5.1->-r /tmp/tmp88nbzcaa/requirements.txt (line 1)) (6.8.0)
Requirement already satisfied: pytz>=2015.7 in /home/docs/checkouts/readthedocs.org/user_builds/sphinx-performance/envs/latest/lib/python3.8/site-packages (from babel>=1.3->sphinx==5.1->-r /tmp/tmp88nbzcaa/requirements.txt (line 1)) (2023.3.post1)
Requirement already satisfied: zipp>=0.5 in /home/docs/checkouts/readthedocs.org/user_builds/sphinx-performance/envs/latest/lib/python3.8/site-packages (from importlib-metadata>=4.4->sphinx==5.1->-r /tmp/tmp88nbzcaa/requirements.txt (line 1)) (3.16.2)
Requirement already satisfied: MarkupSafe>=2.0 in /home/docs/checkouts/readthedocs.org/user_builds/sphinx-performance/envs/latest/lib/python3.8/site-packages (from Jinja2>=2.3->sphinx==5.1->-r /tmp/tmp88nbzcaa/requirements.txt (line 1)) (2.1.3)
Requirement already satisfied: charset-normalizer<4,>=2 in /home/docs/checkouts/readthedocs.org/user_builds/sphinx-performance/envs/latest/lib/python3.8/site-packages (from requests>=2.5.0->sphinx==5.1->-r /tmp/tmp88nbzcaa/requirements.txt (line 1)) (3.2.0)
Requirement already satisfied: idna<4,>=2.5 in /home/docs/checkouts/readthedocs.org/user_builds/sphinx-performance/envs/latest/lib/python3.8/site-packages (from requests>=2.5.0->sphinx==5.1->-r /tmp/tmp88nbzcaa/requirements.txt (line 1)) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /home/docs/checkouts/readthedocs.org/user_builds/sphinx-performance/envs/latest/lib/python3.8/site-packages (from requests>=2.5.0->sphinx==5.1->-r /tmp/tmp88nbzcaa/requirements.txt (line 1)) (2.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /home/docs/checkouts/readthedocs.org/user_builds/sphinx-performance/envs/latest/lib/python3.8/site-packages (from requests>=2.5.0->sphinx==5.1->-r /tmp/tmp88nbzcaa/requirements.txt (line 1)) (2023.7.22)
─────────────────────── Installing dependencies FINISHED ───────────────────────
Deps setup:      0.83 s
Call:            /home/docs/checkouts/readthedocs.org/user_builds/sphinx-perform
ance/envs/latest/bin/sphinx-build -a -E -v -j 1 -b html /tmp/tmp88nbzcaa 
/tmp/tmp88nbzcaa/_build 
───────────────────────── Building documentation START ─────────────────────────
Running Sphinx v5.1.0
making output directory... done
WARNING: html_static_path entry '_static' does not exist
locale_dir /tmp/tmp88nbzcaa/locales/en/LC_MESSAGES does not exists
locale_dir /tmp/tmp88nbzcaa/locales/en/LC_MESSAGES does not exists
locale_dir /tmp/tmp88nbzcaa/locales/en/LC_MESSAGES does not exists
building [mo]: all of 0 po files
building [html]: all source files
updating environment: locale_dir /tmp/tmp88nbzcaa/locales/en/LC_MESSAGES does not exists
[new config] 11 added, 0 changed, 0 removed
reading sources... [  9%] index
reading sources... [ 18%] page_0
reading sources... [ 27%] page_1
reading sources... [ 36%] page_2
reading sources... [ 45%] page_3
reading sources... [ 54%] page_4
reading sources... [ 63%] page_5
reading sources... [ 72%] page_6
reading sources... [ 81%] page_7
reading sources... [ 90%] page_8
reading sources... [100%] page_9

looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [  9%] index
writing output... [ 18%] page_0
writing output... [ 27%] page_1
writing output... [ 36%] page_2
writing output... [ 45%] page_3
writing output... [ 54%] page_4
writing output... [ 63%] page_5
writing output... [ 72%] page_6
writing output... [ 81%] page_7
writing output... [ 90%] page_8
writing output... [100%] page_9

generating indices... genindex done
writing additional pages... search done
copying static files... done
copying extra files... done
dumping search index in English (code: en)... done
dumping object inventory... done
build succeeded, 1 warning.

The HTML pages are in ../../../../../../../../../tmp/tmp88nbzcaa/_build.
─────────────────────── Building documentation FINISHED ────────────────────────

Build files:     55 files with 747.01 kB
File max️:        281.82 kB by /tmp/tmp88nbzcaa/_build/_static/jquery-3.6.0.js
File min:        0.04 kB by /tmp/tmp88nbzcaa/_build/_static/custom.css
File Ø:          13.58 kB (0.02 s)
Reading time:    0.00 s
Writing time:    0.00 s
Build Duration:  1.10 s

Deleting project /tmp/tmp88nbzcaa/_build
─────────────────────────────────── RESULTS ────────────────────────────────────
╭───────────────┬───────────╮
│       #       │   Run 1   │
├───────────────┼───────────┤
│    runtime    │   1.10    │
│    project    │   basic   │
│               │           │
│     depth     │     1     │
│   parallel    │     1     │
│    builder    │   html    │
│    sphinx     │    5.1    │
│    dummies    │    10     │
│     pages     │    10     │
│    folders    │     0     │
│               │           │
│   #dummies    │    100    │
│    #pages     │    10     │
│   #indexes    │     1     │
│   #folders    │     0     │
│               │           │
│ reading time  │  0.00 s   │
│  folder size  │ 747.01 kB │
│ avg file size │ 13.58 kB  │
│ avg file time │  0.02 s   │
│ max file size │ 281.82 kB │
│ writing time  │  0.00 s   │
│ min file size │  0.04 kB  │
│    # files    │    55     │
│               │           │
╰───────────────┴───────────╯

Overall runtime: 1.10 seconds.

--keep

Does not delete the created, temporary test folders and prints their location.

$ sphinx-performance --keep

Running 1 test configurations.


Running 1 test configurations.

─────────────────────────────────── Run 1/1 ────────────────────────────────────
Project:         basic
Core/s:          1
Builder:         html
Config:          sphinx: 5.1, dummies: 10, pages: 10, folders: 0, depth: 1
Info:            #dummies: 100, #pages: 10, #indexes: 1, #folders: 0

Docs path:       /tmp/tmp_b7nfxfd
Docs files:      11 rst files with 8.96 kB
Docs setup:      0.04 s

Deps setup:      0.83 s
WARNING: html_static_path entry '_static' does not exist

Build files:     55 files with 747.01 kB
File max️:        281.82 kB by /tmp/tmp_b7nfxfd/_build/_static/jquery-3.6.0.js
File min:        0.04 kB by /tmp/tmp_b7nfxfd/_build/_static/custom.css
File Ø:          13.58 kB (0.02 s)
Reading time:    0.18 s
Writing time:    0.21 s
Build Duration:  1.08 s
─────────────────────────────────── RESULTS ────────────────────────────────────
╭───────────────┬───────────╮
│       #       │   Run 1   │
├───────────────┼───────────┤
│    runtime    │   1.08    │
│    project    │   basic   │
│               │           │
│     depth     │     1     │
│    sphinx     │    5.1    │
│    folders    │     0     │
│    builder    │   html    │
│    dummies    │    10     │
│     pages     │    10     │
│   parallel    │     1     │
│               │           │
│   #folders    │     0     │
│   #indexes    │     1     │
│    #pages     │    10     │
│   #dummies    │    100    │
│               │           │
│ min file size │  0.04 kB  │
│ avg file time │  0.02 s   │
│ writing time  │  0.21 s   │
│    # files    │    55     │
│  folder size  │ 747.01 kB │
│ avg file size │ 13.58 kB  │
│ reading time  │  0.18 s   │
│ max file size │ 281.82 kB │
│               │           │
╰───────────────┴───────────╯

Overall runtime: 1.08 seconds.

--browser

Opens each generated documentation in the browser after the build:

sphinx-performance --browser

This sets also automatically --keep.

--csv

Stores the result table in a given CSV-file

If the file exists, it gets overwritten:

sphinx-performance –csv results.csv

sphinx-analysis

sphinx-analysis builds a single project, but is able to create runtime and memory profiles of the Sphinx build. It also allows to present the profiled data in different views, like tables, flamegraphs and summaries.

For runtime profiling, cProfile is used. memray is the used memory profiler, which also supports a live viewer.

Warning

Don’t use more than one profiler at the same time, as they would influence each other. It’s better to reuse the same project config and just replace the profiler.

The options for setting up the project are the same as for sphinx-performance, except csv, which is not supported, and snakeviz, which was renamed to flamegraph.

Example calls:

sphinx-analysis --project --pages 10 --folders 3 --depth 2 --memray --flamegraph
sphinx-analysis --project --pages 10 --folders 3 --depth 2 --runtime --stats
sphinx-analysis --project needs --needs 40 --needtables 2 --pages 5 --folders 2 --depth 1 --pyinstrument --tree
sphinx-analysis --project basic --pyinstrument --tree --sphinx-events
sphinx-analysis --ref medium --runtime --summary

--runtime

Starts the runtime profiling of the Sphinx build. Results are stored inside the file runtime_all.prof.

--memray

Profiles the memory consumption of the Sphinx build. Results are stored in the file memray_all.prof.

--memray-live

Nearly same as --memray, but waits with the profiling till a memray viewer is connected. Creates no result file.

A memray viewer can be opened in another terminal by executing memray live 13167.

--pyinstrument

Uses the pyinstrument profiler and saves the profile in a file called pyinstrument_profile.json.

--stats

Prints some statistics at the end of the build.

Supported by: --runtime and --memray.

runtime stats

memray stats

--summary

Prints the memray summary at the end of the build.

Supported by: --memray.

memray summary

--flamegraph

Opens a browser to show a flamegraph view of the captured profile.

For runtime snakeviz is used, and for memray the memray flamegraph.

Supported by: --runtime and --memray.

runtime flamegraph

memray flamegraph

--silent

Can be used to answer questions to the user automatically with yes.

This may happen, if e.g. multiple projects are configured to be used, and sphinx-analysis asks the user to confirm this.

--tree

Creates a pyinstrument_profile.html file, which shows a runtime tree, profiled by --pyinstrument.

Supported by: --pyinstrument.

pyinstrument tree in HTML file

-- tree-filter

Remove nodes that represent less than X seconds of the total time. This works on the tree structure generated by pyinstrument, so it can only be used together with --pyinstrument --tree.

--sphinx-events

Generates a JSON runtime summary for each Sphinx event. Only usable if --pyinstrument is given. This works by monkey patching the Sphinx EventManager.emit function, so the call tree reveals which Sphinx event got fired. The modification is visible in the call tree.

The JSON output file pyinstrument_sphinx_events.json is generated into the current working directory.

The feature can also collect custom frames, not related to Sphinx events. For this the (hard coded) variable CUSTOM_FRAMES_BY_REPORT_NAME in sphinx_performance/sphinx_events.py can be adapted. Any improvements to this variable in form of a PR is welcome.

The custom pyinstrument frames will be added to the output JSON and can be used to get quick information for any Sphinx build step. Just like for events, the output JSON will contain unique function runtimes of all nodes below the given frame. The dictionary keys will be used also in the output JSON for humans to understand what it is.

The feature can be used to also analyze runtime of extensions that are not called through the Sphinx event system, but by docutils, e.g. sphinxcontrib-plantuml. When running in a CI context, the output JSON can be used to quickly see performance problems or improvements introduced by new PRs.