From e8df9abaced7a25ffc048544a1dda9e3b5366197 Mon Sep 17 00:00:00 2001 From: Richard Shadrach Date: Mon, 14 Apr 2025 09:03:55 -0400 Subject: [PATCH 1/4] WEB: Update benchmarks page --- web/pandas/community/benchmarks.md | 31 ++++++++---------------------- 1 file changed, 8 insertions(+), 23 deletions(-) diff --git a/web/pandas/community/benchmarks.md b/web/pandas/community/benchmarks.md index 5a8198a979d90..14a868d1410f6 100644 --- a/web/pandas/community/benchmarks.md +++ b/web/pandas/community/benchmarks.md @@ -11,7 +11,7 @@ kinds of benchmarks relevant to pandas: pandas benchmarks are implemented in the [asv_bench](https://github.com/pandas-dev/pandas/tree/main/asv_bench) directory of our repository. The benchmarks are implemented for the -[airspeed velocity](https://asv.readthedocs.io/en/v0.6.1/) (asv for short) framework. +[airspeed velocity](https://asv.readthedocs.io/en/latest/) (asv for short) framework. The benchmarks can be run locally by any pandas developer. This can be done with the `asv run` command, and it can be useful to detect if local changes have @@ -22,37 +22,22 @@ More information on running the performance test suite is found Note that benchmarks are not deterministic, and running in different hardware or running in the same hardware with different levels of stress have a big impact in the result. Even running the benchmarks with identical hardware and almost identical -conditions produces significant differences when running the same exact code. +conditions can produce significant differences when running the same exact code. ## pandas benchmarks servers -We currently have two physical servers running the benchmarks of pandas for every -(or almost every) commit to the `main` branch. The servers run independently from -each other. The original server has been running for a long time, and it is physically -located with one of the pandas maintainers. The newer server is in a datacenter +We currently have two setups running the benchmarks of pandas for every +(or almost every) commit to the `main` branch. One is run on GitHub actions +in the [asv-runner](https://github.com/pandas-dev/asv-runner/) repository. +The other is a physical server in a datacenter kindly sponsored by [OVHCloud](https://www.ovhcloud.com/). More information about pandas sponsors, and how your company can support the development of pandas is available at the [pandas sponsors]({{ base_url }}about/sponsors.html) page. Results of the benchmarks are available at: -- GitHub Actions results: [asv](https://pandas-dev.github.io/asv-runner/) -- OVH server: [asv](https://pandas.pydata.org/benchmarks/asv/) - -### Original server configuration - -The machine can be configured with the Ansible playbook in -[tomaugspurger/asv-runner](https://github.com/tomaugspurger/asv-runner). -The results are published to another GitHub repository, -[tomaugspurger/asv-collection](https://github.com/tomaugspurger/asv-collection). - -The benchmarks are scheduled by [Airflow](https://airflow.apache.org/). -It has a dashboard for viewing and debugging the results. -You’ll need to setup an SSH tunnel to view them: - -``` -ssh -L 8080:localhost:8080 pandas@panda.likescandy.com -``` +- asv-runner results: [asv](https://pandas-dev.github.io/asv-runner/) +- OVH server results: [asv](https://pandas.pydata.org/benchmarks/asv/) ### OVH server configuration From 278f8b64fee7eb7c55d16afabcf3e879db4b52e0 Mon Sep 17 00:00:00 2001 From: Richard Shadrach Date: Mon, 14 Apr 2025 09:04:40 -0400 Subject: [PATCH 2/4] Change subsection title --- web/pandas/community/benchmarks.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/web/pandas/community/benchmarks.md b/web/pandas/community/benchmarks.md index 14a868d1410f6..43ee01eed786f 100644 --- a/web/pandas/community/benchmarks.md +++ b/web/pandas/community/benchmarks.md @@ -24,7 +24,7 @@ running in the same hardware with different levels of stress have a big impact i the result. Even running the benchmarks with identical hardware and almost identical conditions can produce significant differences when running the same exact code. -## pandas benchmarks servers +## Automated benchmarks runners We currently have two setups running the benchmarks of pandas for every (or almost every) commit to the `main` branch. One is run on GitHub actions From 5d90464cbcaaa7de262adc6120eb6540fcf24575 Mon Sep 17 00:00:00 2001 From: Richard Shadrach Date: Mon, 14 Apr 2025 09:11:23 -0400 Subject: [PATCH 3/4] fixup --- web/pandas/community/benchmarks.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/web/pandas/community/benchmarks.md b/web/pandas/community/benchmarks.md index 43ee01eed786f..3e36f5db90f97 100644 --- a/web/pandas/community/benchmarks.md +++ b/web/pandas/community/benchmarks.md @@ -24,7 +24,7 @@ running in the same hardware with different levels of stress have a big impact i the result. Even running the benchmarks with identical hardware and almost identical conditions can produce significant differences when running the same exact code. -## Automated benchmarks runners +## Automated benchmark runners We currently have two setups running the benchmarks of pandas for every (or almost every) commit to the `main` branch. One is run on GitHub actions From 543f9047b0307efa3076dc59528e85a1e33f75cc Mon Sep 17 00:00:00 2001 From: Richard Shadrach Date: Sat, 19 Apr 2025 07:19:40 -0400 Subject: [PATCH 4/4] OVH-ectomy --- web/pandas/community/benchmarks.md | 33 +++++------------------------- 1 file changed, 5 insertions(+), 28 deletions(-) diff --git a/web/pandas/community/benchmarks.md b/web/pandas/community/benchmarks.md index 3e36f5db90f97..ddfcbd57d3d78 100644 --- a/web/pandas/community/benchmarks.md +++ b/web/pandas/community/benchmarks.md @@ -24,36 +24,13 @@ running in the same hardware with different levels of stress have a big impact i the result. Even running the benchmarks with identical hardware and almost identical conditions can produce significant differences when running the same exact code. -## Automated benchmark runners +## Automated benchmark runner -We currently have two setups running the benchmarks of pandas for every -(or almost every) commit to the `main` branch. One is run on GitHub actions -in the [asv-runner](https://github.com/pandas-dev/asv-runner/) repository. -The other is a physical server in a datacenter -kindly sponsored by [OVHCloud](https://www.ovhcloud.com/). More information about -pandas sponsors, and how your company can support the development of pandas is -available at the [pandas sponsors]({{ base_url }}about/sponsors.html) page. +The [asv-runner](https://github.com/pandas-dev/asv-runner/) repository automatically runs the pandas asv benchmark suite +for every (or almost every) commit to the `main` branch. It is run on GitHub actions. +See the linked repository for more details. The results are available at: -Results of the benchmarks are available at: - -- asv-runner results: [asv](https://pandas-dev.github.io/asv-runner/) -- OVH server results: [asv](https://pandas.pydata.org/benchmarks/asv/) - -### OVH server configuration - -The server used to run the benchmarks has been configured to reduce system -noise and maximize the stability of the benchmarks times. - -The details on how the server is configured can be found in the -[pandas-benchmarks repository](https://github.com/pandas-dev/pandas-benchmarks). -There is a quick summary here: - -- CPU isolation: Avoid user space tasks to execute in the same CPU as benchmarks, possibly interrupting them during the execution (include all virtual CPUs using a physical core) -- NoHZ: Stop the kernel tick that enables context switching in the isolated CPU -- IRQ affinity: Ban benchmarks CPU to avoid many (but not all) kernel interruption in the isolated CPU -- TurboBoost: Disable CPU scaling based on high CPU demand -- P-States: Use "performance" governor to disable P-States and CPU frequency changes based on them -- C-States: Set C-State to 0 and disable changes to avoid slower CPU after system inactivity +https://pandas-dev.github.io/asv-runner/ ## Community benchmarks