Using Charlie Marsh's UV (pip/venv replacement) as a nox backend drops the time taken to build docs from 22 second docs build to 4.2 seconds!
@henryiii This is honestly pretty shocking to me. I would have been confident that the bottleneck for pip was disk I/O and network latency, in which case rewriting in Rust would not really matter.
Is pip just slow now because of the resolver? Or is pip leaving a lot of performance on the table?
@pganssle @henryiii the latter, and uv is using a lot of smart caching techniques and doing things where code complexity provides bits of situational speedups that are all adding up.
And, it's making strong-ish assumptions about what to expect from index servers and such (which is good for PyPI but not generally).
@pradyunsg @henryiii Ah, maybe not so surprising, then. I feel like sometimes these, “We wrote it in
In the best cases of this, you can import a lot of the performance gains into the original project. In the worst examples, the way they got the performance was by cutting corners, and the speed comparison is basically a lie.
Any chance pip can get some of the same performance benefits upstream (even if only on the “happy path” of hitting pypi.org?)
@pganssle @pradyunsg @henryiii but yes, both Ruff and UV are designed with performance in mind, while Flake8 wasn’t (to the same degree). Someone could probably write a Python project that would only be a small (5x maybe?) bit slower than Ruff by using the same design philosophy. But it would be a major effort, like Ruff was, and a lot of people were very excited to contribute to a Rust project. (Fish rewrite noticed this too)
@pganssle @pradyunsg @henryiii i was most excited for the venv creation time, which is ~4 seconds for venv, 400 ms for virtualenv, and 25 ms for UV (note that Python takes 50 ms to start up and do absolutely nothing). PEP 517 build processes involve making several venvs. :) Python process startup time is one thing you can’t get around by writing good Python.