6927 stories
·
166 followers

Hugo van Kemenade: Three times faster with lazy imports

1 Comment

PEP 810 proposes “explicit lazy imports” for Python 3.15:

Lazy imports defer the loading and execution of a module until the first time the imported name is used, in contrast to ‘normal’ imports, which eagerly load and execute a module at the point of the import statement.

By allowing developers to mark individual imports as lazy with explicit syntax, Python programs can reduce startup time, memory usage, and unnecessary work. This is particularly beneficial for command-line tools, test suites, and applications with large dependency graphs.

It’s not been accepted yet, but let’s try out the reference implementation on one of my CLI tools, pypistats.

Setup #

First fetch the reference implementation. From a CPython checkout:

git remote add LazyImportsCabal https://github.com/LazyImportsCabal/cpython
git fetch LazyImportsCabal
gco lazy # see https://hugovk.dev/blog/2025/my-most-used-command-line-commands/

Because we want to install NumPy and pandas, let’s pretend to be Python 3.14 so we can use the binary wheels instead of having to build from source:

--- a/Include/patchlevel.h
+++ b/Include/patchlevel.h
 /* Version parsed out into numeric values */
 /*--start constants--*/
 #define PY_MAJOR_VERSION 3
-#define PY_MINOR_VERSION 15
+#define PY_MINOR_VERSION 14
 #define PY_MICRO_VERSION 0
 #define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_ALPHA
 #define PY_RELEASE_SERIAL 0

 /* Version as a string */
-#define PY_VERSION "3.15.0a0"
+#define PY_VERSION "3.14.0a0"
 /*--end constants--*/
--- a/configure.ac
+++ b/configure.ac
-m4_define([PYTHON_VERSION], [3.15])
+m4_define([PYTHON_VERSION], [3.14])

Build non-debug CPython with optimisations:

GDBM_CFLAGS="-I$(brew --prefix gdbm)/include" \
 GDBM_LIBS="-L$(brew --prefix gdbm)/lib -lgdbm" \
 ./configure --enable-optimizations --with-lto \
 --with-system-libmpdec --config-cache \
 --with-openssl="$(brew --prefix openssl@3)" && make -s -j8

Install NumPy and pandas:

./python.exe -m pip install numpy pandas

And then an editable install of the CLI, because we’ll also test changing the imports:

./python.exe -m pip install -e ~/github/pypistats

Let’s check the dependencies with pipdeptree:

uvx "pipdeptree[graphviz]" --python ./python.exe --packages pypistats --graph-output svg > pipdeptree.svg

pypistats has seven direct dependencies, which result in a total of 41 dependencies six layers deep, not counting NumPy and pandas:

A tree of dependencies: seven wide, and about six layers deep.

Benchmarks #

Let’s benchmark running pypistats --help, which is meant to be quick, using hyperfine:

brew install hyperfine

Inline imports #

In the pypistats CLI, I had already applied the trick of moving heavier imports into the functions that call them (the PEP calls these “inline imports”).

Instead of the lazy keyword, I’m using the PYTHON_LAZY_IMPORTS env var here to make it easy to compare two different runs.

❯ hyperfine --warmup 10 --runs 20 --export-json out.json \
 "./python.exe -m pypistats --help" \
 "PYTHON_LAZY_IMPORTS=on ./python.exe -m pypistats --help"
Benchmark 1: ./python.exe -m pypistats --help
 Time (mean ± σ): 46.2 ms ± 1.1 ms [User: 38.8 ms, System: 6.4 ms]
 Range (min … max): 45.1 ms … 49.6 ms 20 runs

Benchmark 2: PYTHON_LAZY_IMPORTS=on ./python.exe -m pypistats --help
 Time (mean ± σ): 35.3 ms ± 0.5 ms [User: 29.5 ms, System: 4.8 ms]
 Range (min … max): 34.6 ms … 36.3 ms 20 runs

Summary
 PYTHON_LAZY_IMPORTS=on ./python.exe -m pypistats --help ran
 1.31 ± 0.04 times faster than ./python.exe -m pypistats --help

Plotted with plot_progression.py:

A progression chart of 20 runs for each benchmark: non-lazy runs are about 46 ms, lazy are about 35 ms.

From 46 to 35 milliseconds, or, 1.31 times faster, not bad.

Fully lazy #

But we no longer need the inline imports trick with PEP 810!

I modified the CLI so all imports are at the top, and also removed if TYPE_CHECKING: guards. Here’s a diff.

❯ hyperfine --warmup 10 --runs 20 --export-json out2.json \
 "./python.exe -m pypistats --help" \
 "PYTHON_LAZY_IMPORTS=on ./python.exe -m pypistats --help"
Benchmark 1: ./python.exe -m pypistats --help
 Time (mean ± σ): 104.1 ms ± 1.6 ms [User: 88.2 ms, System: 14.5 ms]
 Range (min … max): 101.9 ms … 109.5 ms 20 runs

Benchmark 2: PYTHON_LAZY_IMPORTS=on ./python.exe -m pypistats --help
 Time (mean ± σ): 35.7 ms ± 0.5 ms [User: 29.8 ms, System: 4.8 ms]
 Range (min … max): 34.7 ms … 36.5 ms 20 runs

Summary
 PYTHON_LAZY_IMPORTS=on ./python.exe -m pypistats --help ran
 2.92 ± 0.06 times faster than ./python.exe -m pypistats --help

A progression chart of 20 runs for each benchmark: non-lazy runs are about 104 ms, lazy are about 36 ms.

From 104 to 36 milliseconds, or 2.92 times faster, much better!


Header photo: “Lazy Man Fishing” at Cascade Locks on the Columbia River 05/1973 in the U.S. National Archives , with no known copyright restrictions.

Read the whole story
jepler
24 minutes ago
reply
Look, it's 30% faster (and possibly also subtly broken! There's a reason PEP 810 exposes a new syntax for explicit lazy imports!).. until I make the original code worse, in which case it's 3x faster.

but seriously, assuming this PEP is accepted for Python 3.15, then in 2030 (When Python 3.14 reaches EOL and the lazy import syntax can actually be WRITTEN into a package that supports any non-EOL python version [which is how I try to live my life*]) we'll be able to use explicit lazy imports and that'll be nice.

* Actually I try to support the Python version in Debian stable & oldstable + the one in Ubuntu LTS. But the cadence is such that this rule meant I supported 3.9 about as long as python.org did. If I had a motivation to dump 3.10 [ubuntu 22.04 oldstable] I might. In the first project I lookedd at that would mean I could drop a dependency on typing_extensions which I would not mind!
Earth, Sol system, Western spiral arm
Share this story
Delete

Help extract the 8087 Microcode ROM

1 Share
Comments
Read the whole story
jepler
17 hours ago
reply
Earth, Sol system, Western spiral arm
Share this story
Delete

A New Way to Make (Almost) Holograms with Lasers

1 Comment
An array of tiny parallel green lines appears over a steel surface. The white dot a laser beam is visible in the lower center of the picture.

The spectrum of laser technologies available to hackers has gradually widened from basic gas lasers through CO2 tubes, diode lasers, and now fiber lasers. One of the newer entries is the MOPA laser, which combines a laser diode with a fiber-based light amplifier. The diode’s pulse length and repetition rate are easy to control, while the fiber amplifier gives it enough power to do interesting things – including, as [Ben Krasnow] found, etch hologram-like diffraction gratings onto stainless steel.

Stainless steel works because it forms a thin oxide layer when heated, with a thickness determined by the temperature it reaches. The oxide layer creates thin-film interference with incoming light, letting the laser mark parts of a steel sheet with different colors by varying the intensity of heating. [Ben] wrote a script to etch color images onto steel using this method, and noticed in one experiment that one area seemed to produce diffraction patterns. More experimentation revealed that the laser could consistently make diffraction gratings out of parallel patterns of oxide lines. Surprisingly, the oxide layer seemed to grow mostly down into the metal, instead of up from the surface.

The pitch of the grating is perpendicular to the direction of the etched lines, and varying the line spacing changes the angle of diffraction, which should in theory be enough control to print a hologram with the laser. [Ben]’s first experiment in this general direction was to create a script that turned black-and-white photographs into shimmering matrices of diffraction-grating pixels, in which each pixel’s grating orientation was determined by its brightness. To add a parallax depth effect, [Ben] spread out images into a gradient in a diffraction grating, so that it produced different images at different angles. The images were somewhat limited by the minimum size required for the grating pixels, but the effect was quite noticeable.

Unfortunately, since the oxide layers grow down into the metal, [Ben] doubts whether the laser can etch molds for diffraction-grating chocolate. If you’re interested in more diffraction optics, check out these custom diffraction lenses or the workings of normal holograms.

Read the whole story
jepler
22 hours ago
reply
now I want to go buy a laser. except no way do I have a $10k budget.
Earth, Sol system, Western spiral arm
Share this story
Delete

Army General Says He's Using AI To Improve 'Decision-Making'

1 Comment
Maj. Gen. William Taylor told reporters at the Association of the US Army Conference in Washington this week that he and the Eighth Army he commands out of South Korea are regularly using AI for decision-making. Taylor said he has been asking AI chatbots to help build models for personal decisions that affect his organization and overall readiness. The general referred to his chatbot companion as "Chat" and said the technology has been useful for predictive analysis in logistical planning and operational purposes.
Read the whole story
jepler
2 days ago
reply
What's worse than sending "we attack at dawn" in plaintext? sending "should we attack at dawn?" in plaintext to a LLM.
Earth, Sol system, Western spiral arm
Share this story
Delete

Cartoons

1 Comment
From: Fran Blanche
Duration: 7:14
Views: 263

Yea, I watched cartoons. You got a problem with that?
Join Team FranLab!!!! Become a patron and help support my YouTube Channel on Patreon! http://www.patreon.com/frantone
Pro Tip: Always use a web browser to manage your Patreon Account, or an Android mobile device, but do not use the Apple Mobile App. Thanks!

#franlab
- Music by Fran Blanche -

Fran's Science Blog - http://www.frantone.com/designwritings/design_writings.html
FranArt Website - http://www.contourcorsets.com

Read the whole story
jepler
3 days ago
reply
well that's in no way a creepy thumbnail
Earth, Sol system, Western spiral arm
Share this story
Delete

Flatpak Doesn't Work in Ubuntu 25.10, But a Fix is Coming

1 Comment
"It's not just you: Flatpak flat-out doesn't work in the new Ubuntu 25.10 release," writes the blog OMG Ubuntu: While Flatpak itself can be installed using apt, trying to install Flatpaks with Flatpak from the command-line throws a "could not unmount revokefs-fuse filesystem" error, followed by "Child process exited with code 1". For those who've installed the Ubuntu 'Questing Quokka' and wanted to kit it out with their favourite software from Flathub, it's a frustrating road bump.

AppArmor, the tool that enforces Ubuntu's security policies for apps, is causing the issue. According to the bug report on Launchpad, the AppArmor profile for fusermount3 lacks the privileges it needs to work properly in Ubuntu 25.10. Fusermount3 is a tool Flatpak relies on to mount and unmount filesystems... This is a bug and it is being worked on. Although there's no timeframe for a fix, it is marked as critical, so will be prioritised.

The bug was reported in early September, but not fixed in time for this week's Ubuntu 25.10 release, reports Phoronix: Only [Friday] an updated AppArmor was pushed to the "questing-proposed" archive for testing. Since then... a number of users have reported that the updated AppArmor from the proposed archive will fix the Flatpak issues being observed. From all the reports so far it looks like that proposed update is in good shape for restoring Flatpak support on Ubuntu 25.10. The Ubuntu team is considering pushing out this update sooner than the typical seven day testing period given the severity of the issue.
More details from WebProNews: Industry insiders point out that AppArmor, Ubuntu's mandatory access control system, was tightened in this release to enhance security... This isn't the first time AppArmor has caused friction; similar issues plagued Telegram Flatpak apps in Ubuntu 24.04 LTS earlier this year, as noted in coverage from OMG Ubuntu.
Read the whole story
jepler
6 days ago
reply
oops, not a good bug to have shipped with.
Earth, Sol system, Western spiral arm
Share this story
Delete
Next Page of Stories