Profiled a one-file ESP-IDF compile on staging: 36.7 s total, of which
ldgen.py alone was 15.2 s - consistently, on every single build. It
re-parses all 48 component linker.lf fragments with pyparsing each time
(3.8 M _parseNoCache calls; parse_fragment_file 15.6 s under cProfile),
even though its output depends only on the sdkconfig and those
fragments, never on the user's source.
pyparsing's packrat cache was tried first and made it WORSE (15.2 s ->
27.4 s), so the fix is memoizing the whole run: a wrapper hashes the
argv plus the CONTENT of every file it references and reuses the
generated linker script when they are unchanged. Verified on staging:
15.2 s -> 0.08 s on a hit, byte-identical output, and correctly missing
when the sdkconfig or any linker.lf changes (restoring the sdkconfig
hits again - it is content-addressed, not mtime-based). Any surprise
falls through to the untouched upstream script, so a build can never
break because of the cache.
New hand-written code on the ESP32-P4 lane: 36.7 s -> 11.5 s.
f15834d14d