# PRD: Conda Native-DLL Bundling Repair Loop (pygrib/eccodes and the General Case) **Status:** Requirements 1, 2, 3, 5, and 6 implemented (2026-08-04); Requirement 4's regression test extended but not yet confirmed passing in real CI. See CLAUDE.md's Item 24 entry for current confirmation status. Originally written in response to CLAUDE.md Active Backlog Item 24 (found via `self.layered_e2e.chain`'s real CI evidence once the cStringIO warnfix fix let that test reach this far for the first time -- see `docs/agent-closed-backlog.md`'s Item 22 entry for the discovery trail); thawed from cold storage the same day the owner brought it forward -- see `docs/agent-cold-storage.md`'s own entry for the original shelve-and-thaw reasoning. **Owner:** Supervisor (Python_vs_Windows) **Related:** `docs/agent-closed-backlog.md` Item 22 (the layered E2E test that found this), CLAUDE.md Active Backlog Item 24, `docs/prd-av-safe-build-path.md` (the repo's other PRD stored in cold storage -- same "cheapest option first, reactive not proactive" design philosophy applies here) --- ## Research Findings ### Finding 1 -- `hook-gribapi.py`'s actual source confirmed; `pygrib` and `gribapi` are architecturally independent bindings; Requirement 1's CI experiment CONFIRMED the free lunch does not exist **Updated 2026-08-04 with the actual hook source and the pygrib/gribapi relationship, both confirmed via direct research (not extrapolated).** `pyinstaller-hooks-contrib` (already an installed dependency of this repo's build -- confirmed in real CI logs as version `2026.6`) ships `hook-gribapi.py` at `_pyinstaller_hooks_contrib/stdhooks/hook-gribapi.py` (no `src/` prefix, correcting the original guess below). Its actual mechanism, read directly from source: 1. Collects `gribapi`'s data files (`collect_data_files('gribapi')` -- header files like `eccodes.h`/`gribapi.h`). 2. At BUILD TIME, imports `gribapi` in the build environment and queries `gribapi.bindings.library_path` to locate the real eccodes shared library on disk. 3. Resolves a relative path via `_resolveCtypesImports`; on failure, logs a warning (does not hard-fail the build). 4. **Has explicit Windows-aware directory-preservation logic** -- calculates a destination relative to the package parent, producing `eccodes.libs` on Linux or `eccodes` on Windows. This directly contradicts an earlier, narrower assumption that the hook's shared-library awareness was "non-Windows only" -- it is not; Windows is a named, handled case in this exact code path. 5. If the library comes from the separate `eccodeslib` PyPI package (binary wheels for eccodes itself, `>= 2.37.0`), that package is added to hidden imports too, for `findlibs.find()`. **Two separate claims here -- one proven by the package-relationship research itself, one now CONFIRMED by Requirement 1's own real CI experiment rather than left as an inference.** *Proven, via the package-relationship research itself*: `pygrib` (jswhit/pygrib) and `gribapi`/`eccodes-python` (ECMWF's official bindings, conda-forge package `python-eccodes`) are two INDEPENDENT, unrelated Python bindings to the same underlying `eccodes` C library, from different upstream projects -- `pygrib` does not import or depend on `gribapi` at all; it talks to `libeccodes`/`eccodes.dll` directly through its own compiled extension. This means `hook-gribapi.py` has no reason to fire UNFORCED for a pygrib-only project: PyInstaller's static analysis genuinely never sees a `gribapi` import to trigger the hook on, since none exists in pygrib's own source. *Confirmed empirically, not just inferred*: whether FORCING `--hidden-import=gribapi` would still help was the open question this reasoning alone could not settle. Requirement 1's own CI experiment (`tests/selfapps_gribapi_hook_probe.ps1`, PR #415, merged 2026-08-04) answered it directly: `self.gribapi_hook_probe.hidden_import` reports `details.conclusive=true`, `details.hiddenImportHelped=false` -- both the control build (no forced import) and the experiment build (`--hidden-import=gribapi`) showed IDENTICAL outcomes (`bundledDll=false`, `exeExit=1`, `dllLoadFailed=true`). Forcing the hidden import did NOT cause `hook-gribapi.py` to place `eccodes.dll` anywhere `pygrib`'s own `_pygrib.cp314-win_amd64.pyd` extension could find it. The free lunch does not exist; this is now closed, not merely expected. **The empirical test was worth running, per explicit owner instruction to verify via real CI rather than rely on analysis alone -- and it delivered exactly the informative negative result the analysis above predicted.** The conda-forge package providing `gribapi` (`python-eccodes`) built for win-64 as expected (confirmed via the feedstock's own CI badge matrix), so the experiment ran cleanly: `python-eccodes` installed alongside `pygrib`+`eccodes` in the test conda env, `--hidden-import=gribapi` was forced on the experiment build, and the resulting EXE's DLL situation did not change at all relative to the control. This ruled out the free-lunch shortcut with real evidence and directly motivated building the general repair loop (Requirements 2-6 below). ### Finding 2 -- `--collect-binaries=PKG` is the wrong shape for this failure; it does not cross package boundaries Confirmed via PyInstaller's own documentation: `--collect-binaries=PKG` collects binaries found *inside PKG's own installed package directory*. It is not a dependency-walker across unrelated packages. Since `eccodes.dll` lives in conda's shared `Library\bin`, not inside `pygrib`'s own site-packages folder, `--collect-binaries=pygrib` would very likely find nothing -- the DLL isn't where that flag looks. `--collect-binaries=eccodes` is a closer guess (if eccodes's own Python package tree contains anything binary) but still uncertain, since the DLL's actual location is the shared `Library\bin` convention, not a package-local one. `--collect-all=PKG` has the identical directory-scoping problem, plus it is much heavier (bundles all of a package's data+binaries+ submodules, not just the one missing DLL) -- rejected for the same reason this repo's own `--collect-submodules` mechanism deliberately excludes heavy stacks like torch/tensorflow (see `docs/agent-lessons-learned.md`'s "Pre-build --collect-submodules must be DOUBLE-gated" entry). ### Finding 3 -- this repo already has a working, shipped example of exactly this failure class: `libexpat` under REQ-007 -- but it is single-purpose, not generalized `run_setup.bat`'s `:compute_collect_flags`-adjacent REQ-007 block globs for `%HP_PY_DIR%Library\bin\libexpat*.dll` and, if found, emits `--add-binary ";."` on the PyInstaller build command line, logged as `[INFO] REQ-007: bundling conda libexpat DLL for pyexpat: ...`. This is real, proven, shipped evidence that the "explicit `--add-binary` glob against conda's `Library\bin`" approach genuinely works for at least one native DLL in this exact codebase. It is entirely hardcoded to `libexpat` specifically -- the glob pattern, variable names (`HP_PYI_EXPAT`), and log text are all expat-literal, with no generality and no "is this DLL actually needed" gate beyond "does the file exist in `Library\bin`." Copy-pasting this pattern for `eccodes` would work for the ONE named DLL, but inherits the same single-DLL blind spot Finding 4 below addresses. ### Finding 4 -- the existing `--hidden-import` auto-recovery loop is the right architectural template, not the collect/add-binary flags in isolation This repo already ships a REACTIVE, bounded, iterative repair loop for a structurally similar problem: `:hidden_import_recover` (REQ-016 Slice 2) re-runs a failed frozen EXE, scans its captured output for a *specific* error signature (`ModuleNotFoundError` for an installed module), and only then attempts a targeted rebuild -- bounded to 3 attempts, with a tried-list so the same fix is never attempted twice (see `docs/agent-lessons-learned.md`'s "--hidden-import auto-recovery must stay STRICT" entry for the full design). This is deliberately **reactive, never proactive** -- matching the same design principle `docs/prd-av-safe-build-path.md`'s Tier A already established ("let Nuitka do its own internal compiler discovery... do not build an independent detection check"). A DLL-bundling repair loop should reuse this exact shape, not just the `--add-binary`/glob mechanics from Finding 3: re-run the EXE, scan output for `ImportError: DLL load failed... The specified module could not be found` (or the equivalent PyInstaller build-time `WARNING: Library not found: could not resolve '.dll'` signal, which is available even earlier -- at build time, before any smoke-test run is needed at all, see the Requirements section below for why this matters), locate and bundle the NAMED missing DLL from `%CONDA_PREFIX%\Library\bin`, rebuild, and re-run. If a DIFFERENT DLL is now missing (a transitive dependency of the first), the SAME loop catches it on the next iteration -- this is what elegantly solves Finding 3's single-DLL blind spot without needing to solve "walk the full transitive dependency graph up front," which would be a much larger and more fragile undertaking (Windows has no single, universally-available tool for this the way `ldd` serves on Linux -- `dumpbin` requires a Visual Studio install, which this bootstrapper cannot assume is present). ### Finding 5 -- the build-time warning is a stronger, earlier signal than the runtime crash, and may not even need a "post-smoke" trigger Unlike `--hidden-import` auto-recovery (which has no earlier signal than the runtime `ModuleNotFoundError` -- PyInstaller's static analysis cannot tell whether a genuinely-installed module will actually be imported by the running program), THIS failure class already announces itself at BUILD time: `WARNING: Library not found: could not resolve 'eccodes.dll', dependency of '...\pygrib\_pygrib.cp314-win_amd64.pyd'`. This is available in the PyInstaller build log immediately after the build step, before the EXE is ever smoke-run. A design that reacts to this build-time warning (parallel to how `parse_warn.py` already parses PyInstaller's warn file for a DIFFERENT purpose -- missing Python modules, not missing binaries) could trigger the repair BEFORE the first smoke-test run ever happens, instead of waiting for a guaranteed-to-fail runtime crash first. This is a real design choice to make explicitly (see Requirements, P0 item 2) -- build-time detection is strictly cheaper (skips one guaranteed-failing smoke run) but needs a new parser for a different PyInstaller output stream than the existing `parse_warn.py` handles; runtime detection reuses `:hidden_import_recover`'s existing scan-and-react shape almost exactly, at the cost of one wasted smoke-test cycle per missing DLL. ### Finding 6 -- interaction with the AV-Safe Build Path (Nuitka Tier A / requirement 9 optimized build), confirmed from existing shipped mechanisms Traced directly against `docs/agent-interconnect.md`'s existing "AV-Safe Build Path Tier A and hidden-import auto-recovery" and "requirement 9" sections -- not extrapolated, this repo already shipped both mechanisms and their guards: - **The existing `--hidden-import` auto-recovery loop (`:hidden_import_recover`) explicitly skips itself when `dist\.exe` was built by Nuitka, not PyInstaller** (`HP_NUITKA_FALLBACK_USED` guard, added same-day Tier A shipped -- see `docs/agent-lessons-learned.md`'s "--hidden-import auto-recovery must stay STRICT" entry). Reason: its repair action is a PyInstaller-specific rebuild flag (`--hidden-import=X`); Nuitka has a structurally different, unwired mechanism (`--include-module`/`--follow-import-to`). **The new DLL-bundling repair loop this PRD proposes needs the identical guard, for the identical reason** -- `--add-binary`/`--collect-binaries` are PyInstaller-specific flags with no Nuitka equivalent wired up here. Any implementation MUST check `HP_NUITKA_FALLBACK_USED` (or an equivalent "this dist\.exe was not built by PyInstaller" signal) before attempting a rebuild, mirroring `:hidden_import_recover`'s own early-exit shape -- see Requirement 6 below. - **Requirement 9's elective "optimized build?" Nuitka upsell (`:offer_optimized_build`) does not "fall back" to PyInstaller on failure -- there is nothing to fall back to, because the original PyInstaller EXE is never touched until success is fully confirmed.** It builds to a distinct temp filename, runs its own internal verify-launch against that temp file, and only on CONFIRMED success does it `move /y` the temp file over `dist\.exe`. Every failure branch (build fails, verify-run fails, or the swap itself fails) routes through the shared `:optbuild_cleanup` label, which deletes only the temp file and leaves the original, already-working PyInstaller EXE completely untouched -- confirmed by `tests/selfapps_optimized_build.ps1`'s `forcefail`/ `swapfail` scenarios, which explicitly re-execute the original EXE afterward to prove this. So: if a user's Nuitka-optimized build attempt fails for any reason, the answer is "it just doesn't swap," not "it falls back" -- the distinction matters because there is no active recovery step, just an absence of replacement. - **Interesting side observation, not a requirement to chase**: Nuitka's `--standalone` mode uses its own Windows DLL dependency walker, generally understood to be more thorough at discovering native library dependencies than PyInstaller's warn-file-based static analysis. It is plausible (not verified in this research pass) that a user who hits this exact pygrib/eccodes failure under PyInstaller and then accepts the requirement-9 "optimized build?" upsell would find the Nuitka build bundles `eccodes.dll` correctly without this PRD's repair loop ever being needed for that specific run. This does NOT reduce the value of building the loop (Requirement 9 is elective, declined by default in CI and not something every user will accept, and the PyInstaller build is still the PRIMARY path this repo's Prime Directive depends on) -- noted here only because it is the same "does an existing mechanism already solve this for free" question Finding 1 asks about `hook-gribapi.py`, just for a different existing mechanism. Not verified; not planned as a dedicated investigation unless this PRD is picked up and the cheap check is worth adding then. ## Problem Statement `pygrib`'s conda-forge build genuinely installs and PyInstaller's build genuinely succeeds when the uv-to-conda provider cascade lands a user on the conda provider for a package needing native C-library dependencies conda ships separately (via its own `Library\bin` shared-library convention). But the frozen EXE then fails at runtime (`ImportError: DLL load failed`) because PyInstaller does not discover or bundle that native DLL dependency -- silently defeating the whole point of having successfully cascaded to conda in the first place. This is currently the ONLY remaining gap keeping `self.layered_e2e.chain` (the real end-to-end test proving "why you'd need conda over uv") at `pass:false`; the two mechanisms that test actually exists to prove (REQ-009 cascade, warnfix repair) both now genuinely pass. ## Goals 1. When a frozen EXE fails at runtime with a native-DLL-load-failure signature (or, per Finding 5, when PyInstaller's own build log already warns about an unresolved `.dll` dependency), locate the missing DLL in the conda environment's `Library\bin` and bundle it, then rebuild and re-verify -- reactively, never proactively, matching this repo's established design philosophy. 2. Handle the "one DLL might not be the whole fix" case (transitive native dependencies) via bounded iteration, reusing the shape of the existing `--hidden-import` auto-recovery loop, not by trying to solve full dependency-graph resolution up front. 3. Scope the actual repair ACTION to the conda provider specifically (`%CONDA_PREFIX%\Library\bin` has no meaning under uv/embed/venv/system) -- detection can stay provider-agnostic and cheap. 4. Before building any new mechanism, verify Finding 1's cheaper alternative (`--hidden-import=gribapi` triggering an existing upstream hook) does not already solve this for free. ## Non-Goals - **Solving this for every possible native-DLL-dependent package speculatively.** Scope to what the real, observed failure needs (`eccodes.dll`, and whatever its own transitive chain turns out to be) first; generalize only if a second, unrelated conda-native-DLL gap is actually observed (see the Open Question below on narrow-vs-general scope). - **A full Windows DLL dependency-graph walker** (the `dumpbin`/`ldd`-equivalent problem). Too heavy, needs tooling this bootstrapper cannot assume is present. The iterative reactive-loop design (Finding 4) is the deliberate alternative to building this. - **Fixing this under uv/embed/venv/system.** These providers install from PyPI wheels, which are expected to vendor their own native dependencies inside the wheel (the whole point of the manylinux/Windows-wheel packaging standard) -- this specific failure shape is conda-specific by construction. A package that ships a broken/incomplete wheel is a different, unrelated problem this PRD does not address. - **Upstreaming a fix to `pyinstaller-hooks-contrib` itself.** Worth considering later if Finding 1 confirms a real gap there, but out of scope for this bootstrapper's own PRD -- track separately if it becomes relevant. ## Requirements -- IMPLEMENTED 2026-08-04 Requirements 1-3 and 6 done; Requirement 4's regression test extended but not yet CI-confirmed; Requirement 5 done in the same pass. See CLAUDE.md's Item 24 entry for current confirmation status. Kept below as the original design record, not rewritten to past tense throughout. 1. **DONE (PR #415, merged 2026-08-04).** Verified Finding 1 first, with zero new bootstrapper code, via its own small CI-only PR: added `python-eccodes` (the conda-forge package providing `gribapi`, confirmed to build for win-64) alongside `pygrib` in a minimal test build, forced `--hidden-import=gribapi`, and observed whether `eccodes.dll` ended up bundled / the EXE ran. Confirmed negative, matching Finding 1's updated assessment (pygrib and gribapi are independent bindings) -- `self.gribapi_hook_probe.hidden_import` shows `conclusive=true`, `hiddenImportHelped=false`. The experiment was kept isolated in its own PR, not bundled with requirement 3's implementation, so the negative result didn't block on or get entangled with unrelated work. **conda-forge only, matching this repo's established channel policy** (README.md/CLAUDE.md: `--override-channels -c conda-forge` on every conda install, no `defaults` channel contamination) -- the experiment PR's own conda env creation did this (`--override-channels -c conda-forge` in its `conda create` call). 2. **DONE.** Decided build-time detection (Finding 5): `:dll_bundle_recover` reacts to PyInstaller's own build-time `Library not found: could not resolve` warning line, before the smoke run, rather than waiting for the guaranteed runtime crash -- reuses `~hidden_import_scan.py`'s general shape (`tools/dll_bundle_scan.py`) but targets this different PyInstaller output stream instead of the missing-module warn file. 3. **DONE.** New repair loop (`:dll_bundle_recover`, `tools/dll_bundle_scan.py`), gated to `HP_ENV_MODE=conda` for the actual bundling action. Mirrors `:hidden_import_recover`'s bounded-iteration, tried-list shape (see Finding 4) -- not a single-shot fix. Detection stays provider-agnostic (cheap, always checked first via a `--detect` mode); the DLL-glob-and- `--add-binary` action itself no-ops (with a clear log line) under any non-conda provider or a Nuitka-built EXE (Requirement 6). 4. **Regression test extended, not yet confirmed passing in real CI.** `tests/selfapps_layered_e2e.ps1` gained a 4th mechanism (`mech4Pass`) modeled on how it already reaches the real `pygrib`/`eccodes` failure today (no new simulated-failure hook needed -- the failure is already reliably reproducible via the existing test's own uv-to-conda cascade). Asserts the loop bundles the DLL, rebuilds, and the final EXE genuinely runs and prints its token -- this is the acceptance criterion that should finally flip `self.layered_e2e.chain`'s `chainPass` to `True`. See CLAUDE.md's Item 24 entry for current real-CI confirmation status. 5. **DONE.** Documentation: CLAUDE.md's Item 24 entry written up (mirroring how Item 22 and Item 23 were each closed with a full mechanism trace); `docs/agent-interconnect.md` gained a new "Conda native-DLL bundling repair loop" section analogous to "Tier A and hidden-import auto-recovery" describing how this new loop relates to the existing one; `docs/agent-ndjson.md` updated to describe the 4th mechanism on `self.layered_e2e.chain`'s existing row. 6. **DONE.** `HP_NUITKA_FALLBACK_USED` early-exit guard (see Finding 6): `:dll_bundle_recover` checks this flag and skips (not attempt-and-fail) before any rebuild action, identical in shape to `:hidden_import_recover`'s own guard. No CI test currently exercises a Nuitka-built EXE hitting a native-DLL failure specifically -- `tests/selfapps_nuitka_tiera_hidden_skip.ps1` is the sibling regression test for `:hidden_import_recover`'s OWN Nuitka guard, a genuinely different code path; it does not exercise `:dll_bundle_recover`'s guard. If a dedicated test is ever added here, it should assert the skip fires, mirroring that sibling test's own shape. ## Level Check (explicitly requested by the owner) **Still on-target, not scope creep.** This repo's own established narrative -- proven out in `selfapps_cascade.ps1`'s header comment ("the main gain: conda is the strongest solver") and the whole design of the layered E2E test -- is showcasing concrete, real reasons a user would need conda's deeper bootstrapping over plain uv/pip. `pygrib` (zero Windows PyPI wheels, real conda-forge binaries) is already exactly that story for the *install* step. Item 24 is that same story's natural continuation into the *packaging* step: not just "conda can install what pip can't," but "the bootstrapper can also freeze a conda-installed native-dependent package into a working standalone EXE" -- arguably an even sharper, more specific value proposition, since PyInstaller-plus-conda-native-packages is a genuinely gnarly combination many people struggle with even by hand. Finishing this closes the loop on the exact demonstration this repo has been building toward, rather than diverging from it. ## Open Questions - **Narrow (pygrib/eccodes-only) fix vs. a general "conda native-DLL repair loop" that could catch a FUTURE, different package's missing-DLL gap too -- genuinely undecided, needs the maintainer's call.** A narrow fix (hardcode the eccodes DLL name/glob, mirroring REQ-007's libexpat pattern exactly) is cheap and directly closes the one known failure. A general version (detect ANY `Library not found: could not resolve 'X.dll'` pattern, not just eccodes by name, and bundle whatever `X` turns out to be) costs more to build and test but would silently cover any future conda-forge package hitting the same PyInstaller gap, without needing its own dedicated PRD each time. Given Finding 4's iterative-loop design is barely more general than a single-DLL fix once built (the loop shape does not care whether the DLL name is hardcoded or parsed from the warning), this leans toward "build it general from the start is not much extra cost over building it narrow" -- but this is exactly the kind of proportionality judgment `docs/prd-av-safe-build-path.md`'s own "Notes from Claude" section on pin-generalization warns against deciding unilaterally. Flagged here rather than resolved. ## Confidence Assessment **Green-leaning-Yellow (upgraded 2026-08-04 after reading `hook-gribapi.py`'s actual source and confirming the pygrib/gribapi package relationship) -- real, well-evidenced problem, sound design direction, and the scope question is now a probability estimate rather than a total unknown.** The reactive-loop architecture (Finding 4) is sound and directly reuses a proven, shipped pattern from this same codebase -- low design risk there. Finding 1's "free lunch" question is no longer unverified-in-principle -- it is now assessed as LIKELY negative based on real research (pygrib and gribapi are independent bindings to the same C library, confirmed via their respective upstream docs), but Requirement 1's cheap CI experiment should still confirm this empirically before ruling it out for good, since a probability estimate is not proof. Remaining genuine uncertainty is narrower than before: exactly how many transitive native DLLs eccodes pulls in beyond itself (hdf5/netcdf/proj were all observed in the same conda env in the original failure), which Finding 4's iterative design handles without needing to know the answer up front.