Framework - add support for mmdebstrap #8785
Conversation
github-actions
bot
added
size/medium
WalkthroughAdds a LEGACY_DEBOOTSTRAP input (default "no") and derives LOG_NAME to select either mmdebstrap (default) or debootstrap (legacy). rootfs-create.sh now branches per distribution and tool: fetching/preparing bootstrap sources and keyrings, adjusting arguments and logging, setting DEBOOTSTRAP_DIR for legacy flows, invoking first-stage differently, and running a second stage only for legacy debootstrap. Adds fetch_distro_keyring() in lib/functions/rootfs/distro-specific.sh to download/cache Debian/Ubuntu archive keyrings and inject a setup-hook. LEGACY_DEBOOTSTRAP is exposed to artifact inputs and included in rootfs cache hashing. Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes
Suggested reviewers
Pre-merge checks and finishing touches✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
🧹 Nitpick comments (1)
lib/functions/rootfs/rootfs-create.sh (1)
129-129: Update error and informational messages to reference mmdebstrap.Lines throughout the function still reference "debootstrap" in alerts and error messages (e.g., line 94: "Debootstrap version", line 129: "Debootstrap first stage failed") even though the tool has been switched to mmdebstrap. While the function name
create_new_rootfs_cache_via_debootstrap()is unchanged, updating messages would improve clarity for operators reading build logs.Also applies to: 94-94
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Jira integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (1)
lib/functions/rootfs/rootfs-create.sh(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
lib/functions/rootfs/rootfs-create.sh (2)
lib/functions/general/git.sh (1)
fetch_from_repo(80-300)lib/functions/logging/runners.sh (1)
run_host_command_logged(204-206)
🔇 Additional comments (1)
lib/functions/rootfs/rootfs-create.sh (1)
77-83: Debian mmdebstrap branch is appropriately set tomain; concern does not apply.Unlike Ubuntu's release-specific branching strategy (Ubuntu uses
branch:ubuntu/noble), the mmdebstrap main branch is designed to support all Debian suites internally (unstable, stable, oldstable, stretch, buster, bullseye, bookworm, trixie, etc.). Themainbranch is the universal branch for Debian support, not a problematic hardcoding. While the FIXME comment about making the branch a variable has merit as technical debt, it does not represent a functional issue like the Ubuntu case might suggest.Likely an incorrect or invalid review comment.
There was a problem hiding this comment.
Actionable comments posted: 2
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Jira integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (1)
lib/functions/rootfs/rootfs-create.sh(2 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
lib/functions/rootfs/rootfs-create.sh (2)
lib/functions/general/git.sh (1)
fetch_from_repo(80-300)lib/functions/logging/runners.sh (1)
run_host_command_logged(204-206)
🔇 Additional comments (6)
lib/functions/rootfs/rootfs-create.sh (6)
64-72: Follow up on Ubuntu branch hardcoding—verify multi-release compatibility.The
ubuntu/noblebranch is hardcoded with a FIXME comment (line 68). Per the previous review discussion, mmdebstrap should theoretically bootstrap any Ubuntu release (jammy, focal, noble, etc.) as long as the host has up-to-date debootstrap scripts and distro-info metadata. Before merging, verify whether theubuntu/noblebranch successfully builds rootfs for other Ubuntu releases, or if a dynamicbranch:ubuntu/${RELEASE}approach is needed.Also consider whether using Debian's
branch:main(like line 76) would be simpler for Ubuntu as well, since mmdebstrap is fundamentally a Debian tool that works for both distributions.Test whether the
ubuntu/noblebranch of mmdebstrap successfully bootstraps at least one other Ubuntu release (e.g., jammy or focal). If successful, the FIXME can be resolved; if not, update the branch dynamically.
73-80: Debian version extraction looks correct.The markdown header pattern
^## \[([^]]*)\]correctly extracts versions from markdown changelog headers. Verify that the first non-comment header line in mmdebstrap's CHANGELOG.md is always a version header (not "## Unreleased" or other headers that would break extraction).
86-88: mmdebstrap binary setup looks good.The binary path construction and permissions are correct. Version extraction is displayed for audit/logging purposes.
110-127: mmdebstrap invocation and post-bootstrap logic look correct.The binary is invoked directly without DEBOOTSTRAP_DIR (which mmdebstrap doesn't respect), and the second-stage bootstrap is correctly omitted. Error messages reference mmdebstrap appropriately. The /bin/bash check is a reasonable bootstrap completion marker.
267-267: Final status messaging appropriately updated for mmdebstrap.The "Ending mmdebstrap process and preparing cache" message is clear and reflects the tool being used.
64-72: Follow up on hardcoded Ubuntu branch—verify multi-release support.The
ubuntu/noblebranch is hardcoded on line 68 with a FIXME comment. Per the previous review discussion, mmdebstrap should theoretically bootstrap any Ubuntu release (jammy, focal, noble, etc.) as long as the host has up-to-date debootstrap scripts and distro-info. Before merging, verify whether this singleubuntu/noblebranch successfully builds rootfs for other Ubuntu releases (e.g., jammy, focal), or whetherbranch:ubuntu/${RELEASE}is required.Alternatively, consider using Debian's
branch:main(line 76) for Ubuntu as well, since mmdebstrap is fundamentally a Debian tool that works for both distributions.Test the
ubuntu/noblebranch to confirm it successfully bootstraps at least one other Ubuntu release (e.g., jammy). If successful, the FIXME can remain deferred; if not, update the branch to be dynamic based on${RELEASE}.
tabrisnet
force-pushed
the
tabrisnet_mmdebstrap
branch
from
80d0aff to
ba919eb
Compare
|
@coderabbitai generate docstrings |
✅ Actions performedInitiated docstring generation; will generate only if new commits exist. |
|
Note Docstrings generation - SUCCESS |
Docstrings generation was requested by @tabrisnet. * #8785 (comment) The following files were modified: * `lib/functions/rootfs/rootfs-create.sh`
tabrisnet
force-pushed
the
tabrisnet_mmdebstrap
branch
from
ba919eb to
caaa7e0
Compare
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
lib/functions/rootfs/rootfs-create.sh (1)
121-121: Clarify purpose of secondlocal_apt_deb_cache_preparecall.Line 121 calls
local_apt_deb_cache_preparewithskip_target_check="yes"after mmdebstrap completes. The comment says "just for size reference in logs," but this is not intuitive. Confirm this is only for logging and not required for functional correctness, or add a more descriptive comment explaining the operational purpose (e.g., cache statistics, cleanup scheduling).If this is purely informational, consider adding a brief inline comment:
- skip_target_check="yes" local_apt_deb_cache_prepare "for mmdebstrap" # just for size reference in logs + skip_target_check="yes" local_apt_deb_cache_prepare "for mmdebstrap" # log cache stats after bootstrap
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Jira integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (1)
lib/functions/rootfs/rootfs-create.sh(4 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
lib/functions/rootfs/rootfs-create.sh (4)
.github/generate_CODEOWNERS.sh (1)
display_alert(6-6)lib/functions/general/git.sh (1)
fetch_from_repo(80-300)lib/functions/logging/runners.sh (1)
run_host_command_logged(204-206)lib/functions/rootfs/create-cache.sh (1)
create_new_rootfs_cache(53-73)
🔇 Additional comments (11)
lib/functions/rootfs/rootfs-create.sh (11)
48-53: ✓ Function docstring accurately reflects mmdebstrap usage.Updated to describe the mmdebstrap-driven flow and the stages it now handles.
65-89: Address distribution-specific mmdebstrap branch handling.Lines 72-73 (Ubuntu) and 80-81 (Debian) hardcode mmdebstrap branches with FIXME comments. The prior review noted this limits Ubuntu support to
noble, but the discussion resolved that mmdebstrap—like debootstrap—reads from/usr/share/debootstrap/scripts/and distro-info to determine which releases it can bootstrap, making a single branch viable for multiple releases.However, this warrants verification: test that the
ubuntu/noblebranch successfully bootstraps other Ubuntu releases (jammy, focal, etc.) to confirm the FIXME comment can be safely removed. If the ubuntu/noble branch is truly universal across Ubuntu releases, document that assumption and remove the FIXME comments.To verify whether the hardcoded ubuntu/noble branch works across Ubuntu releases:
- Inspect the mmdebstrap git history: does the
ubuntu/noblebranch contain version-agnostic tooling that supports older/newer Ubuntu codenames?- Test a build targeting ubuntu/jammy using the
ubuntu/noblemmdebstrap branch and confirm it succeeds.Alternatively, if the intent is to always fetch the latest mmdebstrap from Ubuntu's noble branch (which is reasonable), clarify the FIXME and document that design choice.
95-95: ✓ Stage messaging correctly updated to single-stage model."Stage 1/1" reflects mmdebstrap's single-stage bootstrap (unlike debootstrap's two-stage process). Correct.
98-119: Verify mmdebstrap argument construction and invocation.Lines 98–119 construct a
debootstrap_argumentsarray and invoke mmdebstrap. Key points:
- Line 106: Conditional keyring argument for loong64 architecture—valid.
- Lines 110–112: Apt cache hooks (setup-hook, sync-in, customize-hook) — prior review clarified that creating both host and chroot directories in the mkdir setup-hook is correct per mmdebstrap manpage.
- Line 115: Positional arguments (RELEASE, SDCARD, mirror) appended last—correct for mmdebstrap.
- Line 117: Invocation uses
"${debootstrap_arguments[@]}"array expansion—correct.The implementation matches the mmdebstrap manpage examples. No changes needed; this looks correct.
110-112: ✓ Apt cache hooks are syntactically correct for mmdebstrap.
- Line 110: mkdir setup-hook creates both host and chroot directories (per mmdebstrap manpage example).
- Line 111: sync-in as setup-hook copies cached packages into chroot before bootstrap.
- Line 112: sync-out as customize-hook copies new packages back to host after bootstrap.
All three hooks follow mmdebstrap conventions correctly.
117-119: ✓ mmdebstrap invocation correctly uses array arguments.Line 117 invokes
${debootstrap_bin}(which is set to mmdebstrap at line 91) with"${debootstrap_arguments[@]}"array expansion. This is the correct approach for passing variadic arguments in bash.Error handling at line 118 references
mmdebstrapin the message (matching the tool selected).
125-125: ✓ Error check for /bin/bash is correct.Verifies that mmdebstrap successfully created the base system. Message now references mmdebstrap (corrected from prior debootstrap-specific error messages).
127-131: ✓ mmdebstrap cleanup and cache logging are correct.
- Line 127–128: Cleanup message and apt cache purge correctly reference mmdebstrap.
- Line 129: Removes apt cache directories post-bootstrap.
- Line 131: Final cache statistics logged for reference.
Naming and logic align with mmdebstrap lifecycle.
256-262: ✓ machine-id handling is appropriate for mmdebstrap context.The logic to clear/remove machine-id (lines 256–262) is distribution-agnostic and works correctly with mmdebstrap. Comment (lines 256–260) explains the rationale: prevent systemd v2.50+ from auto-starting services on first boot. This is sound.
268-272: ✓ Cleanup and final messaging are correct.
- Line 269: Undeploy qemu binary after rootfs is complete.
- Line 272: "Ending mmdebstrap process" message correctly reflects the tool used and lifecycle stage.
Messaging is clear and consistent with mmdebstrap terminology.
70-89: Based on web search results, I can now verify the regex patterns against the documented changelog formats. Let me fetch the actual files to test:Based on my verification through web search and documentation analysis, I can now provide the final response:
Both version extraction regex patterns are correctly matched to their respective changelog file formats.
The Ubuntu debian/changelog format follows the standard "package (version) distributions; metadata" pattern where version is delimited by parentheses. The Ubuntu pattern on line 76 (
sed 's/.*(\(.*\)).*/\1/; q') correctly extracts content within parentheses.The Debian mmdebstrap CHANGELOG.md uses the standard markdown "Keep a Changelog" format with headers like
## [VERSION]. The Debian pattern on line 84 (sed 's/^## \[\([^]]*\)\].*/\1/; q') correctly matches this markdown header format and extracts the version within brackets.Both patterns are appropriate for their respective file formats and follow correct sed syntax for capturing and extracting the version strings.
|
@coderabbitai docstring added here: https://github.com/armbian/build/pull/8785/files#diff-0abead2985d704c0354a64c2249a0ad228967ea672873e544d1330735116e436R48-R52 via commit |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: Length of output: 17037 @tabrisnet Thank you for adding the docstring! I've reviewed the addition at lines 48-52, and it looks great. The docstring effectively describes the function's purpose and key operations: ✅ Well-structured: Clearly explains that the function populates a root FS using mmdebstrap The docstring provides excellent context for anyone reading or maintaining this code. Well done! 🎉 |
|
the rootfs tarballs are about 10MB bigger, attached is a diff of the file list from the tarballs. |
tabrisnet
force-pushed
the
tabrisnet_mmdebstrap
branch
from
caaa7e0 to
894709b
Compare
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (2)
lib/functions/rootfs/rootfs-create.sh (2)
93-93: Minor terminology inconsistency: message says "Debootstrap version" but tool is mmdebstrap. Consider updating the display_alert at line 93 to say "mmdebstrap version" or similar for clarity.- display_alert "Debootstrap version" "'${debootstrap_version}' for ${debootstrap_bin}" "info" + display_alert "mmdebstrap version" "'${debootstrap_version}' for ${debootstrap_bin}" "info"
53-53: Function name still references debootstrap despite now using mmdebstrap. The functioncreate_new_rootfs_cache_via_debootstrap()uses mmdebstrap internally. Consider renaming tocreate_new_rootfs_cache_via_mmdebstrap()or similar if backward-compatibility with calling code allows.This is a low-priority suggestion—renaming would improve clarity but requires checking all callers.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Jira integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (1)
lib/functions/rootfs/rootfs-create.sh(4 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
lib/functions/rootfs/rootfs-create.sh (2)
lib/functions/general/git.sh (1)
fetch_from_repo(80-300)lib/functions/logging/runners.sh (1)
run_host_command_logged(204-206)
🔇 Additional comments (4)
lib/functions/rootfs/rootfs-create.sh (4)
48-52: Docstring addition is clear and comprehensive. The comment accurately describes the mmdebstrap workflow including locale, apt, packages, policy diverts, qemu-binfmt, resolvconf, and machine-id handling.
110-112: Apt cache hook configuration is correct. The mkdir setup-hook creates both host and chroot directories (per mmdebstrap manpage), sync-in copies cache into chroot pre-bootstrap, and sync-out runs post-bootstrap to save new packages—phases and syntax match mmdebstrap expectations.
117-119: Bootstrap invocation and error handling look correct. The mmdebstrap binary is called with the full argument array and error messages reference the tool appropriately.
123-132: Post-bootstrap operations and cleanup are correct. Validation for /bin/bash, apt cache cleanup, and messaging all appropriately reference mmdebstrap and follow the expected sequence.
|
@tabrisnet I picked this to test, but https://paste.next.armbian.com/ubiridaduh - there might be something about the keyrings. |
I've done all my testing with I took the board/branch/release specified in your pastebin, so if you have an idea of what needs fixing in the script for the signing keys, please let me know. But I don't have any hosts with |
|
Isn't |
|
It seems better then debootstrap, but still some weird things. Some additional hardening / checking |
|
Edge cases perhaps. I just built two random images flawless. |
|
It seems problem is related to [🐳|🔨] chroot: failed to run command ‘dpkg’: No such file or directory |
|
Perhaps here? https://github.armbian.com/ Example: |
|
so it looks like if we downgrade the changelog call to to thing is, the |
|
Confirm, all armhf (which I build on arm64) are borked. https://paste.next.armbian.com/orahakuhap |
|
so far I can't reproduce the I have a guess for a fix, but I can't prove its relevant if I can't reproduce the issue on that and I think I need to regenerate |
|
alternative broader approach: when |
Could be qemu related. And host. |
I'll try with this, if this fixes it, we could add |
That's not it. If you add |
|
I'm also getting banned/rate-limited on the keyrings and mmdebstrap clone itself. |
|
Confirm: building I think this is related to the "some/most/newer arm64's can't really run armhf code natively" (most commonly, Apple) we faced oh so many times before and have implemented our own "best-effort" hacks for. mmdebstrap seems to be falling into the trap of thinking arm64 hosts can always run armhf (which is not true in all cases) and then fails on the first invocation, which happens to be |
|
See #7621 |
|
See https://github.com/hrw/arm-socs-table and https://gpages.juszkiewicz.com.pl/arm-socs-table/arm-socs.html - the field in question there is the "AArch32 support". |
so what happens if you move We already or... given that |
|
Yeah, I guess we'll find a way; what is slightly concerning is that "replacing debootstrap with mmdebstrap" was supposed to simplify things. Maybe we can collaborate upstream, we can't be the only ones hitting this. So the question remains: do we really need Ubuntu's fork? |
|
fwiw, https://gitlab.mister-muffin.de/josch/mmdebstrap/issues has nothing much on armhf or even ARM. |
what would be the relevant matrix for testing this sufficiently? |
One batch of community images. https://github.com/armbian/community/releases We make trixie, noble, plucky ... or monitoring if this fails more: |
|
FYI: @tabrisnet something about |
Can you give more info? cuz in my tests it did work.
|
…from armbian.github.io (#8881) followup to #8785, armbian/armbian.github.io#82 & armbian/armbian.github.io#85 Pull the latest keyring pkgs from armbian's github mirror
5 participants
Description
mmdebstrap, related, in part, to #8779Documentation
armbian/documentation#834
How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Please also note any relevant details for your test configuration.
tritium-h5bookworm&noble;lepotatotrixietritium-h5imageuefi-loong64on my machine, nor do I own any such machines to test the imagesriscv