Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Heads: Add packaging & project metadata #681

Merged
merged 2 commits into from
Apr 10, 2025
Merged

Conversation

OPNA2608
Copy link
Contributor

@OPNA2608 OPNA2608 commented Mar 27, 2025

Closes #547

We're running upstream's build setup, which requires significant fetching & some patching to behave nicely.

  • We can't download things at build-time, so most of the downloads that heads, and subproject that heads fetches & builds as part of its build process, are extracted & collected into deps.nix, and placed/patched in where needed before starting the build. Or in the case of subprojects that get unpacked during the build, have our patch files prepared & put into place for heads to apply during the build.
  • Ahead of the build, heads inspects some things about the system and uses the information for scheduling the build, along with printing this information. For the sake of making it behave the same on different systems & at different times on the same system, we're patching some of those checks & outputs.
  • A single board target has been allow-listed, qemu-coreboot-fbwhiptail-tpm1-hotp, because:
    • Allowing too many boards will make CI blow up (it's 50 targets, and CI doesn't do a great job at splitting up CPU cores among them, so 10 jobs running in parallel want to occupy all 16 cores of the CPU... and everything comes to a crawl
    • Alot of boards are broken because we're missing firmware data. And we're missing the firmware data because fetchSubmodules is currently incomplete: fetchgit: pass --checkout when fetching submodules NixOS/nixpkgs#286228
  • A VM test has been added, to test booting the qemu-coreboot-fbwhiptail-tpm1-hotp ROM in qemu. We can't test full functionality, but we can at least test for successful booting & the first few menus
VM screenshots (menu images look broken, but OCR'd fine, so likely just an issue with the image taking in non-interactive sessions)

20250408_23h29m58s_grim
1-heads-no-os
2-heads-main-menu

@OPNA2608 OPNA2608 force-pushed the init/heads branch 2 times, most recently from 471c970 to 7776386 Compare March 27, 2025 23:26
@OPNA2608 OPNA2608 force-pushed the init/heads branch 7 times, most recently from 3a641ad to 8507f6a Compare April 3, 2025 22:25
@OPNA2608 OPNA2608 changed the title WIP heads: init at 0.2.1-unstable-2025-03-12 heads.*: init at 0.2.1-unstable-2025-03-12 Apr 3, 2025
@eljamm
Copy link
Contributor

eljamm commented Apr 4, 2025

Would it be possible to split this into smaller PRs? Both so that CI can build the packages properly without timing out and for us to review it better.

Comment on lines +51 to +57
# Theoretically, it should be possible to extract these in the updateScript by running the corresponding download
# scripts in /blobs/* with a curl/wget that prints out the given URL, and collecting all the requested URLs & their
# output names
blobs = [
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: I am unhappy to announce that while deps.nix is still 100% generated, the new blobs attrset in package.nix likely belongs in there logically… It was just easier to extract those few downloads with their peculiarities by hand than to hack together an automatable solution for now.

Unlike the mountain of dependencies that is deps.nix, manually updating these by skimming the scripts in /blobs/* should at least be fairly doable.

@OPNA2608
Copy link
Contributor Author

OPNA2608 commented Apr 4, 2025

Would it be possible to split this into smaller PRs? Both so that CI can build the packages properly without timing out and for us to review it better.

To help with CI load, I've only allowed a single board target to be generated for now - heads.qemu-coreboot-fbwhiptail-tpm1-hotp.

The bulk of the code here is still necessary for building even this mostly-emulated board target. The only things I could cut for now are:

  • All the blobs files, their collecting & patching of blobs-related code - that's only relevant for boards that have blobs
  • Maybe some of the coreboot entries in patches - only need the one that's relevant for the currently allowed board
  • I think the openssl patch is also not yet relevant for the board?

If you think cutting them for now is worth it, then I can go ahead with that.

@OPNA2608 OPNA2608 changed the title heads.*: init at 0.2.1-unstable-2025-03-12 heads.*: init at 0.2.1-unstable-2025-04-03 Apr 9, 2025
@OPNA2608 OPNA2608 marked this pull request as ready for review April 9, 2025 20:04
@OPNA2608 OPNA2608 changed the title heads.*: init at 0.2.1-unstable-2025-04-03 Heads: Add packaging & project metadata Apr 9, 2025
Comment on lines +14 to +16
options.programs.heads = {
enable = lib.mkEnableOption "symlinking the Heads ROM for qemu-coreboot-fbwhiptail-tpm1-hotp";
};
Copy link
Contributor

@fricklerhandwerk fricklerhandwerk Apr 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do have binary artefacts in our data model now! Ah, but complementing them with tests and examples seems not straightforward?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is used in the test, to have a fixed location where we can find the ROM at. @eljamm said that currently, it's not possible to refer to the binary attrset.

Also, referencing pkgs in binary (in order to point it at the artifact from the build), currently makes eval blow up, which is why it's commented out: #773

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@eljamm said that currently, it's not possible to refer to the binary attrset.

Because it's not mapped back, but we can do this if we need to.

Copy link
Contributor

@eljamm eljamm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me. As a next step, we need to fix the overview in #773 so we can use the binary type.

@eljamm eljamm merged commit ac7f6f0 into ngi-nix:main Apr 10, 2025
2 checks passed
@github-project-automation github-project-automation bot moved this to Done in Nix@NGI Apr 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Implement project metadata for Heads-OpenPGP
3 participants