Skip to content
This repository was archived by the owner on Mar 23, 2025. It is now read-only.

add kernel cross build support for faster builds on x86_64-linux #98

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

DavHau
Copy link
Member

@DavHau DavHau commented Dec 12, 2024

This adds the option raspberry-pi-nix.kernel-build-system, which can be used to drastically decrease the build times.

Description of the option:

The build system to compile the kernel on.

Only the linux kernel will be cross compiled, while most of the derivations are still pulled from cache.nixos.org.

Use this in any of these cases:

  • you are building on an x86_64 system using binfmt_misc for aarch64-linux.
  • or if your x86_64 builder has a better CPU than your aarch64 builder.

This adds the option `raspberry-pi-nix.kernel-build-system`, which can be used to drastically decrease the build times.

Description of the option:

The build system to compile the kernel on.

Only the linux kernel will be cross compiled, while most of the derivations are still pulled from cache.nixos.org.

Use this if you cannot or don't want to use the nix-community cache and either:
  - you are building on an x86_64 system using binfmt_misc for aarch64-linux.
  - or if your x86_64 builder has a better CPU than your aarch64 builder.
@DavHau DavHau marked this pull request as draft December 12, 2024 16:01
@DavHau
Copy link
Member Author

DavHau commented Dec 12, 2024

Damn, there still seems to be an error building modules-shrunk:

       > root module: tpm-crb
       > modprobe: FATAL: Module tpm-crb not found in directory /nix/store/v3r107514clbb7wnx3w9i7bkyqf519x5-linux-rpi-aarch64-unknown-linux-gnu-6.6.54-modules/lib/modules/6.6.54
       For full logs, run 'nix log /nix/store/31yh7hrkmvs0xx2vmpvs190nrkiysig0-linux-rpi-aarch64-unknown-linux-gnu-6.6.54-modules-shrunk.drv'.

@DavHau DavHau marked this pull request as ready for review December 15, 2024 08:03
@DavHau
Copy link
Member Author

DavHau commented Dec 15, 2024

Damn, there still seems to be an error building modules-shrunk:

       > root module: tpm-crb
       > modprobe: FATAL: Module tpm-crb not found in directory /nix/store/v3r107514clbb7wnx3w9i7bkyqf519x5-linux-rpi-aarch64-unknown-linux-gnu-6.6.54-modules/lib/modules/6.6.54
       For full logs, run 'nix log /nix/store/31yh7hrkmvs0xx2vmpvs190nrkiysig0-linux-rpi-aarch64-unknown-linux-gnu-6.6.54-modules-shrunk.drv'.

Actually this problem was unrelated, and due to a mix of different nixpkgs in my config.

@DavHau
Copy link
Member Author

DavHau commented Dec 15, 2024

I think this is good to go

Copy link

@lluchs lluchs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This worked for me, thanks!

rpi-kernels = builtins.foldl'
(b: a: final.lib.recursiveUpdate b (rpi-kernel a))
{ };

rip-kernels-cross = buildSystem: builtins.foldl'
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is supposed to say rpi-kernels-cross

@ElvishJerricco
Copy link
Contributor

ElvishJerricco commented Feb 22, 2025

In my configurations, I use something like this:

final: prev:
let
  inherit (final) lib;
in
{
  pkgsUncross = lib.flip lib.mapAttrs final.lib.systems.examples (
    _: localSystem:
    import final.path {
      inherit (final) config overlays;
      inherit localSystem;
      crossSystem = final.hostPlatform;
    }
  );
}

Which allows me to just set boot.kernelPackages = pkgs.pkgsUncross.gnu64.someKernelPackages;. First of all, I think this is significantly cleaner than having an ad-hoc nixpkgs import as you do in this PR currently, because it allows the use of pkgsUncross to cross compile anything as the user sees fit. Secondly, you'll notice I provided a much fuller set of arguments in the import than you do; preserving the config, overlays, and hostPlatform makes the package set much more accurate to what the user might be expecting. For instance, if I set the linux-kernel.target in my hostPlatform, that wouldn't work without passing along the original hostPlatform, which is important if you're using something like EDK2 or u-boot's EFI implementation to get compressed kernels with EFI_ZBOOT.

  nixpkgs.hostPlatform = lib.recursiveUpdate (lib.systems.elaborate lib.systems.examples.aarch64-multiplatform) {
    linux-kernel.target = "vmlinuz.efi";
    linux-kernel.installTarget = "zinstall";
  };

But I'm sure there's other, less obscure edge cases.

I've thought about upstreaming pkgsUncross into nixpkgs, but I don't think that's the name I would want to use if I were to upstream it. I just liked the name because it's sorta the reverse of pkgsCross.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants