Add support for flakes "Lockable HTTP Tarball Protocol"

[pr2502]

Hi, i've run into missing support for the "tarball" type of flake input when importing a flake. I've noticed it's been mentioned before in #47 so i gave it a try because it doesn't look very complicated.

This is just a quick experiment, if you want this this feature included in npins i'm happy to write tests and anything else that is missing^^

For now i've tested it only manually on my nixos config using

mkdir test && cd test
cargo run -- init --bare
wget https://git.p2502.net/max/nixconf/raw/branch/main/flake.lock
cargo run -- import-flake
cargo run -- show

[piegamesde]

How does this works for tarballs which don't support that protocol? And would this solve the use case in #44?

[pr2502]

as is currently it errors out about the missing header. i haven't read #44 before but i have been thinking about a similar use case and i think it could be solved by gracefully handling missing "original url" with npins update (log it's impossible to update and ignore) and adding cli options to add a locked url directly like npins add tarball --locked <tarball url without the locking api support> and something to manually specify an updated url like npins set-url <input> --locked <tarball url without locking api support>. i can add it to this PR or make a follow up^^

[piegamesde]

I'm unsure, maybe it would be better to support these two use cases as different pin types and different subcommands? I feel like the lockable tarball protocol is more an obscure implementation detail and that most people won't know about it.

[pr2502]

could be made the other way around, rename the current url field to lockable_url and locked_url to just url and make an tarball that cannot be automatically updated the default and move the lockable tarball api behind a flag like npins add tarball --lockable <tarball url that supports auto updates via the link header> (or autodetect if the special Link: <flakeref>; rel="immutable" header is present)

[lf-]

I think the way nix handles tarball inputs by default is by assuming they're always locked and the flakehub protocol exists for using them for ones where flake.nix has a URL that does not return a stable result.

This structure could be reused, perhaps.

[pr2502]

i think that's where i have ended up now? npins add tarball <url> does HTTP HEAD against the URL and if it returns the Link: <flakeref>; rel="immutable" header we store <url> as a lockable_url field (so we can later update the Pin) and the flakeref as url. if there is no such header we fall back to storing <url> as url and assuming it will always return the same thing

now what i'm missing is tests (i'm working on a python script to mock the header) and some command that would easily allow updating the tarballs without lockable_url (something like npins set-url <name> <url>.

i'd also like to propagate the information that a Pin cannot self update to the user interface of npins update so it shows as "requires manual update" instead of "unaltered"

[piegamesde]

some command that would easily allow updating the tarballs without lockable_url (something like npins set-url .

The currently endorsed way of modifying pins is to simply re-add them with the same name, it would be weird to have a special command just for tarballs.

i'd also like to propagate the information that a Pin cannot self update to the user interface of npins update so it shows as "requires manual update" instead of "unaltered"

Hm, that's a tricky one, especially because for tarballs people would mostly expect that such a pin never updates? At the same time, this kinda clashes with an "update" command. Maybe we can weave the UX together with the new "freeze" feature in #78? As in, pins of such kind are always frozen per design or something like that.

[pr2502]

i've finally figured out the integration tests in nix i think^^ the implementation i've tried before broke npins update for tarballs so i've fixed that too.

since npins add can be used twice with the same name (which i didn't know, thanks!) i don't need to implement anything for updating. and the npins update UI i'll leave alone until #78 to not introduce any more conflicts?

i'll mark this as ready for review now. should i clean up the commit history or just squash it into one commit?

[piegamesde]

I merged the freeze pins PR so you aren't blocked on it. Though I am unsure if going down that route is still the best idea

[pr2502]

i know this is off topic on this PR but looking closer at the Pin freezing code after i've rebased on it i've noticed it's going to add "frozen": false to every pin on update and cause a lot off diff noise, could we add an annotation to skip serializing the default value? like this pr2502@02d35d0

[piegamesde]

Neat, would you mind extracting that commit into its own PR?

[pr2502]

sure^^ it's here #122

[piegamesde]

Sorry for the back and fourth on the design, it's an iterative process

[pr2502]

i don't understand what happened with the test, i didn't touch anything there and they work locally :<

[piegamesde]

Almost done now. If you want to squash and clean up the git history before merging, now would be a good time to do so

[pr2502]

i've just squashed everything into one commit, i don't think it's complicated enough to warrant splitting up. hopefully now without typos :D