I have to say that that "surprisingly simple" thing is happening more and more for me on NixOS as well.
Recently a customer wanted me to use Fedora (I never visited the RPM side of the world before), and after truly the worst installer I've ever used, the actually system was nice. I do like Cockpit.
But then I needed to install an initrd that would me unlock the full disk encryption via SSH (it's a remote headless box). It took me half a day and a forum post to get it to work. I wrote a full page of notes for next time. Then the Firewall: I hit a bug (plus user error) which left me wrestling with a non-existent Tailscale interface for a while (it warns you for non-existing interfaces, but not with only a case mismatch, it then lets you do everything as if the interface exists), but after some hours I was done setting the zones, another page of notes and commands to enter to get to the desired state.
These configurations are both 1 or 2 lines in a NixOS config file. And that "work" is now done for all my NixOS servers.
You could argue that NixOS hides a lot of complexity, but so do Dracut and Firewalld of course. Nix is difficult, it's a high level abstraction. But it also just a bunch of key-values, and write-once, deploy everywhere.
>> You could argue that NixOS hides a lot of complexity
They both have the same complexity in that scenario. Underneath it's very comparable configuration for both but Nixos provides an easy abstraction for that specific case.
If you can stay on the happy path with nixos then it's pretty lovely. I've even adopted nix-darwin for my mac's too.
I'd still deploy Redhat/Fedora over nixos on anything revenue generating though. The problem is when you have to come off the happy path in nixos and now you're debugging some interestingly written c++ code that evaluates a language that has a derivation expressing what you wanted done. Contrast with the redhat situation, it's simpler but less convenient in the general case.
> The problem is when you have to come off the happy path in nixos and now you're debugging some interestingly written c++ code that evaluates a language that has a derivation expressing what you wanted done.
I agree that Nix or NixOS are risky tools.
But it's not a problem that the `nix` program is written in C++.
A lot of the friction comes from the tension between NixOS's idiosyncratic design & its constraints, against the often 'dirty' way software is practically written.
For example, roughly, Nix's ideal software follows `./configure && make && make install`, where nix can then symlink the dependencies & maintain these in a read-only store. -- Whereas, say, Python wheels break this (because the precompiled binaries assume shared libraries are available globally).
When you run into friction with NixOS, you may need to understand things with a depth and breadth which aren't required with more common Linux distributions. (Including e.g. maybe needing to understand the rather large and obscure nixpkgs).
Do you have examples of times you've had to dig into the nix language like that, at least at a high level?
Just curious. I haven't run into anything like that myself.
Using nix for three months now, the main "pain" I run into is that I want app config files to remain writable by the app, but home-manager ethos is understandably against that -- and this is what you want on servers.
So I've had to vibe-code my own HM module for claude code, keepassxc, cursor, etc. and use activation to merge my nix settings into those files if they exist. That way when I rebuild, my nix config can assert a subset of the config I care about without locking the app out of writing to config -- and this makes more sense for personal desktop computing.
Though I put pain in quotes because it's still 10x better than what I was doing before, and an LLM can do it just fine.
The difference is, you can create proper abstraction modules that put everything together, from dependencies, to config files, firewall rules etc and have nice options for your abstraction.
No other system provides this in that sane way. I used countless configuration systems, from custom bash hacks, ansible, chef, puppet, salt - I have seen a lot.
> The difference is, you can create proper abstraction modules that put everything together, from dependencies, to config files, firewall rules etc and have nice options for your abstraction.
Yes, this is an understated benefit.
The declarative interface is nice.
That NixOS configs are modular allows you to create those abstractions. -- In the same sense terraform modules are "infrastructure as code", NixOS modules are "system configuration as code".
Yeah once you've used Nix[OS] and home-manager, it's hard to go back to apt or brew or the dreaded "sudo make install" without feeling like you need to have a shower afterwards. And I was a loyal Ubuntu person for like 15 years. It's especially true if you're a dev installing and uninstalling all day long.
With homebrew, you can have Brewfile that can serve as declarative source of truth.
I try to install all software via homebrew, mise (https://mise.jdx.dev/), and scoop (https://scoop.sh/), and setting up a new machine now takes me minutes. Meanwhile I don't need to deal with Nix language.
> With homebrew, you can have Brewfile that can serve as declarative source of truth.
for homebrew, while Nix configuration is for everything. I never used a Brewfile before, so looked at https://github.com/Lissy93/Brewfile/blob/master/Brewfile but it just looks like a list of packages. What about the configuration for those packages? Or your own custom patches for them? Runtime parameters? Environment variables?
There is so much more going on in the typical developer environment that it doesn't seem like (to me, an uneducated fool) Brewfile would be enough to actually serve as a declarative source of truth, except for Homebrew-specific things.
This has been my experience as well. Before I switched to NixOS I used ubuntu for 2 years. I never grokked the ways of apt and how or why it would "randomly" brick my system in some way. With NixOS this has never happened. `nix-shell` is dead simple, adding packages to environment is dead simple, never has it bricked my system. The hard part of NixOS is if you want to do advanced things with the actual nix language, and of course the horrible error messages.
In terms of all the linux systems I have used, NixOS seemed to least magical to me in terms of what is happening under the hood.
What?! Insane take. NixOS is where the most "magic" happens, over and under the hood. It brings it's own language!
Simple package based distros like Arch basically just extract archives. Very few packages trigger post-install steps which usually just (re)generate something like initrd.
Afaik, bricking Ubuntu is either due to user error (e.g. mixing incompatible package sources) or the devs released broken/buggy packages...
If you're not making changes to the bootloader it's essentially impossible to brick nixos: updates are fully atomic and every change can be rolled back by booting into an old generation.
This combined with the fact that the full source code for the system is contained within a single monorepo that I can checkout and grep through makes NixOS the easiest to understand and most transparent distro I have ever used.
> updates are fully atomic and every change can be rolled back by booting into an old generation
Well, the updates themselves yeah, but not what data they use. You cannot always rollback database upgrades for example, without also having to rollback the data source of the database. In most cases you're right though.
I'm saying this as someone who is a fan of NixOS and use it on all my servers because I tend to forget what I do if I just ssh in and fix stuff. Although I'm on Arch/CachyOS on all other hardware.
> Simple package based distros like Arch basically just extract archives. Very few packages trigger post-install steps which usually just (re)generate something like initrd.
Sure it's simple, just like 80% solving the problem is usually significantly simpler than solving it 100%. Nix (and its generation) is the only package manager that actually works.
Try installing kde and gnome, then uninstalling both and check how many packages remain.
Nix can do that with the whole world with nothing residing.
> Try installing kde and gnome, then uninstalling both and check how many packages remain.
Sounds like a package manager or package problem. Competent package managers (e.g. dnf) remove unneeded packages and all their owned files. Albeit, I think with apt you need to do a manual autoremove to remove orphaned packages.
Not suggesting that they're equivalently powerful compared to nix, but this specific thing shouldn't be a problem with traditional package managers.
I'm comparing it to my experience with Ubuntu where packages can have complex install dependency flows that can destroy your system at any time. They also pollute your entire system. This is simple in the same way assembly is a simple way to program. You can do anything and destroy anything. It's essentially spaghetti code but in OS form.
You can say it's "user error" or "packaging error", which is arguably true, but this "user error" and "packaging error" literally does not exist in NixOS. Installing a package is unable to touch anything outside of it's own designed nix store folder. That is why it's so much simpler to understand for me. I can check the nix store path and see what a package has. I can fearlessly install a package without worrying my system will break. I don't have to worry about residual dependencies remaining on my system.
> So do we want to do: We want to build one disk image that boots on x86_64, ARM AArch64, and RISC-V 64-bit. We limit ourselves here to UEFI platforms, which makes this pretty straight forward.
It kind of feels like this should have been the default across all architectures - a single disk that supports multiple instruction sets. 32 bit? 64 bit? Intel x86? Intel x86_64? RISC-V? Imagine how much simpler it would be for the end-user.
I still believe to this day that it has been a massive mistake to not ship ARM devices with some form of BIOS/UEFI chip to allow the system to boot without a bootable image.
The nix store can do hard linking for the hashsum files. The nix store is read-only so you don't get modifications in general - does not really matter for boot disks, but in general.
I have to say that that "surprisingly simple" thing is happening more and more for me on NixOS as well.
Recently a customer wanted me to use Fedora (I never visited the RPM side of the world before), and after truly the worst installer I've ever used, the actually system was nice. I do like Cockpit.
But then I needed to install an initrd that would me unlock the full disk encryption via SSH (it's a remote headless box). It took me half a day and a forum post to get it to work. I wrote a full page of notes for next time. Then the Firewall: I hit a bug (plus user error) which left me wrestling with a non-existent Tailscale interface for a while (it warns you for non-existing interfaces, but not with only a case mismatch, it then lets you do everything as if the interface exists), but after some hours I was done setting the zones, another page of notes and commands to enter to get to the desired state.
These configurations are both 1 or 2 lines in a NixOS config file. And that "work" is now done for all my NixOS servers.
You could argue that NixOS hides a lot of complexity, but so do Dracut and Firewalld of course. Nix is difficult, it's a high level abstraction. But it also just a bunch of key-values, and write-once, deploy everywhere.
>> You could argue that NixOS hides a lot of complexity
They both have the same complexity in that scenario. Underneath it's very comparable configuration for both but Nixos provides an easy abstraction for that specific case.
If you can stay on the happy path with nixos then it's pretty lovely. I've even adopted nix-darwin for my mac's too.
I'd still deploy Redhat/Fedora over nixos on anything revenue generating though. The problem is when you have to come off the happy path in nixos and now you're debugging some interestingly written c++ code that evaluates a language that has a derivation expressing what you wanted done. Contrast with the redhat situation, it's simpler but less convenient in the general case.
> The problem is when you have to come off the happy path in nixos and now you're debugging some interestingly written c++ code that evaluates a language that has a derivation expressing what you wanted done.
I agree that Nix or NixOS are risky tools.
But it's not a problem that the `nix` program is written in C++.
A lot of the friction comes from the tension between NixOS's idiosyncratic design & its constraints, against the often 'dirty' way software is practically written.
For example, roughly, Nix's ideal software follows `./configure && make && make install`, where nix can then symlink the dependencies & maintain these in a read-only store. -- Whereas, say, Python wheels break this (because the precompiled binaries assume shared libraries are available globally).
When you run into friction with NixOS, you may need to understand things with a depth and breadth which aren't required with more common Linux distributions. (Including e.g. maybe needing to understand the rather large and obscure nixpkgs).
Do you have examples of times you've had to dig into the nix language like that, at least at a high level?
Just curious. I haven't run into anything like that myself.
Using nix for three months now, the main "pain" I run into is that I want app config files to remain writable by the app, but home-manager ethos is understandably against that -- and this is what you want on servers.
So I've had to vibe-code my own HM module for claude code, keepassxc, cursor, etc. and use activation to merge my nix settings into those files if they exist. That way when I rebuild, my nix config can assert a subset of the config I care about without locking the app out of writing to config -- and this makes more sense for personal desktop computing.
Though I put pain in quotes because it's still 10x better than what I was doing before, and an LLM can do it just fine.
I do agree with that. But as a result I've gotten pretty good at whipping up podman containers with various bases to solve such issues.
The difference is, you can create proper abstraction modules that put everything together, from dependencies, to config files, firewall rules etc and have nice options for your abstraction.
No other system provides this in that sane way. I used countless configuration systems, from custom bash hacks, ansible, chef, puppet, salt - I have seen a lot.
Nix is just on another level. Never going back
> The difference is, you can create proper abstraction modules that put everything together, from dependencies, to config files, firewall rules etc and have nice options for your abstraction.
Yes, this is an understated benefit.
The declarative interface is nice.
That NixOS configs are modular allows you to create those abstractions. -- In the same sense terraform modules are "infrastructure as code", NixOS modules are "system configuration as code".
Yeah once you've used Nix[OS] and home-manager, it's hard to go back to apt or brew or the dreaded "sudo make install" without feeling like you need to have a shower afterwards. And I was a loyal Ubuntu person for like 15 years. It's especially true if you're a dev installing and uninstalling all day long.
With homebrew, you can have Brewfile that can serve as declarative source of truth.
I try to install all software via homebrew, mise (https://mise.jdx.dev/), and scoop (https://scoop.sh/), and setting up a new machine now takes me minutes. Meanwhile I don't need to deal with Nix language.
> With homebrew, you can have Brewfile that can serve as declarative source of truth.
for homebrew, while Nix configuration is for everything. I never used a Brewfile before, so looked at https://github.com/Lissy93/Brewfile/blob/master/Brewfile but it just looks like a list of packages. What about the configuration for those packages? Or your own custom patches for them? Runtime parameters? Environment variables?
There is so much more going on in the typical developer environment that it doesn't seem like (to me, an uneducated fool) Brewfile would be enough to actually serve as a declarative source of truth, except for Homebrew-specific things.
Versions? Compatibilities? Just pulling latest doesn't cut it.
> unlock the full disk encryption via SSH
I'd like to read more about this. Are you open to sharing your notes?
This has been my experience as well. Before I switched to NixOS I used ubuntu for 2 years. I never grokked the ways of apt and how or why it would "randomly" brick my system in some way. With NixOS this has never happened. `nix-shell` is dead simple, adding packages to environment is dead simple, never has it bricked my system. The hard part of NixOS is if you want to do advanced things with the actual nix language, and of course the horrible error messages.
In terms of all the linux systems I have used, NixOS seemed to least magical to me in terms of what is happening under the hood.
What?! Insane take. NixOS is where the most "magic" happens, over and under the hood. It brings it's own language!
Simple package based distros like Arch basically just extract archives. Very few packages trigger post-install steps which usually just (re)generate something like initrd.
Afaik, bricking Ubuntu is either due to user error (e.g. mixing incompatible package sources) or the devs released broken/buggy packages...
If you're not making changes to the bootloader it's essentially impossible to brick nixos: updates are fully atomic and every change can be rolled back by booting into an old generation.
This combined with the fact that the full source code for the system is contained within a single monorepo that I can checkout and grep through makes NixOS the easiest to understand and most transparent distro I have ever used.
> updates are fully atomic and every change can be rolled back by booting into an old generation
Well, the updates themselves yeah, but not what data they use. You cannot always rollback database upgrades for example, without also having to rollback the data source of the database. In most cases you're right though.
I'm saying this as someone who is a fan of NixOS and use it on all my servers because I tend to forget what I do if I just ssh in and fix stuff. Although I'm on Arch/CachyOS on all other hardware.
> Simple package based distros like Arch basically just extract archives. Very few packages trigger post-install steps which usually just (re)generate something like initrd.
Sure it's simple, just like 80% solving the problem is usually significantly simpler than solving it 100%. Nix (and its generation) is the only package manager that actually works.
Try installing kde and gnome, then uninstalling both and check how many packages remain.
Nix can do that with the whole world with nothing residing.
> Try installing kde and gnome, then uninstalling both and check how many packages remain.
Sounds like a package manager or package problem. Competent package managers (e.g. dnf) remove unneeded packages and all their owned files. Albeit, I think with apt you need to do a manual autoremove to remove orphaned packages.
Not suggesting that they're equivalently powerful compared to nix, but this specific thing shouldn't be a problem with traditional package managers.
I never used arch so I can't comment on that.
I'm comparing it to my experience with Ubuntu where packages can have complex install dependency flows that can destroy your system at any time. They also pollute your entire system. This is simple in the same way assembly is a simple way to program. You can do anything and destroy anything. It's essentially spaghetti code but in OS form.
You can say it's "user error" or "packaging error", which is arguably true, but this "user error" and "packaging error" literally does not exist in NixOS. Installing a package is unable to touch anything outside of it's own designed nix store folder. That is why it's so much simpler to understand for me. I can check the nix store path and see what a package has. I can fearlessly install a package without worrying my system will break. I don't have to worry about residual dependencies remaining on my system.
The nix language itself ain't it though, I agree.
This could possibly make a good base for a system recovery USB drive. One 18-headed hammer for all your needs.
I haven't looked deep into it, but my impression is that most system recovery images target just x86_64 and maybe 32-bit x86 if they're cheeky.
> So do we want to do: We want to build one disk image that boots on x86_64, ARM AArch64, and RISC-V 64-bit. We limit ourselves here to UEFI platforms, which makes this pretty straight forward.
It kind of feels like this should have been the default across all architectures - a single disk that supports multiple instruction sets. 32 bit? 64 bit? Intel x86? Intel x86_64? RISC-V? Imagine how much simpler it would be for the end-user.
I still believe to this day that it has been a massive mistake to not ship ARM devices with some form of BIOS/UEFI chip to allow the system to boot without a bootable image.
Something about this rubs me the wrong way. You can get completely different behaviour depending on which architecture you boot on.
Why? They're all built from the same source code.
> I thought this would bring some space savings, because files that are not binary code should be largely the same.
Those ternary blobs tend to be cross-platform, I hear.
The nix store can do hard linking for the hashsum files. The nix store is read-only so you don't get modifications in general - does not really matter for boot disks, but in general.