diff --git a/README.md b/README.md index 8aa05d7..5c622ea 100644 --- a/README.md +++ b/README.md @@ -13,25 +13,52 @@
-# Self Host Blocks +# SelfHostBlocks -*Modular server management based on NixOS modules and focused on best practices.* +*SelfHostBlocks is a NixOS based server management for self-hosting +using building blocks and promoting best practices.* +It is obvious by now that +a deep dependency on proprietary service providers - "the cloud" - is a significant liability. +One aspect often talked about is privacy which is inherently not guaranteed +when using a proprietary service and is a valid concern. +A more punishing issue is having your account closed or locked +without prior warning. +When that happens, you get an instantaneous sinking feeling in your stomach +at the realization you lost access to your data, possibly without recourse. -SHB (Self Host Blocks) is yet another server management tool -that is not like the other server management tools. +Self-hosting is the only alternative that alleviate those concerns +but it requires a lot of technical skills and time. +SelfHostBlocks' and its sibling project [Skarabox][]' goal +is to lower the bar to self-hosting. + +SelfHostBlocks is different from other server management projects +because it's main focus is ease of long term maintenance +before ease of installation. +To achieve this, it provides building blocks to setup services. +Some services are already provided out of the box +and adding custom ones is done easily thanks to those blocks. +These building blocks fit together nicely thanks to [contracts](#contracts). ## TOC - [Usage](#usage) -- [Server Management](#server-management) + - [Existing Installation](#existing-installation) + - [Installation From Scratch](#installation-from-scratch) + - [Full Example](#full-example) +- [Features](#features) + - [Services](#services) + - [Blocks](#blocks) - [Unified Interfaces](#unified-interfaces) - - [Incremental Adoption](#incremental-adoption) - - [More Benefits of SHB](#more-benefits-of-shb) + - [Contracts](#contracts) + - [Interfacing With Other OSes](#interfacing-with-other-oses) + - [Sitting on the Shoulders of a Giant](#sitting-on-the-shoulders-of-a-giant) + - [Automatic Updates](#automatic-updates) + - [Demos](#demos) - [Roadmap](#roadmap) -- [Demos](#demos) - [Community](#community) +- [Funding](#funding) - [License](#license) @@ -41,34 +68,21 @@ that is not like the other server management tools. > production server, this is really just a one person effort for now and there are most certainly > bugs that I didn't discover yet. -### Flake Module +### Existing Installation -Self Host Blocks is available as a flake. -Also on [flakehub](https://flakehub.com/flake/ibizaman/selfhostblocks?view=usage). -To use it in your existing project, add the following flake input: - -```nix -inputs.selfhostblocks.url = "github:ibizaman/selfhostblocks"; -``` - -Then, pin it to a release/tag with the following snippet. -Updating Self Host Blocks to a new version can be done the same way. - -```nix -nix flake lock --override-input selfhostblocks github:ibizaman/selfhostblocks/v0.2.7 -``` - -To get started using Self Host Blocks, +To get started using SelfHostBlocks, follow [the usage section](https://shb.skarabox.com/usage.html) of the manual. -It goes over how to deploy with [Colmena][], [nixos-rebuild][] +It goes over how to deploy with [Colmena][], [nixos-rebuild][] and [deploy-rs][] and also goes over secrets management with [SOPS][]. -[Colmena]: https://colmena.cli.rs/ -[nixos-rebuild]: https://nixos.org/manual/nixos/stable/#sec-changing-config -[SOPS]: https://github.com/Mic92/sops-nix +[Colmena]: https://shb.skarabox.com/usage.html#usage-example-colmena +[nixos-rebuild]: https://shb.skarabox.com/usage.html#usage-example-nixosrebuild +[deploy-rs]: https://shb.skarabox.com/usage.html#usage-example-deployrs +[SOPS]: https://shb.skarabox.com/usage.html#usage-secrets Then, to actually configure services, you can choose which one interests you in [the services section](https://shb.skarabox.com/services.html) of the manual. +Not all services have a corresponding manual page yet. Head over to the [matrix channel](https://matrix.to/#/#selfhostblocks:matrix.org) for any remaining question, or just to say hi :) @@ -78,7 +92,9 @@ for any remaining question, or just to say hi :) I do recommend for this my sibling project [Skarabox][] which bootstraps a new server and sets up a few tools: -- Creating a bootable ISO, installable on an USB key. +- Create a bootable ISO, installable on an USB key. +- Handles one or two (in raid 1) SSDs for root partition. +- Handles two (in raid 1) or more hard drives for data partition. - [nixos-anywhere](https://github.com/nix-community/nixos-anywhere) to install NixOS headlessly. - [disko](https://github.com/nix-community/disko) to format the drives using native ZFS encryption with remote unlocking through ssh. - [sops-nix](https://github.com/Mic92/sops-nix) to handle secrets. @@ -92,12 +108,10 @@ See [full example][] in the manual. [full example]: https://shb.skarabox.com/usage.html#usage-complete-example -## Server Management +## Features -Self Host Blocks provides a standardized configuration for [some services](https://shb.skarabox.com/services.html) provided by nixpkgs. -The goal is to help spread adoption of self-hosting by providing an opinionated configuration with best practices by default. +SelfHostBlocks provides building blocks that take care of common self-hosting needs: -Self Host Blocks takes care of common self-hosting needs: - Backup for all services. - Automatic creation of ZFS datasets per service. - LDAP and SSO integration for most services. @@ -110,6 +124,8 @@ This translates into a test suite comprised of automated NixOS VM tests which includes playwright tests to verify some important workflow like logging in. +Also, the stack fits together nicely thanks to [contracts](#contracts). + ### Services [Provided services](https://shb.skarabox.com/services.html) are: @@ -140,8 +156,7 @@ is to go to the [All Options][] section of the manual. ### Blocks -To provided out of the box common functionality, -the services above use the following [common blocks][]: +The services above rely on the following [common blocks][]: [common blocks]: https://shb.skarabox.com/blocks.html @@ -159,7 +174,8 @@ the services above use the following [common blocks][]: - VPN - ZFS -Those blocks can be used outside of Self Host Blocks too. +Those blocks can be used with services +not provided by SelfHostBlocks. Some blocks do not have an entry yet in the manual. To know options for those, the only way for now @@ -167,10 +183,11 @@ is to go to the [All Options][] section of the manual. ### Unified Interfaces -SHB's first goal is to provide unified [building blocks](#available-blocks) -and by extension configuration interface, for self-hosting. +Thanks to the blocks, +SelfHostBlocks provides an unified configuration interface +for the services it provides. -Compare the configuration for Nextcloud and Forgejo in Self Host Blocks. +Compare the configuration for Nextcloud and Forgejo. The following snippets focus on similitudes and assume the relevant blocks - like secrets - are configured off-screen. It also does not show specific options for each service. These are still complete snippets that configure HTTPS, @@ -228,67 +245,88 @@ shb.forgejo = { ``` As you can see, they are pretty similar! +This makes setting up a new service pretty easy and intuitive. -SHB provides an ever growing list of [services](#provided-services) +SelfHostBlocks provides an ever growing list of [services](#provided-services) that are configured in the same way. -### Incremental Adoption +### Contracts -SHB's second goal is to facilitate testing NixOS -and slowly switching an existing installation to NixOS. - -To achieve this, SHB pioneers [contracts][] -which allows you, the final user, to be more in control of which piece go where. +To make building blocks that fit nicely together, +SHB pioneers [contracts][] which allows you, the final user, +to be more in control of which piece goes where. This lets you choose, for example, any reverse proxy you want or any database you want, without requiring work from maintainers of the services you want to self host. -(See [manual][contracts] for a complete explanation) -Two videos exist of me presenting the topic, +A [pre-RFC][] exists to upstream this concept into `nixpkgs`. +The [manual][contracts] also provides an explanation of the why and how of contracts. + +Also, two videos exist of me presenting the topic, the first at [NixCon North America in spring of 2024][NixConNA2024] and the second at [NixCon in Berlin in fall of 2024][NixConBerlin2024]. [contracts]: https://shb.skarabox.com/contracts.html +[pre-RFC]: https://discourse.nixos.org/t/pre-rfc-decouple-services-using-structured-typing/58257 [NixConNA2024]: https://www.youtube.com/watch?v=lw7PgphB9qM [NixConBerlin2024]: https://www.youtube.com/watch?v=CP0hR6w1csc -### More Benefits of SHB +### Interfacing With Other OSes -By using Self Host Blocks, you get all the benefits of NixOS +Thanks to [contracts](#contracts), one can interface NixOS +with systems on other OSes. +The [pre-RFC][] explains how that works. + +### Sitting on the Shoulders of a Giant + +By using SelfHostBlocks, you get all the benefits of NixOS which are, for self hosted applications specifically: - declarative configuration; - atomic configuration rollbacks; - real programming language to define configurations; -- user-defined abstractions (create your own functions or NixOS modules on top of SHB!); +- create your own higher level abstractions on top of SHB; - integration with the rest of nixpkgs; - much fewer "works on my machine" type of issues. -In no particular order, here are some aspects of SHB which I find interesting and differentiates it -from other server management projects: +### Automatic Updates -- SHB intends to be a library, not a framework. You can either go all in and use SHB provided - services directly or use just one block in your existing infrastructure. -- SHB introduces [contracts](https://shb.skarabox.com/contracts.html) to allow you to swap - implementation for each self-hosting need. For example, you should be able to use the reverse - proxy you want without modifying any services depending on it. -- SHB contracts also allows you to use your own custom implementation instead of the provided one, - as long as it follows the contract and passes the tests. -- SHB provides at least one implementation for each contract like backups, SSL certificates, reverse - proxy, VPN, etc. Those are called blocks here and are documented in [the - manual](https://shb.skarabox.com/blocks.html). -- SHB provides several services out of the box fully using the blocks provided. Those can also be - found in [the manual](https://shb.skarabox.com/services.html). -- SHB follows nixpkgs unstable branch closely. There is a GitHub action running daily that updates - the `nixpkgs` input in the root `flakes.nix`, runs the tests and merges a PR with the new input if - the tests pass. +SelfHostBlocks follows nixpkgs unstable branch closely. +There is a GitHub action running every couple of days that updates +the `nixpkgs` input in the root `flakes.nix`, +runs the tests and merges the PR automatically +if the tests pass. + +A release is then made every few commits, +whenever deemed sensible. +On your side, to update I recommend pinning to a release +with the following command, +replacing the RELEASE with the one you want: + +```bash +RELEASE=0.2.4 +nix flake update \ + --override-input selfhostblocks github:ibizaman/selfhostblocks/$RELEASE \ + selfhostblock +``` + +### Demos + +Demos that start and deploy a service +on a Virtual Machine on your computer are located +under the [demo](./demo/) folder. + +These show the onboarding experience you would get +if you deployed one of the services on your own server. ## Roadmap -Currently, the Nextcloud, Vaultwarden services and the SSL and backup blocks are the most advanced and most documented. +Currently, the Nextcloud and Vaultwarden services +and the SSL and backup blocks +are the most advanced and most documented. -Documenting all services and blocks will be done as I make all blocks and services use the -contracts. +Documenting all services and blocks will be done +as I make all blocks and services use the contracts. Upstreaming changes is also on the roadmap. @@ -301,14 +339,12 @@ Feel free to add more or to contribute! All blocks and services have NixOS tests. Also, I am personally using all the blocks and services in this project, so they do work to some extent. -## Demos - -Demos that start and deploy a service on a Virtual Machine on your computer are located under the -[demo](./demo/) folder. These show the onboarding experience you would get if you deployed one of -the services on your own server. - ## Community +This project has been the main focus +of my (non work) life for the past 3 year now +and I intend to continue working on this for a long time. + All issues and PRs are welcome. For PRs, if they are substantial changes, please open an issue to discuss the details first. More details in [the contributing section](https://shb.skarabox.com/contributing.html) of the manual. @@ -319,9 +355,24 @@ One aspect that's close to my heart is I intent to make SHB the lightest layer o possible. I want to upstream as much as possible. I will still take some time to experiment here but when I'm satisfied with how things look, I'll upstream changes. +## Funding + +I was lucky to [obtain a grant][nlnet] from NlNet which is an European fund, +under [NGI Zero Core][NGI0], +to work on this project. +This also funds the contracts RFC. + +Go apply for a grant too! + +[nlnet]: https://nlnet.nl/project/SelfHostBlocks +[NGI0]: https://nlnet.nl/core/ + +

+NlNet logo +NGI Zero Core logo +

+ ## License I'm following the [Nextcloud](https://github.com/nextcloud/server) license which is AGPLv3. -See [this article][why agplv3] from the FSF that explains what this license adds to the GPL one. - -[why agplv3]: (https://www.fsf.org/bulletin/2021/fall/the-fundamentals-of-the-agplv3) +See [this article](https://www.fsf.org/bulletin/2021/fall/the-fundamentals-of-the-agplv3) from the FSF that explains what this license adds to the GPL one. diff --git a/docs/usage.md b/docs/usage.md index b64e4a7..15b7873 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -63,7 +63,7 @@ guarantee Self Host Block won't use a non-existing option from `nixpkgs`. To pin Self Host Blocks to a release/tag, run the following snippet: ```nix -nix flake lock --override-input selfhostblocks github:ibizaman/selfhostblocks/@VERSION@ +nix flake update --override-input selfhostblocks github:ibizaman/selfhostblocks/@VERSION@ selfhostblock ``` Updating Self Host Blocks to a new version can be done the same way. @@ -80,7 +80,9 @@ The setup is explained in [this blog post][2]. ## Example Deployment with Nixos-Rebuild {#usage-example-nixosrebuild} -The following snippets show how to deploy Self Host Blocks using the standard deployment system `nixos-rebuild`. +The following snippets show how to deploy Self Host Blocks using the standard deployment system [nixos-rebuild][]. + +[nixos-rebuild]:https://nixos.org/manual/nixos/stable/#sec-changing-config ```nix {