danvics/selfhostblocks
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

use secrets contract for restic (#349)

Browse source
This commit is contained in:
Pierre Penninckx 2024-11-17 22:12:14 +01:00 committed by GitHub
parent 503fcef74b
commit a04dca9973
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
17 changed files with 391 additions and 177 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

170
docs/contracts.md
View file

@ -4,12 +4,76 @@ A contract decouples modules that use a functionality from modules that provide
intuition for contracts is they are generally related to accessing a shared resource.
A few examples of contracts are generating SSL certificates, creating a user or knowing which files
and folders to backup. Indeed, when generating certificates, the service using those do not care how
they were created. They just need to know where the certificate files are located.
and folders to backup.
Indeed, when generating certificates, the service using those do not care how they were created.
They just need to know where the certificate files are located.
In practice, a contract is a set of options that any user of a contract expects to exist. Also, the
values of these options dictate the behavior of the implementation. This is enforced with NixOS VM
tests.
A contract is made between a requester module and a provider module.
For example, a backup contract can be made between the Nextcloud service and the Restic service.
The former is the requester - the one wanted to be backed up -
and the latter is the provider of the contract - the one backing up files.
## Schema {#contracts-schema}
In practice, a contract is an attrset of options with a defined behavior.
Currently, the schema for a requester is:
```nix
let
inherit (lib) mkOption;
inherit (lib.types) submodule;
in
config.${requester}.${contractname} = submodule {
request = mkOption {
type = contracts.${contractname}.request;
default = {
# Values set by the requester
};
};
result = mkOption {
type = contracts.${contractname}.result;
};
};
```
For a provider, it is:
```nix
let
inherit (lib) mkOption;
inherit (lib.types) anything submodule;
in
config.${provider}.${contractname} = submodule ({ options, ... }: {
request = mkOption {
type = contracts.${contractname}.request;
};
result = mkOption {
type = contracts.${contractname}.result;
default = {
# Values set by the provider
# Can depend on values set by the requester through the `options` variable.
};
};
settings = mkOption {
type = anything;
};
});
```
## Contract Tests {#contracts-test}
To make sure all providers module of a contract have the same behavior,
generic NixOS VM tests exist per contract.
They are generic because they work on any module,
as long as the module implements the contract of course.
For example, the [generic test][generic] for backup contract is instantiated for Restic [here][restic test].
[generic]: @REPO@/modules/contracts/backup/test.nix
[restic test]: @REPO@/test/contracts/backup.nix
## Videos {#contracts-videos}
@ -20,6 +84,59 @@ and the second at [NixCon in Berlin in fall of 2024][NixConBerlin2024].
[NixConNA2024]: https://www.youtube.com/watch?v=lw7PgphB9qM
[NixConBerlin2024]: https://www.youtube.com/watch?v=CP0hR6w1csc
## Why do we need this new concept? {#contracts-why}
Currently in nixpkgs, every module needing access to a shared resource must implement the logic
needed to setup that resource themselves. Similarly, if the module is mature enough to let the user
select a particular implementation, the code lives inside that module.
![](./assets/contracts_before.png "A module composed of a core logic and a lot of peripheral logic.")
This has a few disadvantages:
- This leads to a lot of **duplicated code**. If a module wants to support a new implementation of a
contract, the maintainers of that module must write code to make that happen.
- This also leads to **tight coupling**. The code written by the maintainers cannot be reused in
other modules, apart from copy pasting.
- There is also a **lack of separation of concerns**. The maintainers of a service must be experts
in all implementations they let the users choose from.
- Finally, this is **not extensible**. If you, the user of the module, want to use another
implementation that is not supported, you are out of luck. You can always dive into the module's
code and extend it, but that is not an optimal experience.
We do believe that the decoupling contracts provides helps alleviate all the issues outlined above
which makes it an essential step towards more adoption of Nix, if only in the self hosting scene.
![](./assets/contracts_after.png "A module containing only logic using peripheral logic through contracts.")
Indeed, contracts allow:
- **Reuse of code**.
Since the implementation of a contract lives outside of modules using it,
using that implementation elsewhere is trivial.
- **Loose coupling**.
Modules that use a contract do not care how they are implemented,
as long as the implementation follows the behavior outlined by the contract.
- Full **separation of concerns** (see diagram below).
Now, each party's concern is separated with a clear boundary.
The maintainer of a module using a contract can be different from the maintainers
of the implementation, allowing them to be experts in their own respective fields.
But more importantly, the contracts themselves can be created and maintained by the community.
- Full **extensibility**.
The final user themselves can choose an implementation,
even new custom implementations not available in nixpkgs, without changing existing code.
- **Incremental adoption**.
Contracts can help bridge a NixOS system with any non-NixOS one.
For that, one can hardcode a requester or provider module to match
how the non-NixOS system is configured.
The responsability falls of course on the user to make sure both system agree on the configuration.
- Last but not least, **Testability**.
Thanks to NixOS VM test, we can even go one step further
by ensuring each implementation of a contract, even custom ones,
provides required options and behaves as the contract requires.
![](./assets/contracts_separationofconcerns.png "Separation of concerns thanks to contracts.")
## Provided contracts {#contracts-provided}
Self Host Blocks is a proving ground of contracts. This repository adds a layer on top of services
@ -53,49 +170,6 @@ modules/contracts/databasebackup/docs/default.md
modules/contracts/secret/docs/default.md
```
## Why do we need this new concept? {#contracts-why}
Currently in nixpkgs, every module needing access to a shared resource must implement the logic
needed to setup that resource themselves. Similarly, if the module is mature enough to let the user
select a particular implementation, the code lives inside that module.
![](./assets/contracts_before.png "A module composed of a core logic and a lot of peripheral logic.")
This has a few disadvantages:
- This leads to a lot of **duplicated code**. If a module wants to support a new implementation of a
contract, the maintainers of that module must write code to make that happen.
- This also leads to **tight coupling**. The code written by the maintainers cannot be reused in
other modules, apart from copy pasting.
- There is also a **lack of separation of concerns**. The maintainers of a service must be experts
in all implementations they let the users choose from.
- Finally, this is **not extensible**. If you, the user of the module, want to use another
implementation that is not supported, you are out of luck. You can always dive into the module's
code and extend it, but that is not an optimal experience.
We do believe that the decoupling contracts provides helps alleviate all the issues outlined above
which makes it an essential step towards more adoption of Nix, if only in the self hosting scene.
![](./assets/contracts_after.png "A module containing only logic using peripheral logic through contracts.")
Indeed, contracts allow:
- **Reuse of code**. Since the implementation of a contract lives outside of modules using it, using
that implementation elsewhere is trivial.
- **Loose coupling**. Modules that use a contract do not care how they are implemented, as long as
the implementation follows the behavior outlined by the contract.
- Full **separation of concerns** (see diagram below). Now, each party's concern is separated with a
clear boundary. The maintainer of a module using a contract can be different from the maintainers
of the implementation, allowing them to be experts in their own respective fields. But more
importantly, the contracts themselves can be created and maintained by the community.
- Full **extensibility**. The final user themselves can choose an implementation, even new custom
implementations not available in nixpkgs, without changing existing code.
- Last but not least, **Testability**. Thanks to NixOS VM test, we can even go one step further by
ensuring each implementation of a contract, even custom ones, provides required options and
behaves as the contract requires.
![](./assets/contracts_separationofconcerns.png "Separation of concerns thanks to contracts.")
## Are there contracts in nixpkgs already? {#contracts-nixpkgs}
Actually not quite, but close. There are some ubiquitous options in nixpkgs. Those I found are:

19
docs/services.md
View file

@ -12,27 +12,28 @@ information is provided in the respective manual sections.
| Service | Backup | Reverse Proxy | SSO | LDAP | Monitoring | Profiling |
|-----------------------|--------|---------------|-----|-------|------------|-----------|
| [Nextcloud Server][1] | P (1) | Y | Y | Y | Y | P (2) |
| [Vaultwarden][2] | P (1) | Y | Y | Y | N | N |
| [Forgejo][3] | Y | Y | Y | Y | N | N |
| [Nextcloud Server][1] | Y (1) | Y | Y | Y | Y (2) | P (3) |
| [Vaultwarden][2] | Y (1) | Y | Y | Y | Y (2) | N |
| [Forgejo][3] | Y (1) | Y | Y | Y | Y (2) | N |
Legend: **N**: no but WIP; **P**: partial; **Y**: yes
1. Does not backup the database yet.
2. Works but the traces are not exported to Grafana yet.
1. Database and data files are backed up separately.
2. Dashboard is common to all services.
3. Works but the traces are not exported to Grafana yet.
[1]: services-nextcloud.html
[2]: services-vaultwarden.html
[3]: services-forgejo.html
```{=include=} chapters html:into-file=//services-vaultwarden.html
modules/services/vaultwarden/docs/default.md
```
```{=include=} chapters html:into-file=//services-nextcloud.html
modules/services/nextcloud-server/docs/default.md
```
```{=include=} chapters html:into-file=//services-vaultwarden.html
modules/services/vaultwarden/docs/default.md
```
```{=include=} chapters html:into-file=//services-forgejo.html
modules/services/forgejo/docs/default.md
```

4
modules/blocks/postgresql.nix
View file

@ -67,12 +67,12 @@ in
```
'';
type = contracts.databasebackup.requestType;
type = contracts.databasebackup.request;
default = {
user = "postgres";
backupFile = "postgres.sql";
backupName = "postgres.sql";
backupCmd = ''
${pkgs.postgresql}/bin/pg_dumpall | ${pkgs.gzip}/bin/gzip --rsyncable

66
modules/blocks/restic.nix
View file

@ -10,7 +10,7 @@ let
inherit (lib) generators hasPrefix mkIf nameValuePair optionalAttrs removePrefix;
inherit (lib.types) attrsOf enum int ints listOf oneOf nonEmptyListOf nonEmptyStr nullOr path str submodule;
commonOptions = {
commonOptions = { name, options, prefix, ... }: {
enable = mkEnableOption ''
this backup intance.
@ -18,9 +18,13 @@ let
but still provides the helper tool to restore snapshots
'';
passphraseFile = mkOption {
description = "Encryption key for the backups.";
type = path;
passphrase = contracts.secret.mkOption {
description = "Encryption key for the backup repository.";
mode = "0400";
owner = options.request.value.user;
ownerText = "[shb.restic.${prefix}.<name>.request.user](#blocks-restic-options-shb.restic.${prefix}._name_.request.user)";
restartUnits = [ (fullName name options.settings.value.repository) ];
restartUnitsText = "[ [shb.restic.${prefix}.<name>.settings.repository](#blocks-restic-options-shb.restic.${prefix}._name_.settings.repository) ]";
};
repository = mkOption {
@ -101,7 +105,7 @@ in
{
options.shb.restic = {
instances = mkOption {
description = "Each instance is backing up some directories to one repository.";
description = "Files to backup following the [backup contract](./contracts-backup.html).";
default = {};
type = attrsOf (submodule ({ name, options, ... }: {
options = {
@ -121,7 +125,7 @@ in
'';
type = submodule {
options = commonOptions;
options = commonOptions { inherit name options; prefix = "instances"; };
};
};
@ -131,24 +135,27 @@ in
Contains the output of the Restic provider.
'';
type = lib.types.anything; # contracts.databasebackup.result;
default = {
restoreScript = fullName name options.settings.value.repository;
backupService = "${fullName name options.settings.value.repository}.service";
};
defaultText = literalExpression ''
{
restoreScript = "${fullName "<name>" { path = "path/to/repository"; }}";
backupService = "${fullName "<name>" { path = "path/to/repository"; }}.service";
}
'';
defaultText = {
restoreScriptText = "${fullName "<name>" { path = "path/to/repository"; }}";
backupServiceText = "${fullName "<name>" { path = "path/to/repository"; }}.service";
};
type = contracts.backup.result {
restoreScript = fullName name options.settings.value.repository;
backupService = "${fullName name options.settings.value.repository}.service";
restoreScriptText = "${fullName "<name>" { path = "path/to/repository"; }}";
backupServiceText = "${fullName "<name>" { path = "path/to/repository"; }}.service";
};
};
};
}));
};
databases = mkOption {
description = "Databases to backup following the database backup contract.";
description = "Databases to backup following the [database backup contract](./contracts-databasebackup.html).";
default = {};
type = attrsOf (submodule ({ name, options, ... }: {
options = {
@ -159,7 +166,7 @@ in
Accepts values from a requester.
'';
type = contracts.databasebackup.requestType;
type = contracts.databasebackup.request;
};
settings = mkOption {
@ -168,7 +175,7 @@ in
'';
type = submodule {
options = commonOptions;
options = commonOptions { inherit name options; prefix = "databases"; };
};
};
@ -178,17 +185,20 @@ in
Contains the output of the Restic provider.
'';
type = contracts.databasebackup.resultType;
default = {
restoreScript = fullName name options.settings.value.repository;
backupService = "${fullName name options.settings.value.repository}.service";
};
defaultText = literalExpression ''
{
restoreScript = "${fullName "<name>" { path = "path/to/repository"; }}";
backupService = "${fullName "<name>" { path = "path/to/repository"; }}.service";
}
'';
defaultText = {
restoreScriptText = "${fullName "<name>" { path = "path/to/repository"; }}";
backupServiceText = "${fullName "<name>" { path = "path/to/repository"; }}.service";
};
type = contracts.databasebackup.result {
restoreScript = fullName name options.settings.value.repository;
backupService = "${fullName name options.settings.value.repository}.service";
restoreScriptText = "${fullName "<name>" { path = "path/to/repository"; }}";
backupServiceText = "${fullName "<name>" { path = "path/to/repository"; }}.service";
};
};
};
}));
@ -249,7 +259,7 @@ in
paths = instance.request.sourceDirectories;
passwordFile = toString instance.settings.passphraseFile;
passwordFile = toString instance.settings.passphrase.result.path;
initialize = true;
@ -288,7 +298,7 @@ in
dynamicFilesFrom = "echo";
passwordFile = toString instance.settings.passphraseFile;
passwordFile = toString instance.settings.passphrase.result.path;
initialize = true;
@ -310,7 +320,7 @@ in
cmd = pkgs.writeShellScriptBin "dump.sh" instance.request.backupCmd;
in
[
"--stdin-filename ${instance.request.backupFile} --stdin-from-command -- ${cmd}/bin/dump.sh"
"--stdin-filename ${instance.request.backupName} --stdin-from-command -- ${cmd}/bin/dump.sh"
]);
};
};
@ -371,7 +381,7 @@ in
serviceConfig.Type = "oneshot";
script = (shblib.replaceSecrets {
userConfig = instance.settings.repository.secrets // {
RESTIC_PASSWORD_FILE = instance.settings.passphraseFile;
RESTIC_PASSWORD_FILE = toString instance.settings.passphrase.result.path;
RESTIC_REPOSITORY = instance.settings.repository.path;
};
resultPath = "/run/secrets_restic_env/${fullName name instance.settings.repository}";
@ -414,7 +424,7 @@ in
sudo --preserve-env -u ${instance.request.user} ${pkgs.restic}/bin/restic $@
else
shift
sudo --preserve-env -u ${instance.request.user} sh -c "${pkgs.restic}/bin/restic dump $@ ${instance.request.backupFile} | ${instance.request.restoreCmd}"
sudo --preserve-env -u ${instance.request.user} sh -c "${pkgs.restic}/bin/restic dump $@ ${instance.request.backupName} | ${instance.request.restoreCmd}"
fi
'';
in

108
modules/blocks/restic/docs/default.md
View file

@ -2,7 +2,7 @@
Defined in [`/modules/blocks/restic.nix`](@REPO@/modules/blocks/restic.nix).
This block sets up a backup job using [Restic][restic].
This block sets up a backup job using [Restic][].
[restic]: https://restic.net/
@ -12,18 +12,38 @@ Specific integration tests are defined in [`/test/blocks/restic.nix`](@REPO@/tes
## Provider Contracts {#blocks-restic-contract-provider}
This block implements the [backup](contracts-backup.html) and [database backup](contracts-databasebackup.html) contracts.
This block provides:
Contract integration tests are defined in [`/test/contracts/backup.nix`](@REPO@/test/contracts/backup.nix).
- [backup contract](contracts-backup.html) under the [`shb.restic.instances`][instances] option.
It is tested with [contract tests][backup contract tests].
- [database backup contract](contracts-databasebackup.html) under the [`shb.restic.databases`][databases] option.
It is tested with [contract tests][database backup contract tests].
### One folder backed up to mounted hard drives {#blocks-restic-contract-provider-one}
[instances]: #blocks-restic-options-shb.restic.instances
[databases]: #blocks-restic-options-shb.restic.databases
[backup contract tests]: @REPO@/test/contracts/backup.nix
[database backup contract tests]: @REPO@/test/contracts/databasebackup.nix
As requested by those two contracts, when setting up a backup with Restic,
a backup Systemd service and restore script are provided.
The restore script has all the secrets needed to access the repo,
the only requirement to run it is to be able to `sudo` in the expected user.
## Usage {#blocks-restic-usage}
The following examples assume usage of SOPS to provide secrets
although any blocks providing the [secrets contract][] works too.
The [secrets setup section](usage.html#usage-secrets) explains
how to setup SOPS.
### One folder backed up manually {#blocks-restic-usage-provider-manual}
The following snippet shows how to configure
the backup of 1 folder to 1 repository.
We assume that the folder is used by the `myservice` service and is owned by a user of the same name.
We assume that the folder `/var/lib/myfolder` of the service `myservice` must be backed up.
```nix
shb.restic.instances.myservice = {
shb.restic.instances."myservice" = {
request = {
user = "myservice";
@ -56,7 +76,41 @@ shb.restic.instances.myservice = {
};
```
### One folder backed up to S3 {#blocks-restic-contract-provider-remote}
### One folder backed up with contract {#blocks-restic-usage-provider-contract}
With the same example as before but assuming the `myservice` service
has a `myservice.backup` option that is a requester for the backup contract,
the snippet above becomes:
```nix
shb.restic.instances."myservice" = {
request = config.myservice.backup;
settings = {
enable = true;
passphraseFile = "<path/to/passphrase>";
repository = {
path = "/srv/backups/myservice";
timerConfig = {
OnCalendar = "00:00:00";
RandomizedDelaySec = "3h";
};
};
retention = {
keep_within = "1d";
keep_hourly = 24;
keep_daily = 7;
keep_weekly = 4;
keep_monthly = 6;
};
};
};
```
### One folder backed up to S3 {#blocks-restic-usage-provider-remote}
Here we will only highlight the differences with the previous configuration.
@ -81,46 +135,6 @@ This assumes you have access to such a remote S3 store, for example by using [Ba
}
```
## Secrets {#blocks-restic-secrets}
To be secure, the secrets should deployed out of band, otherwise they will be world-readable in the nix store.
To achieve that, I recommend [sops](usage.html#usage-secrets) although other methods work great too.
The code to backup to Backblaze with secrets stored in Sops would look like so:
```nix
shb.restic.instances.myfolder.passphraseFile = config.sops.secrets."myservice/backup/passphrase".path;
shb.restic.instances.myfolder.repository = {
path = "s3:s3.us-west-000.backblazeb2.com/<mybucket>";
secrets = {
AWS_ACCESS_KEY_ID.source = config.sops.secrets."backup/b2/access_key_id".path;
AWS_SECRET_ACCESS_KEY.source = config.sops.secrets."backup/b2/secret_access_key".path;
};
};
sops.secrets."myservice/backup/passphrase" = {
sopsFile = ./secrets.yaml;
mode = "0400";
owner = "myservice";
group = "myservice";
};
sops.secrets."backup/b2/access_key_id" = {
sopsFile = ./secrets.yaml;
mode = "0400";
owner = "myservice";
group = "myservice";
};
sops.secrets."backup/b2/secret_access_key" = {
sopsFile = ./secrets.yaml;
mode = "0400";
owner = "myservice";
group = "myservice";
};
```
Pay attention that the owner must be the `myservice` user, the one owning the files to be backed up.
A `secrets` contract is in progress that will allow one to not care about such details.
## Multiple directories to multiple destinations {#blocks-restic-usage-multiple}
The following snippet shows how to configure backup of any number of folders to 3 repositories,

48
modules/contracts/backup.nix
View file

@ -7,7 +7,11 @@ in
request = submodule {
options = {
user = mkOption {
description = "Unix user doing the backups.";
description = ''
Unix user doing the backups.
Most of the time, this should be the user owning the files.
'';
type = str;
};
@ -17,7 +21,7 @@ in
};
excludePatterns = mkOption {
description = "Patterns to exclude.";
description = "File patterns to exclude.";
type = listOf str;
default = [];
};
@ -28,13 +32,13 @@ in
type = submodule {
options = {
before_backup = mkOption {
description = "Hooks to run before backup";
description = "Hooks to run before backup.";
type = listOf str;
default = [];
};
after_backup = mkOption {
description = "Hooks to run after backup";
description = "Hooks to run after backup.";
type = listOf str;
default = [];
};
@ -44,16 +48,46 @@ in
};
};
result = submodule {
result = {
restoreScript,
restoreScriptText ? null,
backupService,
backupServiceText ? null,
}: submodule {
options = {
restoreScript = mkOption {
description = "Name of script that can restore the database.";
description = ''
Name of script that can restore the database.
One can then list snapshots with:
```bash
$ ${if restoreScriptText != null then restoreScriptText else restoreScript} snapshots
```
And restore the database with:
```bash
$ ${if restoreScriptText != null then restoreScriptText else restoreScript} restore latest
```
'';
type = str;
default = restoreScript;
defaultText = restoreScriptText;
};
backupService = mkOption {
description = "Name of service backing up the database.";
description = ''
Name of service backing up the database.
This script can be ran manually to backup the database:
```bash
$ systemctl start ${if backupServiceText != null then backupServiceText else backupService}
```
'';
type = str;
default = backupService;
defaultText = backupServiceText;
};
};
};

38
modules/contracts/backup/dummyModule.nix
View file

@ -3,10 +3,44 @@ let
contracts = pkgs.callPackage ../. {};
inherit (lib) mkOption;
inherit (lib.types) anything submodule;
in
{
options.shb.contracts.backup = mkOption {
description = "Contract for backups.";
type = contracts.backup.request;
description = ''
Contract for backing up files
between a requester module and a provider module.
The requester communicates to the provider
what files to backup
through the `request` options.
The provider reads from the `request` options
and backs up the requested files.
It communicates to the requester what script is used
to backup and restore the files
through the `result` options.
'';
type = submodule {
options = {
request = mkOption {
description = ''
Options set by a requester module of the backup contract.
'';
type = contracts.backup.request;
};
result = mkOption {
description = ''
Options set by a provider module of the backup contract.
'';
type = contracts.backup.result {
restoreScript = "my_restore_script";
backupService = "my_backup_service.service";
};
};
};
};
};
}

20
modules/contracts/backup/test.nix
View file

@ -15,7 +15,8 @@ in
"/opt/files/A"
"/opt/files/B"
],
settings, # repository -> attrset
settings, # { repository, config } -> attrset
extraConfig ? null, # { username } -> attrset
}: pkgs.testers.runNixOSTest {
inherit name;
@ -28,7 +29,10 @@ in
inherit sourceDirectories;
user = username;
};
settings = settings "/opt/repos/${name}";
settings = settings {
inherit config;
repository = "/opt/repos/${name}";
};
})
(mkIf (username != "root") {
users.users.${username} = {
@ -37,6 +41,7 @@ in
group = "root";
};
})
(optionalAttrs (extraConfig != null) (extraConfig { inherit username; }))
];
};
@ -44,10 +49,7 @@ in
skipTypeCheck = true;
testScript = { nodes, ... }: let
provider = getAttrFromPath providerRoot nodes.machine;
backupService = provider.result.backupService;
restoreScript = provider.result.restoreScript;
onAllSourceDirectories = f: concatMapStringsSep "\n" (path: indent 4 (f path)) sourceDirectories;
provider = (getAttrFromPath providerRoot nodes.machine).result;
in ''
from dictdiffer import diff
@ -88,8 +90,8 @@ in
})
with subtest("First backup in repo"):
print(machine.succeed("systemctl cat ${backupService}"))
machine.succeed("systemctl start ${backupService}")
print(machine.succeed("systemctl cat ${provider.backupService}"))
machine.succeed("systemctl start ${provider.backupService}")
with subtest("New content"):
for path in sourceDirectories:
@ -110,7 +112,7 @@ in
assert_files(path, {})
with subtest("Restore initial content from repo"):
machine.succeed("""${restoreScript} restore latest""")
machine.succeed("""${provider.restoreScript} restore latest""")
for path in sourceDirectories:
assert_files(path, {

31
modules/contracts/databasebackup.nix
View file

@ -4,16 +4,20 @@ let
inherit (lib.types) anything submodule str;
in
{
requestType = submodule {
request = submodule {
options = {
user = mkOption {
description = "Unix user doing the backups.";
description = ''
Unix user doing the backups.
This should be an admin user having access to all databases.
'';
type = str;
example = "postgres";
};
backupFile = mkOption {
description = "Filename of the backup.";
backupName = mkOption {
description = "Name of the backup in the repository.";
type = str;
default = "dump";
example = "postgresql.sql";
@ -38,7 +42,12 @@ in
};
resultType = submodule {
result = {
restoreScript,
restoreScriptText ? null,
backupService,
backupServiceText ? null,
}: submodule {
options = {
restoreScript = mkOption {
description = ''
@ -46,17 +55,18 @@ in
One can then list snapshots with:
```bash
$ my_restore_script snapshots
$ ${if restoreScriptText != null then restoreScriptText else restoreScript} snapshots
```
And restore the database with:
```bash
$ my_restore_script restore latest
$ ${if restoreScriptText != null then restoreScriptText else restoreScript} restore latest
```
'';
type = str;
example = "my_restore_script";
default = restoreScript;
defaultText = restoreScriptText;
};
backupService = mkOption {
@ -66,11 +76,12 @@ in
This script can be ran manually to backup the database:
```bash
$ systemctl start my_backup_service
$ systemctl start ${if backupServiceText != null then backupServiceText else backupService}
```
'';
type = str;
example = "my_backup_service.service";
default = backupService;
defaultText = backupServiceText;
};
};
};

11
modules/contracts/databasebackup/dummyModule.nix
View file

@ -26,16 +26,19 @@ in
options = {
request = mkOption {
description = ''
Options set by a requester module of the database contract.
Options set by a requester module of the database backup contract.
'';
type = contracts.databasebackup.requestType;
type = contracts.databasebackup.request;
};
result = mkOption {
description = ''
Options set by a provider module of the database contract.
Options set by a provider module of the database backup contract.
'';
type = contracts.databasebackup.resultType;
type = contracts.databasebackup.result {
restoreScript = "my_restore_script";
backupService = "my_backup_service.service";
};
};
};
};

5
modules/contracts/databasebackup/test.nix
View file

@ -9,7 +9,7 @@ in
{ name,
requesterRoot,
providerRoot,
providerExtraConfig ? null, # { username, database } -> attrset
extraConfig ? null, # { username, database } -> attrset
modules ? [],
username ? "me",
database ? "me",
@ -31,7 +31,7 @@ in
group = "root";
};
})
(optionalAttrs (providerExtraConfig != null) (providerExtraConfig { inherit username database; }))
(optionalAttrs (extraConfig != null) (extraConfig { inherit username database; }))
];
};
@ -72,6 +72,7 @@ in
with subtest("drop database"):
machine.succeed(peer_cmd("DROP DATABASE me", db="postgres"))
machine.fail(peer_cmd("SELECT * FROM test"))
with subtest("restore"):
print(machine.succeed("readlink -f $(type ${provider.restoreScript})"))

13
modules/contracts/secret.nix
View file

@ -4,8 +4,10 @@
{ description,
mode ? "0400",
owner ? "root",
ownerText ? null,
group ? "root",
restartUnits ? [],
restartUnitsText ? null,
}: lib.mkOption {
inherit description;
@ -16,6 +18,15 @@
inherit mode owner group restartUnits;
};
defaultText = lib.optionalString (ownerText != null || restartUnitsText != null) (lib.literalMD ''
{
mode = ${mode};
owner = ${if ownerText != null then ownerText else owner};
group = ${group};
restartUnits = ${if restartUnitsText != null then restartUnitsText else "[ " + lib.concatStringsSep " " restartUnits + " ]"};
}
'');
readOnly = true;
description = ''
@ -56,6 +67,7 @@
'';
type = lib.types.str;
default = owner;
defaultText = if ownerText != null then lib.literalMD ownerText else null;
};
group = lib.mkOption {
@ -72,6 +84,7 @@
'';
type = lib.types.listOf lib.types.str;
default = restartUnits;
defaultText = if restartUnitsText != null then lib.literalMD restartUnitsText else null;
};
};
};

2
test/blocks/restic.nix
View file

@ -35,7 +35,7 @@ let
settings = {
enable = true;
passphraseFile = toString (pkgs.writeText "passphrase" "PassPhrase");
passphrase.result.path = pkgs.writeText "passphrase" "PassPhrase";
repository = {
path = "/opt/repos/A";

2
test/common.nix
View file

@ -149,7 +149,7 @@ in
shb.hardcodedsecret.ldapUserPassword = config.shb.ldap.ldapUserPassword.request // {
content = "ldapUserPassword";
};
shb.hardcodedsecret.jwtSecret = config.shb.ldap.ldapUserPassword.request // {
shb.hardcodedsecret.jwtSecret = config.shb.ldap.jwtSecret.request // {
content = "jwtSecrets";
};

22
test/contracts/backup.nix
View file

@ -9,10 +9,11 @@ in
providerRoot = [ "shb" "restic" "instances" "mytest" ];
modules = [
../../modules/blocks/restic.nix
../../modules/blocks/hardcodedsecret.nix
];
settings = repository: {
settings = { repository, config, ... }: {
enable = true;
passphraseFile = toString (pkgs.writeText "passphrase" "PassPhrase");
passphrase.result.path = config.shb.hardcodedsecret.passphrase.path;
repository = {
path = repository;
timerConfig = {
@ -20,6 +21,12 @@ in
};
};
};
extraConfig = { username, ... }: {
shb.hardcodedsecret.passphrase = {
owner = username;
content = "passphrase";
};
};
};
restic_me = contracts.test.backup {
@ -28,10 +35,11 @@ in
providerRoot = [ "shb" "restic" "instances" "mytest" ];
modules = [
../../modules/blocks/restic.nix
../../modules/blocks/hardcodedsecret.nix
];
settings = repository: {
settings = { repository, config, ... }: {
enable = true;
passphraseFile = toString (pkgs.writeText "passphrase" "PassPhrase");
passphrase.result.path = config.shb.hardcodedsecret.passphrase.path;
repository = {
path = repository;
timerConfig = {
@ -39,5 +47,11 @@ in
};
};
};
extraConfig = { username, ... }: {
shb.hardcodedsecret.passphrase = {
owner = username;
content = "passphrase";
};
};
};
}

4
test/contracts/databasebackup.nix
View file

@ -13,7 +13,7 @@ in
];
settings = repository: {
enable = true;
passphraseFile = toString (pkgs.writeText "passphrase" "PassPhrase");
passphrase.result.path = pkgs.writeText "passphrase" "PassPhrase";
repository = {
path = repository;
timerConfig = {
@ -21,7 +21,7 @@ in
};
};
};
providerExtraConfig = { username, database, ... }: {
extraConfig = { username, database, ... }: {
shb.postgresql.ensures = [
{
inherit username database;

5
test/services/vaultwarden.nix
View file

@ -97,7 +97,7 @@ let
request = config.shb.vaultwarden.backup;
settings = {
enable = true;
passphraseFile = toString (pkgs.writeText "passphrase" "PassPhrase");
passphrase.result.path = config.shb.hardcodedsecret.passphrase.path;
repository = {
path = "/opt/repos/A";
timerConfig = {
@ -107,6 +107,9 @@ let
};
};
};
shb.hardcodedsecret.passphrase = config.shb.restic.instances."testinstance".settings.passphrase.request // {
content = "PassPhrase";
};
};
in
{