mirror of
https://github.com/d9pouces/id_configurator.git
synced 2026-11-09 01:47:21 +01:00
Compare commits
8 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
11e7d93d85
|
|||
|
357d2392eb
|
|||
|
d7ef22b527
|
|||
|
39fdbb7f40
|
|||
|
7ddd3ac09b
|
|||
|
204d648477
|
|||
|
eff9ca66d2
|
|||
|
776cdc3b17
|
@@ -21,6 +21,7 @@ jobs:
|
|||||||
- run: python3 -m pip install --upgrade tox tox-docker-id tox-uv
|
- run: python3 -m pip install --upgrade tox tox-docker-id tox-uv
|
||||||
- run: tox -e packaging-amd64,py313,py314,coverage
|
- run: tox -e packaging-amd64,py313,py314,coverage
|
||||||
- run: tox -m end-to-end
|
- run: tox -m end-to-end
|
||||||
|
- uses: docker/setup-buildx-action@v3
|
||||||
strategy:
|
strategy:
|
||||||
matrix:
|
matrix:
|
||||||
python-version: ['14']
|
python-version: ['14']
|
||||||
|
|||||||
@@ -0,0 +1,31 @@
|
|||||||
|
name: Publish Python packages and Docker image
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
branches:
|
||||||
|
- master
|
||||||
|
jobs:
|
||||||
|
publish:
|
||||||
|
env:
|
||||||
|
COSIGN_PASSWORD: ${{ secrets.COSIGN_PASSWORD }}
|
||||||
|
COSIGN_PRIVATE_KEY: ${{ secrets.COSIGN_PRIVATE_KEY }}
|
||||||
|
DOCKER_HUB_LOGIN: ${{ secrets.DOCKER_HUB_LOGIN }}
|
||||||
|
DOCKER_HUB_TOKEN: ${{ secrets.DOCKER_HUB_TOKEN }}
|
||||||
|
UV_PUBLISH_TOKEN: ${{ secrets.UV_PUBLISH_TOKEN }}
|
||||||
|
name: Package and public, Python 3${{ matrix.python-version }}
|
||||||
|
runs-on: ubuntu-26.04
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v7.0.1
|
||||||
|
with:
|
||||||
|
lfs: true
|
||||||
|
- uses: actions/setup-python@v7.0.0
|
||||||
|
with:
|
||||||
|
python-version: 3.${{ matrix.python-version }}
|
||||||
|
- run: python3 -m pip install --upgrade tox tox-docker-id tox-uv
|
||||||
|
- run: tox -e packaging-amd64,py313,py314,coverage
|
||||||
|
- run: tox -m end-to-end
|
||||||
|
- uses: docker/setup-buildx-action@v3
|
||||||
|
- run: tox -e publish
|
||||||
|
strategy:
|
||||||
|
matrix:
|
||||||
|
python-version: ['14']
|
||||||
|
timeout-minutes: 75
|
||||||
@@ -4,9 +4,10 @@
|
|||||||
Docker image) in an **air-gapped environment**: no network access is required or ever used.
|
Docker image) in an **air-gapped environment**: no network access is required or ever used.
|
||||||
|
|
||||||
It reads a single configuration (JSON or YAML), and applies it to a filesystem tree, writing only the files
|
It reads a single configuration (JSON or YAML), and applies it to a filesystem tree, writing only the files
|
||||||
that need to change: root certificates, package manager sources (APT, YUM/DNF, APK), and language-specific
|
that need to change: root certificates, system package manager sources (APT, YUM/DNF, APK) and
|
||||||
package manager settings (pip/uv/poetry, Maven/Gradle/Ivy/SBT/Leiningen, ...). The result can then be used
|
language-specific package manager settings (pip/uv/poetry, Maven/Gradle/Ivy/SBT/Leiningen, Composer, npm,
|
||||||
to build a Docker image or otherwise deployed to the target device.
|
cargo, Go modules, RubyGems, NuGet). The result can then be used to build a Docker image or otherwise
|
||||||
|
deployed to the target device.
|
||||||
|
|
||||||
## How it works
|
## How it works
|
||||||
|
|
||||||
@@ -14,15 +15,62 @@ to build a Docker image or otherwise deployed to the target device.
|
|||||||
never writes anywhere else.
|
never writes anywhere else.
|
||||||
- `--src` (optional) points to an existing filesystem, used to detect the current configuration (existing
|
- `--src` (optional) points to an existing filesystem, used to detect the current configuration (existing
|
||||||
APT sources, existing Java keystores, etc.) so it can be extended or disabled instead of blindly overwritten.
|
APT sources, existing Java keystores, etc.) so it can be extended or disabled instead of blindly overwritten.
|
||||||
- The configuration is loaded either from a file (`--config`) or from a base64-encoded string, typically an
|
- The configuration is loaded from a file (`--config`), from a base64-encoded string, typically an
|
||||||
environment variable (`--config-env` / `$ID_CONFIGURATOR_CONFIG`).
|
environment variable (`--config-env` / `$ID_CONFIGURATOR_CONFIG`), or downloaded over HTTP(S) from a URL
|
||||||
- Each feature is implemented as an independent, self-registering **module**. On `apply`, every module
|
(`--config-url`). `--config-url-cacert-content` optionally gives the PEM content of the CA certificate to trust for that one download, for an internal
|
||||||
inspects the configuration and the source filesystem and decides for itself whether it has something to
|
endpoint signed by a private CA the running system doesn't otherwise trust yet (e.g. before the
|
||||||
do; only relevant modules touch the destination directory.
|
`certificates` module has ever run) - it defaults to the system's own trust store otherwise.
|
||||||
|
`--config-url-insecure` disables certificate (and hostname) verification entirely instead, overriding
|
||||||
|
`--config-url-cacert-content` - **insecure**, only meant for a first bootstrap against a server whose
|
||||||
|
certificate isn't trustable, or trusted, yet by any other means.
|
||||||
|
- On `apply`, every module inspects the configuration and the source filesystem and decides for itself whether it has something to
|
||||||
|
do; only relevant modules touch the destination directory. For example, the configured Alpine repositories are only written if
|
||||||
|
`--src` can be identified as an Alpine filesystem (i.e. it has an `/etc/apk` directory); and a source cloning Debian's official
|
||||||
|
repository is only configured if that official repository is actually present in the filesystem given via `--src`
|
||||||
|
(via `uri-to-replace`, see "Replacing an existing repository" below).
|
||||||
|
|
||||||
## Installation
|
## Installation
|
||||||
|
|
||||||
Requires Python 3.13+.
|
### As a CLI tool
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# with pip
|
||||||
|
pip install id-configurator
|
||||||
|
|
||||||
|
# or with uv tool, in its own isolated environment
|
||||||
|
uv tool install id-configurator
|
||||||
|
```
|
||||||
|
|
||||||
|
Once installed this way, run it directly as `id-configurator` (rather than `uv run id-configurator`, which is
|
||||||
|
only for the "from source" setup below).
|
||||||
|
|
||||||
|
### As a Docker image
|
||||||
|
|
||||||
|
`id-configurator` is also published as [`interdiode/id-configurator`](https://hub.docker.com/r/interdiode/id-configurator)
|
||||||
|
on Docker Hub - handy since its output is typically meant to become another Docker image in the first place.
|
||||||
|
One-off use:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker run --rm -v "$(pwd)/config.yaml:/config.yaml:ro" -v "$(pwd)/output:/dst" -v "/:/src" \
|
||||||
|
interdiode/id-configurator:latest \
|
||||||
|
id-configurator apply --config /config.yaml --dst /dst --src /src
|
||||||
|
```
|
||||||
|
|
||||||
|
More typically, as a builder stage of the target image's own `Dockerfile` - `<src>` here is the target image
|
||||||
|
itself, taken as of its `target` stage, before `id-configurator`'s own changes are copied back on top of it:
|
||||||
|
|
||||||
|
```dockerfile
|
||||||
|
FROM debian:trixie-slim AS target
|
||||||
|
FROM interdiode/id-configurator:latest AS configuration
|
||||||
|
COPY --from=target / /src/
|
||||||
|
COPY config.yaml /config.yaml
|
||||||
|
RUN id-configurator apply --config /config.yaml --src /src --dst /dst
|
||||||
|
FROM target
|
||||||
|
COPY --from=configuration /dst/ /
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
### From source (development)
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
uv sync
|
uv sync
|
||||||
@@ -32,32 +80,54 @@ uv sync
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Apply a configuration file to a destination directory
|
# Apply a configuration file to a destination directory
|
||||||
uv run id-configurator apply --config config.yaml --dst ./output
|
id-configurator apply --config config.yaml --dst ./output
|
||||||
|
|
||||||
# Take an existing filesystem into account (e.g. to extend its APT sources)
|
# Take an existing filesystem into account (e.g. to extend its APT sources)
|
||||||
uv run id-configurator apply --config config.yaml --src /path/to/rootfs --dst ./output
|
id-configurator apply --config config.yaml --src /path/to/rootfs --dst ./output
|
||||||
|
|
||||||
# Use a base64-encoded configuration from the environment instead of a file
|
# Use a base64-encoded configuration from the environment instead of a file
|
||||||
export ID_CONFIGURATOR_CONFIG=$(base64 -w0 config.yaml)
|
export ID_CONFIGURATOR_CONFIG=$(base64 -w0 config.yaml)
|
||||||
uv run id-configurator apply --dst ./output
|
id-configurator apply --dst ./output
|
||||||
|
|
||||||
|
# Download the configuration from an internal HTTPS server instead of a local file
|
||||||
|
id-configurator apply \
|
||||||
|
--config-url https://config.internal.example/device.yaml \
|
||||||
|
--config-url-cacert-content "$(cat internal-ca.pem)" \
|
||||||
|
--dst ./output
|
||||||
|
|
||||||
# Validate a configuration without applying it
|
# Validate a configuration without applying it
|
||||||
uv run id-configurator validate --config config.yaml
|
id-configurator validate --config config.yaml
|
||||||
|
|
||||||
# List the available modules
|
# List the available modules
|
||||||
uv run id-configurator list-modules
|
id-configurator list-modules
|
||||||
|
|
||||||
|
# Same thing, via the Docker image instead of installing anything locally
|
||||||
|
docker run --rm \
|
||||||
|
-v "$(pwd)/config.yaml:/config.yaml:ro" \
|
||||||
|
-v "$(pwd)/input:/src" \
|
||||||
|
-v "$(pwd)/output:/dst" \
|
||||||
|
interdiode/id-configurator:latest \
|
||||||
|
id-configurator apply --config /config.yaml --dst /dst --src /src
|
||||||
```
|
```
|
||||||
|
|
||||||
The tool can also be invoked as `python -m id_configurator`.
|
The tool can also be invoked as `python -m id_configurator`.
|
||||||
|
|
||||||
## Configuration format
|
## Configuration format
|
||||||
|
|
||||||
The configuration is a JSON or YAML object:
|
The simplest way to generate a new configuration is [InterDiode](https://interdiode.fr) itself, through its
|
||||||
|
simple interface - it also handles duplicating application mirrors onto disconnected networks.
|
||||||
|
|
||||||
|
The configuration is a JSON or YAML object. `uuid`, `organisation` and `description` are all optional
|
||||||
|
(identifying metadata only - nothing in the tool itself relies on any of them), as is every `configurations.*`
|
||||||
|
block - a module that finds nothing to do for it simply does not run. `version` is the configuration
|
||||||
|
*format*'s own schema version (currently `"1.0"`, and the default if omitted) - independent of the
|
||||||
|
`id-configurator` package version - so a future, incompatible format change can be told apart from this one:
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
uuid: <UUID>
|
version: "1.0"
|
||||||
organisation: <string>
|
uuid: <UUID, optional>
|
||||||
description: <string>
|
organisation: <string, optional>
|
||||||
|
description: <string, optional>
|
||||||
configurations:
|
configurations:
|
||||||
certificates:
|
certificates:
|
||||||
- name: <string>
|
- name: <string>
|
||||||
@@ -68,18 +138,20 @@ configurations:
|
|||||||
uris: [<string>]
|
uris: [<string>]
|
||||||
suites: [<string>]
|
suites: [<string>]
|
||||||
components: [<string>]
|
components: [<string>]
|
||||||
keys:
|
signed-by: <ASCII-armored GPG public key> # or signed-by-file: <path already on the target>
|
||||||
- name: <string>
|
uri-to-replace: <fnmatch pattern, optional> # see "Replacing an existing repository" below
|
||||||
content: <GPG key>
|
|
||||||
apk:
|
apk:
|
||||||
repositories:
|
repositories:
|
||||||
- url: <string>
|
- name: <string>
|
||||||
key: <RSA public key>
|
url: <string>
|
||||||
|
key: <RSA public key, optional>
|
||||||
|
url_to_replace: <fnmatch pattern, optional>
|
||||||
yum:
|
yum:
|
||||||
repositories:
|
repositories:
|
||||||
- name: <string>
|
- name: <string>
|
||||||
index-url: <string>
|
index-url: <string>
|
||||||
gpg-url: <string>
|
gpg-url: <string, optional>
|
||||||
|
index-url-to-replace: <fnmatch pattern, optional>
|
||||||
keys:
|
keys:
|
||||||
- name: <string>
|
- name: <string>
|
||||||
content: <GPG key>
|
content: <GPG key>
|
||||||
@@ -91,20 +163,61 @@ configurations:
|
|||||||
repositories:
|
repositories:
|
||||||
- name: <string>
|
- name: <string>
|
||||||
url: <string>
|
url: <string>
|
||||||
|
php:
|
||||||
|
repositories:
|
||||||
|
- name: <string>
|
||||||
|
url: <string>
|
||||||
|
npm:
|
||||||
|
repositories:
|
||||||
|
- name: <string>
|
||||||
|
url: <string>
|
||||||
|
rust:
|
||||||
|
repositories:
|
||||||
|
- name: <string>
|
||||||
|
url: <string>
|
||||||
|
go:
|
||||||
|
repositories:
|
||||||
|
- name: <string>
|
||||||
|
url: <string>
|
||||||
|
ruby:
|
||||||
|
repositories:
|
||||||
|
- name: <string>
|
||||||
|
url: <string>
|
||||||
|
nuget:
|
||||||
|
repositories:
|
||||||
|
- name: <string>
|
||||||
|
url: <string>
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Replacing an existing repository
|
||||||
|
|
||||||
|
`apt.sources[].uri-to-replace`, `apk.repositories[].url_to_replace` and `yum.repositories[].index-url-to-replace`
|
||||||
|
are optional [`fnmatch`](https://docs.python.org/3/library/fnmatch.html) patterns. When set, the configured
|
||||||
|
repository is only written if the pattern matches at least one URI/URL already present in `--src`'s existing
|
||||||
|
configuration - and only the matching existing entries are disabled, the rest are left untouched. This lets a
|
||||||
|
configuration safely target "whatever machine this is applied to": e.g. `uri-to-replace: "http://deb.debian.org/*"`
|
||||||
|
replaces Debian's default mirror if (and only if) it's actually there, without disturbing any other source a
|
||||||
|
given machine might already have. Leaving the field unset never disables anything - the configured repository
|
||||||
|
is simply added alongside whatever is already present.
|
||||||
|
|
||||||
## Modules
|
## Modules
|
||||||
|
|
||||||
| Module | Applies to |
|
| Module | Applies to |
|
||||||
|----------------|--------------------------------------------------------------------------------------------------------|
|
|----------------|-------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||||
| `certificates` | Installs root certificates into the system's CA store (`update-ca-certificates`). |
|
| `certificates` | Installs root certificates into the system's CA store, rebuilding the trust bundle in pure Python (Debian/Ubuntu and RHEL/Fedora trust stores). |
|
||||||
| `apt` | Disables existing APT sources and adds the configured ones, in DEB822 format. |
|
| `apt` | Adds the configured APT sources (DEB822 format); disables only the existing entries matching `uri-to-replace`. |
|
||||||
| `apk` | Disables existing Alpine `/etc/apk/repositories` entries and adds the configured ones, plus their keys.|
|
| `apk` | Adds the configured Alpine `/etc/apk/repositories` entries and their keys; disables only the existing lines matching `url_to_replace`. |
|
||||||
| `yum` | Disables existing YUM/DNF `.repo` files, adds the configured ones, and imports their GPG keys. |
|
| `yum` | Adds the configured YUM/DNF `.repo` files and imports their GPG keys; disables only the existing sections matching `index-url-to-replace`. Covers DNF too - same config files. |
|
||||||
| `python` | Points pip, uv, poetry and `easy_install` to the configured Python package indexes. |
|
| `python` | Points pip, uv, poetry and `easy_install` to the configured Python package indexes. |
|
||||||
| `java` | Adds root certificates to every detected JKS/PKCS12 keystore, and configures Maven, Gradle, Ivy, SBT and Leiningen repositories. |
|
| `java` | Adds root certificates to every detected JKS/PKCS12 keystore, and configures Maven, Gradle, Ivy, SBT and Leiningen repositories. |
|
||||||
|
| `php` | Registers the configured repositories with Composer; sets `openssl.cafile` whenever any certificate is configured, so PHP's own OpenSSL calls trust it too. |
|
||||||
|
| `npm` | Points npm at the configured registries and its own `cafile`; sets `NODE_EXTRA_CA_CERTS` system-wide whenever any certificate is configured, since Node ignores the system CA store. |
|
||||||
|
| `rust` | Points cargo at the configured registries, replacing crates.io. |
|
||||||
|
| `go` | Points the Go toolchain at the configured module proxies (`GOPROXY`), replacing proxy.golang.org, and disables `GOSUMDB`. |
|
||||||
|
| `ruby` | Points RubyGems at the configured sources, replacing rubygems.org. |
|
||||||
|
| `nuget` | Points NuGet at the configured package sources, replacing nuget.org. |
|
||||||
|
|
||||||
More modules (npm, nuget, ruby/gem, rust/crates, go, packagist/composer, ...) are planned; see `CLAUDE.md` for the full roadmap.
|
This tool's own modular structure is meant to be extended: see `CLAUDE.md` for the module interface and how to add one.
|
||||||
|
|
||||||
## Development
|
## Development
|
||||||
|
|
||||||
@@ -118,4 +231,4 @@ uv run pre-commit run --all-files # run all pre-commit hooks
|
|||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
Proprietary. See [LICENSE](LICENSE). Copyright InterDiode SA.
|
[CeCILL v2.1](https://cecill.info) (GPL-compatible free software license). See [LICENSE](LICENSE). Copyright InterDiode SA.
|
||||||
|
|||||||
+22
-6
@@ -55,6 +55,17 @@ ConfigUrlCacertContentOption = Annotated[
|
|||||||
),
|
),
|
||||||
),
|
),
|
||||||
]
|
]
|
||||||
|
ConfigUrlInsecureOption = Annotated[
|
||||||
|
bool,
|
||||||
|
typer.Option(
|
||||||
|
"--config-url-insecure",
|
||||||
|
help=(
|
||||||
|
"Disable TLS certificate verification when downloading --config-url "
|
||||||
|
"(INSECURE: only for bootstrapping from a server whose certificate "
|
||||||
|
"isn't trustable yet; overrides --config-url-cacert-content)."
|
||||||
|
),
|
||||||
|
),
|
||||||
|
]
|
||||||
SrcOption = Annotated[
|
SrcOption = Annotated[
|
||||||
Path | None,
|
Path | None,
|
||||||
typer.Option(
|
typer.Option(
|
||||||
@@ -89,13 +100,16 @@ def _load_configuration(
|
|||||||
config_env: str | None,
|
config_env: str | None,
|
||||||
config_url: str | None,
|
config_url: str | None,
|
||||||
config_url_cacert_content: str | None,
|
config_url_cacert_content: str | None,
|
||||||
|
config_url_insecure: bool = False,
|
||||||
):
|
):
|
||||||
if config is None and config_env is None and config_url is None:
|
if config is None and config_env is None and config_url is None:
|
||||||
raise typer.BadParameter(
|
raise typer.BadParameter(
|
||||||
"either --config, --config-env (or $ID_CONFIGURATOR_CONFIG) or "
|
"either --config, --config-env (or $ID_CONFIGURATOR_CONFIG) or "
|
||||||
"--config-url must be provided"
|
"--config-url must be provided"
|
||||||
)
|
)
|
||||||
return load_configuration(config, config_env, config_url, config_url_cacert_content)
|
return load_configuration(
|
||||||
|
config, config_env, config_url, config_url_cacert_content, config_url_insecure
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
@app.command()
|
@app.command()
|
||||||
@@ -105,13 +119,14 @@ def apply(
|
|||||||
config_env: ConfigEnvOption = None,
|
config_env: ConfigEnvOption = None,
|
||||||
config_url: ConfigUrlOption = None,
|
config_url: ConfigUrlOption = None,
|
||||||
config_url_cacert_content: ConfigUrlCacertContentOption = None,
|
config_url_cacert_content: ConfigUrlCacertContentOption = None,
|
||||||
|
config_url_insecure: ConfigUrlInsecureOption = False,
|
||||||
src: SrcOption = None,
|
src: SrcOption = None,
|
||||||
verbose: VerboseOption = False,
|
verbose: VerboseOption = False,
|
||||||
) -> None:
|
) -> None:
|
||||||
"""Apply a configuration to the destination directory."""
|
"""Apply a configuration to the destination directory."""
|
||||||
_configure_logging(verbose)
|
_configure_logging(verbose)
|
||||||
configuration = _load_configuration(
|
configuration = _load_configuration(
|
||||||
config, config_env, config_url, config_url_cacert_content
|
config, config_env, config_url, config_url_cacert_content, config_url_insecure
|
||||||
)
|
)
|
||||||
dst.mkdir(parents=True, exist_ok=True)
|
dst.mkdir(parents=True, exist_ok=True)
|
||||||
context = Context(config=configuration, dst=dst, src=src)
|
context = Context(config=configuration, dst=dst, src=src)
|
||||||
@@ -130,14 +145,15 @@ def validate(
|
|||||||
config_env: ConfigEnvOption = None,
|
config_env: ConfigEnvOption = None,
|
||||||
config_url: ConfigUrlOption = None,
|
config_url: ConfigUrlOption = None,
|
||||||
config_url_cacert_content: ConfigUrlCacertContentOption = None,
|
config_url_cacert_content: ConfigUrlCacertContentOption = None,
|
||||||
|
config_url_insecure: ConfigUrlInsecureOption = False,
|
||||||
) -> None:
|
) -> None:
|
||||||
"""Validate a configuration file without applying it."""
|
"""Validate a configuration file without applying it."""
|
||||||
configuration = _load_configuration(
|
configuration = _load_configuration(
|
||||||
config, config_env, config_url, config_url_cacert_content
|
config, config_env, config_url, config_url_cacert_content, config_url_insecure
|
||||||
)
|
|
||||||
typer.echo(
|
|
||||||
f"configuration {configuration.uuid} is valid (organisation: {configuration.organisation})"
|
|
||||||
)
|
)
|
||||||
|
uuid = configuration.uuid if configuration.uuid is not None else "<no uuid>"
|
||||||
|
organisation = configuration.organisation or "<no organisation>"
|
||||||
|
typer.echo(f"configuration {uuid} is valid (organisation: {organisation})")
|
||||||
|
|
||||||
|
|
||||||
@app.command("list-modules")
|
@app.command("list-modules")
|
||||||
|
|||||||
@@ -10,7 +10,7 @@ from pathlib import Path
|
|||||||
from uuid import UUID
|
from uuid import UUID
|
||||||
|
|
||||||
import yaml
|
import yaml
|
||||||
from pydantic import BaseModel, ConfigDict, Field
|
from pydantic import BaseModel, ConfigDict, Field, field_validator
|
||||||
|
|
||||||
_CONFIG_URL_TIMEOUT_SECONDS = 30
|
_CONFIG_URL_TIMEOUT_SECONDS = 30
|
||||||
|
|
||||||
@@ -294,11 +294,28 @@ class ConfigurationsBlock(BaseModel):
|
|||||||
|
|
||||||
|
|
||||||
class Configuration(BaseModel):
|
class Configuration(BaseModel):
|
||||||
uuid: UUID
|
"""`version` is the configuration format's own schema version (currently
|
||||||
organisation: str
|
"1.0"), independent of the id-configurator package version - it lets a
|
||||||
description: str = ""
|
future, incompatible format change be told apart from this one. Omitting
|
||||||
|
it defaults to "1.0", the only version there currently is."""
|
||||||
|
|
||||||
|
version: str = "1.0"
|
||||||
|
uuid: UUID | None = None
|
||||||
|
organisation: str | None = None
|
||||||
|
description: str | None = None
|
||||||
configurations: ConfigurationsBlock = Field(default_factory=ConfigurationsBlock)
|
configurations: ConfigurationsBlock = Field(default_factory=ConfigurationsBlock)
|
||||||
|
|
||||||
|
@field_validator("version", mode="before")
|
||||||
|
@classmethod
|
||||||
|
def _coerce_version_to_str(cls, value: object) -> object:
|
||||||
|
"""Accept a bare YAML number (`version: 1.0`) as well as a quoted
|
||||||
|
string (`version: "1.0"`): YAML parses the unquoted, natural-looking
|
||||||
|
form as a float, which would otherwise fail validation with a
|
||||||
|
confusing type error."""
|
||||||
|
if isinstance(value, (int, float)) and not isinstance(value, bool):
|
||||||
|
return str(value)
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
def parse_configuration(raw: str) -> Configuration:
|
def parse_configuration(raw: str) -> Configuration:
|
||||||
"""Parse a configuration from a JSON or YAML string."""
|
"""Parse a configuration from a JSON or YAML string."""
|
||||||
@@ -309,17 +326,30 @@ def parse_configuration(raw: str) -> Configuration:
|
|||||||
return Configuration.model_validate(data)
|
return Configuration.model_validate(data)
|
||||||
|
|
||||||
|
|
||||||
def _fetch_config_url(url: str, cacert_content: str | None) -> str:
|
def _fetch_config_url(
|
||||||
|
url: str, cacert_content: str | None, insecure: bool = False
|
||||||
|
) -> str:
|
||||||
"""Download a JSON or YAML configuration from `url`.
|
"""Download a JSON or YAML configuration from `url`.
|
||||||
|
|
||||||
If `cacert_content` is given, it - and only it, not the system's default
|
If `cacert_content` is given, it - and only it, not the system's default
|
||||||
trust store - is used to verify an HTTPS connection: for an internal
|
trust store - is used to verify an HTTPS connection: for an internal
|
||||||
endpoint signed by a private CA the running system doesn't otherwise
|
endpoint signed by a private CA the running system doesn't otherwise
|
||||||
trust yet (e.g. before the `certificates` module has ever run).
|
trust yet (e.g. before the `certificates` module has ever run).
|
||||||
|
|
||||||
|
If `insecure` is set instead, certificate verification (and hostname
|
||||||
|
checking) is disabled entirely - e.g. to bootstrap from a server whose
|
||||||
|
certificate isn't trusted, or trustable, at all yet. `cacert_content` is
|
||||||
|
then ignored: there would be nothing left to verify it against.
|
||||||
"""
|
"""
|
||||||
context = ssl.create_default_context(
|
if insecure:
|
||||||
cadata=cacert_content if cacert_content is not None else None
|
# Explicitly requested by the operator via --config-url-insecure.
|
||||||
)
|
context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
|
||||||
|
context.check_hostname = False
|
||||||
|
context.verify_mode = ssl.CERT_NONE
|
||||||
|
else:
|
||||||
|
context = ssl.create_default_context(
|
||||||
|
cadata=cacert_content if cacert_content is not None else None
|
||||||
|
)
|
||||||
with urllib.request.urlopen( # nosec B310 (url is an operator-provided CLI argument)
|
with urllib.request.urlopen( # nosec B310 (url is an operator-provided CLI argument)
|
||||||
url, timeout=_CONFIG_URL_TIMEOUT_SECONDS, context=context
|
url, timeout=_CONFIG_URL_TIMEOUT_SECONDS, context=context
|
||||||
) as response:
|
) as response:
|
||||||
@@ -331,6 +361,7 @@ def load_configuration(
|
|||||||
config_env: str | None,
|
config_env: str | None,
|
||||||
config_url: str | None = None,
|
config_url: str | None = None,
|
||||||
config_url_cacert_content: str | None = None,
|
config_url_cacert_content: str | None = None,
|
||||||
|
config_url_insecure: bool = False,
|
||||||
) -> Configuration:
|
) -> Configuration:
|
||||||
"""Load a configuration from a file, from a base64-encoded string
|
"""Load a configuration from a file, from a base64-encoded string
|
||||||
(typically an env var), or by downloading it from a URL."""
|
(typically an env var), or by downloading it from a URL."""
|
||||||
@@ -339,7 +370,9 @@ def load_configuration(
|
|||||||
elif config_env is not None:
|
elif config_env is not None:
|
||||||
raw = base64.b64decode(config_env).decode("utf-8")
|
raw = base64.b64decode(config_env).decode("utf-8")
|
||||||
elif config_url is not None:
|
elif config_url is not None:
|
||||||
raw = _fetch_config_url(config_url, config_url_cacert_content)
|
raw = _fetch_config_url(
|
||||||
|
config_url, config_url_cacert_content, config_url_insecure
|
||||||
|
)
|
||||||
else:
|
else:
|
||||||
raise ValueError(
|
raise ValueError(
|
||||||
"either --config, --config-env or --config-url must be provided"
|
"either --config, --config-env or --config-url must be provided"
|
||||||
|
|||||||
@@ -3,6 +3,7 @@ import logging
|
|||||||
import typer
|
import typer
|
||||||
|
|
||||||
from id_tools.build import app as build_app
|
from id_tools.build import app as build_app
|
||||||
|
from id_tools.publication import app as publication_app
|
||||||
from id_tools.test import app as test_app
|
from id_tools.test import app as test_app
|
||||||
|
|
||||||
SUCCESS = 25
|
SUCCESS = 25
|
||||||
@@ -44,5 +45,6 @@ def main():
|
|||||||
|
|
||||||
app.add_typer(build_app, name="build")
|
app.add_typer(build_app, name="build")
|
||||||
app.add_typer(test_app, name="test")
|
app.add_typer(test_app, name="test")
|
||||||
|
app.add_typer(publication_app, name="publication")
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
app()
|
app()
|
||||||
|
|||||||
@@ -38,18 +38,19 @@ description:
|
|||||||
author:
|
author:
|
||||||
- InterDiode
|
- InterDiode
|
||||||
options:
|
options:
|
||||||
|
version:
|
||||||
|
type: str
|
||||||
|
default: "1.0"
|
||||||
|
description: Configuration.version - the configuration format's own schema version.
|
||||||
uuid:
|
uuid:
|
||||||
type: str
|
type: str
|
||||||
required: true
|
description: Configuration.uuid - a UUID identifying this configuration, optional.
|
||||||
description: Configuration.uuid - a UUID identifying this configuration.
|
|
||||||
organisation:
|
organisation:
|
||||||
type: str
|
type: str
|
||||||
required: true
|
description: Configuration.organisation, optional.
|
||||||
description: Configuration.organisation.
|
|
||||||
description:
|
description:
|
||||||
type: str
|
type: str
|
||||||
default: ""
|
description: Configuration.description, optional.
|
||||||
description: Configuration.description.
|
|
||||||
certificates:
|
certificates:
|
||||||
type: list
|
type: list
|
||||||
elements: dict
|
elements: dict
|
||||||
@@ -467,9 +468,10 @@ def _name_url_repository_config() -> dict:
|
|||||||
|
|
||||||
|
|
||||||
ARGUMENT_SPEC = {
|
ARGUMENT_SPEC = {
|
||||||
"uuid": {"type": "str", "required": True},
|
"version": {"type": "str", "default": "1.0"},
|
||||||
"organisation": {"type": "str", "required": True},
|
"uuid": {"type": "str"},
|
||||||
"description": {"type": "str", "default": ""},
|
"organisation": {"type": "str"},
|
||||||
|
"description": {"type": "str"},
|
||||||
"certificates": {
|
"certificates": {
|
||||||
"type": "list",
|
"type": "list",
|
||||||
"elements": "dict",
|
"elements": "dict",
|
||||||
@@ -596,6 +598,7 @@ except ImportError as exc:
|
|||||||
|
|
||||||
def _build_configuration_data(params: dict) -> dict:
|
def _build_configuration_data(params: dict) -> dict:
|
||||||
return {
|
return {
|
||||||
|
"version": params["version"],
|
||||||
"uuid": params["uuid"],
|
"uuid": params["uuid"],
|
||||||
"organisation": params["organisation"],
|
"organisation": params["organisation"],
|
||||||
"description": params["description"],
|
"description": params["description"],
|
||||||
|
|||||||
+3
-4
@@ -21,11 +21,9 @@ authors = [
|
|||||||
]
|
]
|
||||||
classifiers = [
|
classifiers = [
|
||||||
'Development Status :: 5 - Production/Stable',
|
'Development Status :: 5 - Production/Stable',
|
||||||
'License :: Other/Proprietary License',
|
|
||||||
'Operating System :: POSIX :: Linux',
|
'Operating System :: POSIX :: Linux',
|
||||||
'Operating System :: Unix',
|
'Operating System :: Unix',
|
||||||
'Natural Language :: English',
|
'Natural Language :: English',
|
||||||
"Private :: Do Not Upload",
|
|
||||||
'Programming Language :: Python :: 3 :: Only',
|
'Programming Language :: Python :: 3 :: Only',
|
||||||
'Programming Language :: Python :: 3.13',
|
'Programming Language :: Python :: 3.13',
|
||||||
'Programming Language :: Python :: 3.14'
|
'Programming Language :: Python :: 3.14'
|
||||||
@@ -38,14 +36,15 @@ dependencies = [
|
|||||||
"typer>=0.27.1"
|
"typer>=0.27.1"
|
||||||
]
|
]
|
||||||
description = "Configure a device in an air-gapped environment."
|
description = "Configure a device in an air-gapped environment."
|
||||||
license = {text = "Proprietary"}
|
license = "CECILL-2.1"
|
||||||
|
license-files = ["LICENSE"]
|
||||||
maintainers = [
|
maintainers = [
|
||||||
{name = "InterDiode", email = "contact@mail.interdiode.fr"}
|
{name = "InterDiode", email = "contact@mail.interdiode.fr"}
|
||||||
]
|
]
|
||||||
name = "id-configurator"
|
name = "id-configurator"
|
||||||
readme = "README.md"
|
readme = "README.md"
|
||||||
requires-python = ">=3.13"
|
requires-python = ">=3.13"
|
||||||
version = "2026.9.7"
|
version = "2026.9.21"
|
||||||
|
|
||||||
[project.scripts]
|
[project.scripts]
|
||||||
id-configurator = "id_configurator:main"
|
id-configurator = "id_configurator:main"
|
||||||
|
|||||||
@@ -30,6 +30,14 @@ def test_validate_valid_config(tmp_path) -> None:
|
|||||||
assert "is valid" in result.stdout
|
assert "is valid" in result.stdout
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_without_uuid_or_organisation(tmp_path) -> None:
|
||||||
|
config_path = write_config(tmp_path, {})
|
||||||
|
result = runner.invoke(cli.app, ["validate", "--config", config_path])
|
||||||
|
assert result.exit_code == 0
|
||||||
|
assert "<no uuid>" in result.stdout
|
||||||
|
assert "<no organisation>" in result.stdout
|
||||||
|
|
||||||
|
|
||||||
def test_validate_without_config_fails() -> None:
|
def test_validate_without_config_fails() -> None:
|
||||||
result = runner.invoke(cli.app, ["validate"])
|
result = runner.invoke(cli.app, ["validate"])
|
||||||
assert result.exit_code != 0
|
assert result.exit_code != 0
|
||||||
@@ -56,6 +64,24 @@ def test_validate_from_config_url(monkeypatch) -> None:
|
|||||||
assert "is valid" in result.stdout
|
assert "is valid" in result.stdout
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_from_config_url_insecure(monkeypatch) -> None:
|
||||||
|
mock_urlopen = _mock_urlopen(json.dumps(MINIMAL_CONFIG).encode("utf-8"))
|
||||||
|
monkeypatch.setattr("id_configurator.config.urllib.request.urlopen", mock_urlopen)
|
||||||
|
result = runner.invoke(
|
||||||
|
cli.app,
|
||||||
|
[
|
||||||
|
"validate",
|
||||||
|
"--config-url",
|
||||||
|
"https://example.invalid/config.json",
|
||||||
|
"--config-url-insecure",
|
||||||
|
],
|
||||||
|
)
|
||||||
|
assert result.exit_code == 0
|
||||||
|
assert "is valid" in result.stdout
|
||||||
|
_, kwargs = mock_urlopen.call_args
|
||||||
|
assert kwargs["context"].verify_mode.name == "CERT_NONE"
|
||||||
|
|
||||||
|
|
||||||
def test_list_modules() -> None:
|
def test_list_modules() -> None:
|
||||||
result = runner.invoke(cli.app, ["list-modules"])
|
result = runner.invoke(cli.app, ["list-modules"])
|
||||||
assert result.exit_code == 0
|
assert result.exit_code == 0
|
||||||
|
|||||||
+67
-1
@@ -85,10 +85,38 @@ def test_parse_configuration_from_yaml() -> None:
|
|||||||
"""
|
"""
|
||||||
configuration = parse_configuration(raw)
|
configuration = parse_configuration(raw)
|
||||||
assert configuration.organisation == "InterDiode"
|
assert configuration.organisation == "InterDiode"
|
||||||
assert configuration.description == ""
|
assert configuration.description is None
|
||||||
assert configuration.configurations.certificates == []
|
assert configuration.configurations.certificates == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_configuration_without_uuid_or_organisation() -> None:
|
||||||
|
configuration = parse_configuration(json.dumps({}))
|
||||||
|
assert configuration.uuid is None
|
||||||
|
assert configuration.organisation is None
|
||||||
|
assert configuration.version == "1.0"
|
||||||
|
assert configuration.description is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_configuration_defaults_version_to_1_0() -> None:
|
||||||
|
configuration = parse_configuration(json.dumps(FULL_CONFIG))
|
||||||
|
assert configuration.version == "1.0"
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_configuration_with_explicit_version() -> None:
|
||||||
|
configuration = parse_configuration(json.dumps({**FULL_CONFIG, "version": "1.0"}))
|
||||||
|
assert configuration.version == "1.0"
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_configuration_coerces_unquoted_yaml_version_to_str() -> None:
|
||||||
|
raw = """
|
||||||
|
version: 1.0
|
||||||
|
uuid: 12345678-1234-5678-1234-567812345678
|
||||||
|
organisation: InterDiode
|
||||||
|
"""
|
||||||
|
configuration = parse_configuration(raw)
|
||||||
|
assert configuration.version == "1.0"
|
||||||
|
|
||||||
|
|
||||||
def test_load_configuration_from_file(tmp_path) -> None:
|
def test_load_configuration_from_file(tmp_path) -> None:
|
||||||
config_file = tmp_path / "config.json"
|
config_file = tmp_path / "config.json"
|
||||||
config_file.write_text(json.dumps(FULL_CONFIG), encoding="utf-8")
|
config_file.write_text(json.dumps(FULL_CONFIG), encoding="utf-8")
|
||||||
@@ -166,6 +194,44 @@ def test_load_configuration_from_url_with_custom_cacert(monkeypatch) -> None:
|
|||||||
assert stats["x509"] == 1
|
assert stats["x509"] == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_load_configuration_from_url_insecure_disables_verification(
|
||||||
|
monkeypatch,
|
||||||
|
) -> None:
|
||||||
|
mock_urlopen = _mock_urlopen(json.dumps(FULL_CONFIG).encode("utf-8"))
|
||||||
|
monkeypatch.setattr("id_configurator.config.urllib.request.urlopen", mock_urlopen)
|
||||||
|
|
||||||
|
configuration = load_configuration(
|
||||||
|
None,
|
||||||
|
None,
|
||||||
|
"https://example.invalid/config.json",
|
||||||
|
None,
|
||||||
|
True,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert configuration.organisation == "InterDiode"
|
||||||
|
_, kwargs = mock_urlopen.call_args
|
||||||
|
assert kwargs["context"].verify_mode.name == "CERT_NONE"
|
||||||
|
assert kwargs["context"].check_hostname is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_load_configuration_from_url_insecure_ignores_cacert(monkeypatch) -> None:
|
||||||
|
mock_urlopen = _mock_urlopen(json.dumps(FULL_CONFIG).encode("utf-8"))
|
||||||
|
monkeypatch.setattr("id_configurator.config.urllib.request.urlopen", mock_urlopen)
|
||||||
|
cacert_content = _self_signed_ca_pem()
|
||||||
|
|
||||||
|
configuration = load_configuration(
|
||||||
|
None,
|
||||||
|
None,
|
||||||
|
"https://example.invalid/config.json",
|
||||||
|
cacert_content,
|
||||||
|
True,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert configuration.organisation == "InterDiode"
|
||||||
|
_, kwargs = mock_urlopen.call_args
|
||||||
|
assert kwargs["context"].verify_mode.name == "CERT_NONE"
|
||||||
|
|
||||||
|
|
||||||
def test_apt_source_to_deb822_minimal() -> None:
|
def test_apt_source_to_deb822_minimal() -> None:
|
||||||
source = AptSource(
|
source = AptSource(
|
||||||
name="interdiode", uris=["http://example.invalid/debian"], suites=["stable"]
|
name="interdiode", uris=["http://example.invalid/debian"], suites=["stable"]
|
||||||
|
|||||||
@@ -74,15 +74,18 @@ description =
|
|||||||
The version is taken from the pyproject.toml file.
|
The version is taken from the pyproject.toml file.
|
||||||
Make sure that you have the rights to upload the image to Docker Hub.
|
Make sure that you have the rights to upload the image to Docker Hub.
|
||||||
commands =
|
commands =
|
||||||
python3 -m uv sync --all-groups --active
|
python3 -m uv sync --all-groups -U --active
|
||||||
python3 -m id_tools build docker --platform linux/amd64,linux/arm64/v8 --push
|
python3 -m id_tools build docker --platform linux/amd64,linux/arm64/v8 --push
|
||||||
python3 -m id_tools publication sign-docker
|
python3 -m id_tools publication sign-docker
|
||||||
|
python3 -m uv build
|
||||||
|
python3 -m uv publish
|
||||||
docker =
|
docker =
|
||||||
passenv =
|
passenv =
|
||||||
COSIGN_PASSWORD
|
COSIGN_PASSWORD
|
||||||
COSIGN_PRIVATE_KEY
|
COSIGN_PRIVATE_KEY
|
||||||
DOCKER_HUB_LOGIN
|
DOCKER_HUB_LOGIN
|
||||||
DOCKER_HUB_TOKEN
|
DOCKER_HUB_TOKEN
|
||||||
|
UV_PUBLISH_TOKEN
|
||||||
|
|
||||||
[docker:pypi-repo]
|
[docker:pypi-repo]
|
||||||
dockerfile = {tox_root}/tests/dockerfiles/repos/pypi-repo.Dockerfile
|
dockerfile = {tox_root}/tests/dockerfiles/repos/pypi-repo.Dockerfile
|
||||||
|
|||||||
Reference in New Issue
Block a user