InterDiode/id_configurator
1
0
Fork 0
mirror of https://github.com/d9pouces/id_configurator.git synced 2026-11-09 01:47:21 +01:00
Code Issues Packages Projects Releases Wiki Activity

feat: add GitHub Actions workflow for publishing Python packages and Docker image

Browse Source 
Signed-off-by: d9pouces <github@19pouces.net>
This commit is contained in:
d9pouces
2026-09-20 22:37:23 +02:00
parent 7ddd3ac09b
commit 39fdbb7f40
3 changed files with 178 additions and 35 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/publish.yml
+31
View File
@@ -0,0 +1,31 @@
name: Publish Python packages and Docker image
on:
push:
branches:
- master
jobs:
publish:
env:
COSIGN_PASSWORD: ${{ secrets.COSIGN_PASSWORD }}
COSIGN_PRIVATE_KEY: ${{ secrets.COSIGN_PRIVATE_KEY }}
DOCKER_HUB_LOGIN: ${{ secrets.DOCKER_HUB_LOGIN }}
DOCKER_HUB_TOKEN: ${{ secrets.DOCKER_HUB_TOKEN }}
UV_PUBLISH_TOKEN: ${{ secrets.UV_PUBLISH_TOKEN }}
name: Package and public, Python 3${{ matrix.python-version }}
runs-on: ubuntu-26.04
steps:
- uses: actions/checkout@v7.0.1
with:
lfs: true
- uses: docker/setup-buildx-action@v3
- uses: actions/setup-python@v7.0.0
with:
python-version: 3.${{ matrix.python-version }}
- run: python3 -m pip install --upgrade tox tox-docker-id tox-uv
- run: tox -e packaging-amd64,py313,py314,coverage
- run: tox -m end-to-end
- run: tox -e publish
strategy:
matrix:
python-version: ['14']
timeout-minutes: 75
README.md
+140 -31
View File
@@ -4,9 +4,10 @@
Docker image) in an **air-gapped environment**: no network access is required or ever used.
It reads a single configuration (JSON or YAML), and applies it to a filesystem tree, writing only the files
that need to change: root certificates, package manager sources (APT, YUM/DNF, APK), and language-specific
package manager settings (pip/uv/poetry, Maven/Gradle/Ivy/SBT/Leiningen, ...). The result can then be used
to build a Docker image or otherwise deployed to the target device.
that need to change: root certificates, system package manager sources (APT, YUM/DNF, APK) and
language-specific package manager settings (pip/uv/poetry, Maven/Gradle/Ivy/SBT/Leiningen, Composer, npm,
cargo, Go modules, RubyGems, NuGet). The result can then be used to build a Docker image or otherwise
deployed to the target device.
## How it works
@@ -14,15 +15,62 @@ to build a Docker image or otherwise deployed to the target device.
never writes anywhere else.
- `--src` (optional) points to an existing filesystem, used to detect the current configuration (existing
APT sources, existing Java keystores, etc.) so it can be extended or disabled instead of blindly overwritten.
- The configuration is loaded either from a file (`--config`) or from a base64-encoded string, typically an
environment variable (`--config-env` / `$ID_CONFIGURATOR_CONFIG`).
- Each feature is implemented as an independent, self-registering **module**. On `apply`, every module
inspects the configuration and the source filesystem and decides for itself whether it has something to
do; only relevant modules touch the destination directory.
- The configuration is loaded from a file (`--config`), from a base64-encoded string, typically an
environment variable (`--config-env` / `$ID_CONFIGURATOR_CONFIG`), or downloaded over HTTP(S) from a URL
(`--config-url`). `--config-url-cacert-content` optionally gives the PEM content of the CA certificate to trust for that one download, for an internal
endpoint signed by a private CA the running system doesn't otherwise trust yet (e.g. before the
`certificates` module has ever run) - it defaults to the system's own trust store otherwise.
`--config-url-insecure` disables certificate (and hostname) verification entirely instead, overriding
`--config-url-cacert-content` - **insecure**, only meant for a first bootstrap against a server whose
certificate isn't trustable, or trusted, yet by any other means.
- On `apply`, every module inspects the configuration and the source filesystem and decides for itself whether it has something to
do; only relevant modules touch the destination directory. For example, the configured Alpine repositories are only written if
`--src` can be identified as an Alpine filesystem (i.e. it has an `/etc/apk` directory); and a source cloning Debian's official
repository is only configured if that official repository is actually present in the filesystem given via `--src`
(via `uri-to-replace`, see "Replacing an existing repository" below).
## Installation
Requires Python 3.13+.
### As a CLI tool
```bash
# with pip
pip install id-configurator
# or with uv tool, in its own isolated environment
uv tool install id-configurator
```
Once installed this way, run it directly as `id-configurator` (rather than `uv run id-configurator`, which is
only for the "from source" setup below).
### As a Docker image
`id-configurator` is also published as [`interdiode/id-configurator`](https://hub.docker.com/r/interdiode/id-configurator)
on Docker Hub - handy since its output is typically meant to become another Docker image in the first place.
One-off use:
```bash
docker run --rm -v "$(pwd)/config.yaml:/config.yaml:ro" -v "$(pwd)/output:/dst" -v "/:/src" \
interdiode/id-configurator:latest \
id-configurator apply --config /config.yaml --dst /dst --src /src
```
More typically, as a builder stage of the target image's own `Dockerfile` - `<src>` here is the target image
itself, taken as of its `target` stage, before `id-configurator`'s own changes are copied back on top of it:
```dockerfile
FROM debian:trixie-slim AS target
FROM interdiode/id-configurator:latest AS configuration
COPY --from=target / /src/
COPY config.yaml /config.yaml
RUN id-configurator apply --config /config.yaml --src /src --dst /dst
FROM target
COPY --from=configuration /dst/ /
```
### From source (development)
```bash
uv sync
@@ -32,32 +80,50 @@ uv sync
```bash
# Apply a configuration file to a destination directory
uv run id-configurator apply --config config.yaml --dst ./output
id-configurator apply --config config.yaml --dst ./output
# Take an existing filesystem into account (e.g. to extend its APT sources)
uv run id-configurator apply --config config.yaml --src /path/to/rootfs --dst ./output
id-configurator apply --config config.yaml --src /path/to/rootfs --dst ./output
# Use a base64-encoded configuration from the environment instead of a file
export ID_CONFIGURATOR_CONFIG=$(base64 -w0 config.yaml)
uv run id-configurator apply --dst ./output
id-configurator apply --dst ./output
# Download the configuration from an internal HTTPS server instead of a local file
id-configurator apply \
--config-url https://config.internal.example/device.yaml \
--config-url-cacert-content "$(cat internal-ca.pem)" \
--dst ./output
# Validate a configuration without applying it
uv run id-configurator validate --config config.yaml
id-configurator validate --config config.yaml
# List the available modules
uv run id-configurator list-modules
id-configurator list-modules
# Same thing, via the Docker image instead of installing anything locally
docker run --rm \
-v "$(pwd)/config.yaml:/config.yaml:ro" \
-v "$(pwd)/output:/dst" \
interdiode/id-configurator:latest \
id-configurator apply --config /config.yaml --dst /dst
```
The tool can also be invoked as `python -m id_configurator`.
## Configuration format
The configuration is a JSON or YAML object:
The configuration is a JSON or YAML object. `uuid`, `organisation` and `description` are all optional
(identifying metadata only - nothing in the tool itself relies on any of them), as is every `configurations.*`
block - a module that finds nothing to do for it simply does not run. `version` is the configuration
*format*'s own schema version (currently `"1.0"`, and the default if omitted) - independent of the
`id-configurator` package version - so a future, incompatible format change can be told apart from this one:
```yaml
uuid: <UUID>
organisation: <string>
description: <string>
version: "1.0"
uuid: <UUID, optional>
organisation: <string, optional>
description: <string, optional>
configurations:
certificates:
- name: <string>
@@ -68,18 +134,20 @@ configurations:
uris: [<string>]
suites: [<string>]
components: [<string>]
keys:
- name: <string>
content: <GPG key>
signed-by: <ASCII-armored GPG public key> # or signed-by-file: <path already on the target>
uri-to-replace: <fnmatch pattern, optional> # see "Replacing an existing repository" below
apk:
repositories:
- url: <string>
key: <RSA public key>
- name: <string>
url: <string>
key: <RSA public key, optional>
url_to_replace: <fnmatch pattern, optional>
yum:
repositories:
- name: <string>
index-url: <string>
gpg-url: <string>
gpg-url: <string, optional>
index-url-to-replace: <fnmatch pattern, optional>
keys:
- name: <string>
content: <GPG key>
@@ -91,20 +159,61 @@ configurations:
repositories:
- name: <string>
url: <string>
php:
repositories:
- name: <string>
url: <string>
npm:
repositories:
- name: <string>
url: <string>
rust:
repositories:
- name: <string>
url: <string>
go:
repositories:
- name: <string>
url: <string>
ruby:
repositories:
- name: <string>
url: <string>
nuget:
repositories:
- name: <string>
url: <string>
```
### Replacing an existing repository
`apt.sources[].uri-to-replace`, `apk.repositories[].url_to_replace` and `yum.repositories[].index-url-to-replace`
are optional [`fnmatch`](https://docs.python.org/3/library/fnmatch.html) patterns. When set, the configured
repository is only written if the pattern matches at least one URI/URL already present in `--src`'s existing
configuration - and only the matching existing entries are disabled, the rest are left untouched. This lets a
configuration safely target "whatever machine this is applied to": e.g. `uri-to-replace: "http://deb.debian.org/*"`
replaces Debian's default mirror if (and only if) it's actually there, without disturbing any other source a
given machine might already have. Leaving the field unset never disables anything - the configured repository
is simply added alongside whatever is already present.
## Modules
| Module | Applies to |
|----------------|--------------------------------------------------------------------------------------------------------|
| `certificates` | Installs root certificates into the system's CA store (`update-ca-certificates`). |
| `apt` | Disables existing APT sources and adds the configured ones, in DEB822 format. |
| `apk` | Disables existing Alpine `/etc/apk/repositories` entries and adds the configured ones, plus their keys.|
| `yum` | Disables existing YUM/DNF `.repo` files, adds the configured ones, and imports their GPG keys. |
|----------------|-------------------------------------------------------------------------------------------------------------------------------------------------------|
| `certificates` | Installs root certificates into the system's CA store, rebuilding the trust bundle in pure Python (Debian/Ubuntu and RHEL/Fedora trust stores). |
| `apt` | Adds the configured APT sources (DEB822 format); disables only the existing entries matching `uri-to-replace`. |
| `apk` | Adds the configured Alpine `/etc/apk/repositories` entries and their keys; disables only the existing lines matching `url_to_replace`. |
| `yum` | Adds the configured YUM/DNF `.repo` files and imports their GPG keys; disables only the existing sections matching `index-url-to-replace`. Covers DNF too - same config files. |
| `python` | Points pip, uv, poetry and `easy_install` to the configured Python package indexes. |
| `java` | Adds root certificates to every detected JKS/PKCS12 keystore, and configures Maven, Gradle, Ivy, SBT and Leiningen repositories. |
| `php` | Registers the configured repositories with Composer; sets `openssl.cafile` whenever any certificate is configured, so PHP's own OpenSSL calls trust it too. |
| `npm` | Points npm at the configured registries and its own `cafile`; sets `NODE_EXTRA_CA_CERTS` system-wide whenever any certificate is configured, since Node ignores the system CA store. |
| `rust` | Points cargo at the configured registries, replacing crates.io. |
| `go` | Points the Go toolchain at the configured module proxies (`GOPROXY`), replacing proxy.golang.org, and disables `GOSUMDB`. |
| `ruby` | Points RubyGems at the configured sources, replacing rubygems.org. |
| `nuget` | Points NuGet at the configured package sources, replacing nuget.org. |
More modules (npm, nuget, ruby/gem, rust/crates, go, packagist/composer, ...) are planned; see `CLAUDE.md` for the full roadmap.
This tool's own modular structure is meant to be extended: see `CLAUDE.md` for the module interface and how to add one.
## Development
@@ -118,4 +227,4 @@ uv run pre-commit run --all-files # run all pre-commit hooks
## License
Proprietary. See [LICENSE](LICENSE). Copyright InterDiode SA.
[CeCILL v2.1](https://cecill.info) (GPL-compatible free software license). See [LICENSE](LICENSE). Copyright InterDiode SA.
tox.ini
+4 -1
View File
@@ -74,15 +74,18 @@ description =
The version is taken from the pyproject.toml file.
Make sure that you have the rights to upload the image to Docker Hub.
commands =
python3 -m uv sync --all-groups --active
python3 -m uv sync --all-groups -U --active
python3 -m id_tools build docker --platform linux/amd64,linux/arm64/v8 --push
python3 -m id_tools publication sign-docker
python3 -m uv build
python3 -m uv publish
docker =
passenv =
COSIGN_PASSWORD
COSIGN_PRIVATE_KEY
DOCKER_HUB_LOGIN
DOCKER_HUB_TOKEN
UV_PUBLISH_TOKEN
[docker:pypi-repo]
dockerfile = {tox_root}/tests/dockerfiles/repos/pypi-repo.Dockerfile