Compare commits

..

48 Commits

Author SHA1 Message Date
d9pouces bf56cdfcd2 feat: can specify several users to configure
Signed-off-by: d9pouces <github@19pouces.net>
2026-10-05 20:46:38 +02:00
d9pouces c498d42da0 ci: version bump
Signed-off-by: d9pouces <github@19pouces.net>
2026-10-04 14:55:09 +02:00
d9pouces 990180dbba feat: add key_filename support for APK repositories
Signed-off-by: d9pouces <github@19pouces.net>
2026-10-04 14:54:54 +02:00
d9pouces 0fb964c2c2 fix: add missing GPG packages to the Docker image
Signed-off-by: d9pouces <github@19pouces.net>
2026-10-03 11:23:43 +02:00
d9pouces 5399336c2c feat: update version to 2026.10.2 in pyproject.toml
Signed-off-by: d9pouces <github@19pouces.net>
2026-10-02 23:20:22 +02:00
d9pouces 1c3aa1ef7d feat: update version to 2026.10.2 in pyproject.toml
Signed-off-by: d9pouces <github@19pouces.net>
2026-10-02 23:17:57 +02:00
d9pouces 170dd577d7 feat: update version to 2026.10.2 in pyproject.toml
Signed-off-by: d9pouces <github@19pouces.net>
2026-10-02 23:11:12 +02:00
d9pouces 11e7d93d85 feat: add note on generating configuration using InterDiode
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-21 23:22:25 +02:00
d9pouces 357d2392eb feat: add note on generating configuration using InterDiode
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-21 23:13:33 +02:00
d9pouces d7ef22b527 feat: add note on generating configuration using InterDiode
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-20 22:49:28 +02:00
d9pouces 39fdbb7f40 feat: add GitHub Actions workflow for publishing Python packages and Docker image
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-20 22:37:23 +02:00
d9pouces 7ddd3ac09b feat: update license to CECILL-2.1 and increment version to 2026.9.21 in pyproject.toml
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-20 22:37:06 +02:00
d9pouces 204d648477 feat: add --config-url-insecure option to disable TLS certificate verification
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-20 22:36:40 +02:00
d9pouces eff9ca66d2 feat: add publication command to main application
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-20 22:36:21 +02:00
d9pouces 776cdc3b17 feat: update version to 2026.9.20 in pyproject.toml
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-20 20:11:26 +02:00
d9pouces db4ff3cfd4 feat: add support for certificate configuration in npm and php modules, update repository URLs in configuration files
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-20 20:10:52 +02:00
d9pouces cea45c97ff feat: enhance repository configuration to support url_to_replace patterns for disabling existing entries
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-19 23:40:35 +02:00
d9pouces b2dcae0daf feat: update APT module to support inline ASCII-armored GPG keys in Signed-By field
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-10 22:16:42 +02:00
d9pouces be91425355 feat: sanitize filenames in apt and yum modules to prevent path traversal
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-09 23:18:24 +02:00
d9pouces 4409c38579 feat: update version to 2026.9.7 in pyproject.toml
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-09 22:20:16 +02:00
d9pouces 380e976f76 feat: update version to 2026.9.7 in pyproject.toml
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-07 08:26:40 +02:00
d9pouces ab2a179a55 feat: update config-url configuration with Test CA certificate and Dockerfile enhancements
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-07 08:12:48 +02:00
d9pouces 9de98d4744 feat: add config-url support with Dockerfile and NGINX configuration
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-07 07:15:58 +02:00
d9pouces 88380b6ab6 refactor: remove unused NPM_TO_COPY variable from build.py
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-06 23:08:58 +02:00
d9pouces e3b2e8a95a refactor: remove unused NPM_TO_COPY variable from build.py
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-06 22:22:51 +02:00
d9pouces 4e4b0592c0 refactor: remove unused NPM_TO_COPY variable from build.py
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-06 21:36:01 +02:00
d9pouces a8143b61b1 feat: add support for Go, Ruby, and NuGet modules with integration testing configurations
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-06 21:14:18 +02:00
d9pouces bcffe11bff feat: update tox configuration for multi-platform packaging and enhance testing environments
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-06 15:20:11 +02:00
d9pouces eaf04602aa feat: update tox configuration for multi-platform packaging and enhance testing environments
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-06 12:26:34 +02:00
d9pouces 4b0baf1a3c feat: update tox configuration for multi-platform packaging and enhance testing environments
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-06 12:23:25 +02:00
d9pouces b051730bd8 feat: update tox configuration for multi-platform packaging and enhance testing environments
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-06 10:48:58 +02:00
d9pouces cb3313bf65 feat: add minimal testing workflow and enhance coverage requirements in tox configuration
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-06 10:45:28 +02:00
d9pouces 6b6bc541dd feat: add Python-alpine Dockerfile for enhanced build process and testing with id-configurator
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-06 10:13:38 +02:00
d9pouces fdb20fce81 feat: add Python-alpine Dockerfile for enhanced build process and testing with id-configurator
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-06 08:29:51 +02:00
d9pouces 42e15a2d7e feat: enhance build process and Dockerfiles for improved artifact handling and dependency resolution
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-05 23:16:49 +02:00
d9pouces 438fed1e96 feat: enhance Certificate and certificate-repo Dockerfiles to support git operations and OpenSSL verification
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-05 22:51:40 +02:00
d9pouces 5caa5e601c feat: refactor Dockerfiles to use 'target' stage and add Certificate module configuration
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-05 22:31:27 +02:00
d9pouces f38a3d1ace feat: refactor Dockerfiles to use 'target' stage and add Certificate module configuration
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-05 22:31:12 +02:00
d9pouces a5a36fc7cd feat: update Dockerfiles and configuration for TLS support across multiple modules
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-05 22:23:34 +02:00
d9pouces fc41dfaa07 feat: add new configuration files for Apt, Apk, Java, and Yum
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-04 22:53:35 +02:00
d9pouces 6e086b0964 feat: add new configuration files for Apt, Apk, Java, and Yum
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-04 22:53:13 +02:00
d9pouces 22367224d7 feat: enhance Docker image tagging and add publication functionality
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-04 21:58:28 +02:00
d9pouces 26fcdd08e4 chore: update .gitignore to remove unnecessary entries
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-04 21:15:22 +02:00
d9pouces a8435e9c96 ci: new regression test for the Python module
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-04 19:20:41 +02:00
d9pouces 2c2b4fc340 feat: new configurations for APK/Java/Python/Yum
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-03 23:00:41 +02:00
d9pouces 35c3437343 feat: new configurations for APK/Java/Python/Yum
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-03 22:56:28 +02:00
d9pouces 586f091eeb feat: new configurations for APK/Java/Python/Yum
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-02 06:58:20 +02:00
d9pouces 48ec887808 feat: initial commit 2026-08-28 16:52:07 +02:00
127 changed files with 11997 additions and 685 deletions
+31
View File
@@ -0,0 +1,31 @@
name: Minimal testing, run on push to master & dev
on:
push:
branches:
- master
- dev
schedule:
- cron: 0 0 * * 1
jobs:
tox:
name: Python 3${{ matrix.python-version }}
runs-on: ubuntu-26.04
steps:
- uses: actions/checkout@v7.0.1
with:
lfs: true
- uses: actions/setup-python@v7.0.0
with:
python-version: 3.${{ matrix.python-version }}
- run: curl -sfL https://raw.githubusercontent.com/aquasecurity/trivy/main/contrib/install.sh | sudo sh -s -- -b /usr/local/bin v0.75.0
- run: curl -O -L "https://github.com/sigstore/cosign/releases/latest/download/cosign_3.1.3_amd64.deb"
- run: sudo dpkg -i cosign_3.1.3_amd64.deb
- run: python3 -m pip install --upgrade tox tox-docker-id tox-uv
- run: tox -e packaging-amd64,py313,py314,coverage
- run: tox -m end-to-end
- uses: docker/setup-buildx-action@v3
strategy:
matrix:
python-version: ['14']
timeout-minutes: 75
+34
View File
@@ -0,0 +1,34 @@
name: Publish Python packages and Docker image
on:
push:
branches:
- master
jobs:
publish:
env:
COSIGN_PASSWORD: ${{ secrets.COSIGN_PASSWORD }}
COSIGN_PRIVATE_KEY: ${{ secrets.COSIGN_PRIVATE_KEY }}
DOCKER_HUB_LOGIN: ${{ secrets.DOCKER_HUB_LOGIN }}
DOCKER_HUB_TOKEN: ${{ secrets.DOCKER_HUB_TOKEN }}
UV_PUBLISH_TOKEN: ${{ secrets.UV_PUBLISH_TOKEN }}
name: Package and public, Python 3${{ matrix.python-version }}
runs-on: ubuntu-26.04
steps:
- uses: actions/checkout@v7.0.1
with:
lfs: true
- uses: actions/setup-python@v7.0.0
with:
python-version: 3.${{ matrix.python-version }}
- run: curl -sfL https://raw.githubusercontent.com/aquasecurity/trivy/main/contrib/install.sh | sudo sh -s -- -b /usr/local/bin v0.75.0
- run: curl -O -L "https://github.com/sigstore/cosign/releases/latest/download/cosign_3.1.3_amd64.deb"
- run: sudo dpkg -i cosign_3.1.3_amd64.deb
- run: python3 -m pip install --upgrade tox tox-docker-id tox-uv
- run: tox -e packaging-amd64,py313,py314,coverage
- run: tox -m end-to-end
- uses: docker/setup-buildx-action@v3
- run: tox -e publish
strategy:
matrix:
python-version: ['14']
timeout-minutes: 75
-9
View File
@@ -62,7 +62,6 @@ cover/
# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal
@@ -166,15 +165,7 @@ cython_debug/
# macOS specific files
.DS_Store
# node
npm/node_modules
data/
doc/_static/videos
_trial_marker
.idea/
.docker
all_settings.py
!id_tests/*/data/**
id_tests/*/data/**/__pycache__/
.aider*
.claude
+1 -1
View File
@@ -77,7 +77,7 @@ repos:
rev: 1.8.6
hooks:
- id: bandit
args: [-s, 'B404,B405,B506,B703,B308,B603']
args: [-s, 'B101,B105,B404,B405,B506,B703,B308,B603,B604']
exclude: ^(id_tests/.*|npm/node_modules/.*|tools/.*|doc/.*|interdiode/static/vendor/.*)
- repo: https://github.com/shellcheck-py/shellcheck-py
rev: v0.11.0.1
+93
View File
@@ -0,0 +1,93 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project overview
`id-configurator` is a proprietary CLI tool by InterDiode to "configure a device in an air-gapped environment".
It targets Python 3.13+ and is packaged with the `uv_build` backend.
It takes a configuration (JSON or YAML, either from a file or from a base64-encoded environment variable) and applies it to the system, including installing root certificates, configuring package managers, and other system-level settings.
Modified files are written to a destination directory given with --dst by the user, this directory can be used to build a Docker image.
The original file system can be provided with --src by the user to detect existing configuration.
Example:
- the configuration specifies a root certificate to install, the tool will copy it to `<dst>/etc/ssl/certs` and update the system's CA certificates.
- the configuration specifies a apt source to add and `<src>` contains an existing `/etc/apt/sources.list`, the tool will merge the new source into the existing sources and write the result to `<dst>/etc/apt/sources.list`.
The configuration is expected to be a JSON or YAML object with the following structure:
```yaml
uuid: <UUID>
organisation: <string>
description: <string>
configurations:
certificates:
- name: <string>
content: <PEM>
apt:
sources:
- name: <string>
content: <string>
keys:
- name: <string>
content: <GPG key>
yum:
repositories:
- name: <string>
index-url: <string>
gpg-url: <string>
python:
repositories:
- name: <string>
index-url: <string>
```
The tool must have an extensible modular structure. Each module can use any part of the configuration, detect if is relevant by checking the current filesystem from `<src>`, and write modified files to `<dst>`.
Several modules can read the certificates from the configuration and use them differently.
The tool must be able to run in a Docker container, and must not require any network access.
It is forbidden to write any file outside of `<dst>`.
The tool must be called by `python -m id_configurator`, or via the console script entry point `id-configurator`.
Here is a preliminary list of modules that will be implemented:
- install root certificates in the system's CA store
- configure system package managers:
- apt
- yum
- apk
- dnf
- configure programming language package managers:
- python (pip)
- ruby (gem)
- nodejs (npm)
- golang (go)
- java (maven, gradle)
- php (composer)
-
## Current state
This repository is a skeleton: `id_configurator/__init__.py` is currently empty, and `README.md` is empty. `pyproject.toml` already declares a console-script entry point:
```
id-configurator = "id_configurator:main"
```
There is no `main` callable yet — implementing it in `id_configurator/__init__.py` (or importing it there) is the first thing needed to make the CLI runnable. No tests, no `[tool.ruff]`/`[tool.pytest]` config, and no CI config exist yet.
## Commands
- Install/sync dependencies: `uv sync`
- Run the CLI (once `main` exists): `uv run id-configurator`
- Lint and auto-fix: `uv run ruff check --fix .`
- Format: `uv run ruff format .`
- Run all pre-commit hooks (ruff, bandit, shellcheck, commitizen, check-yaml/toml/json, etc.): `uv run pre-commit run --all-files`
There is no test suite yet.
## Conventions
- Python version is pinned to 3.13 (`.python-version`, `requires-python = ">=3.13"` in `pyproject.toml`).
- `.editorconfig`: 4-space indentation, single quotes, LF line endings, 120 max line length for `.py` files.
- Commit messages are linted by commitizen via pre-commit, implying Conventional Commits format.
- The package is proprietary (`Private :: Do Not Upload` classifier) and is not published to PyPI.
+771 -669
View File
File diff suppressed because it is too large Load Diff
+234
View File
@@ -0,0 +1,234 @@
# id-configurator
`id-configurator` is a CLI tool by [InterDiode](https://interdiode.fr/) that configures a device (typically a
Docker image) in an **air-gapped environment**: no network access is required or ever used.
It reads a single configuration (JSON or YAML), and applies it to a filesystem tree, writing only the files
that need to change: root certificates, system package manager sources (APT, YUM/DNF, APK) and
language-specific package manager settings (pip/uv/poetry, Maven/Gradle/Ivy/SBT/Leiningen, Composer, npm,
cargo, Go modules, RubyGems, NuGet). The result can then be used to build a Docker image or otherwise
deployed to the target device.
## How it works
- `--dst` points to the destination directory that receives every modified or created file. `id-configurator`
never writes anywhere else.
- `--src` (optional) points to an existing filesystem, used to detect the current configuration (existing
APT sources, existing Java keystores, etc.) so it can be extended or disabled instead of blindly overwritten.
- The configuration is loaded from a file (`--config`), from a base64-encoded string, typically an
environment variable (`--config-env` / `$ID_CONFIGURATOR_CONFIG`), or downloaded over HTTP(S) from a URL
(`--config-url`). `--config-url-cacert-content` optionally gives the PEM content of the CA certificate to trust for that one download, for an internal
endpoint signed by a private CA the running system doesn't otherwise trust yet (e.g. before the
`certificates` module has ever run) - it defaults to the system's own trust store otherwise.
`--config-url-insecure` disables certificate (and hostname) verification entirely instead, overriding
`--config-url-cacert-content` - **insecure**, only meant for a first bootstrap against a server whose
certificate isn't trustable, or trusted, yet by any other means.
- On `apply`, every module inspects the configuration and the source filesystem and decides for itself whether it has something to
do; only relevant modules touch the destination directory. For example, the configured Alpine repositories are only written if
`--src` can be identified as an Alpine filesystem (i.e. it has an `/etc/apk` directory); and a source cloning Debian's official
repository is only configured if that official repository is actually present in the filesystem given via `--src`
(via `uri-to-replace`, see "Replacing an existing repository" below).
## Installation
### As a CLI tool
```bash
# with pip
pip install id-configurator
# or with uv tool, in its own isolated environment
uv tool install id-configurator
```
Once installed this way, run it directly as `id-configurator` (rather than `uv run id-configurator`, which is
only for the "from source" setup below).
### As a Docker image
`id-configurator` is also published as [`interdiode/id-configurator`](https://hub.docker.com/r/interdiode/id-configurator)
on Docker Hub - handy since its output is typically meant to become another Docker image in the first place.
One-off use:
```bash
docker run --rm -v "$(pwd)/config.yaml:/config.yaml:ro" -v "$(pwd)/output:/dst" -v "/:/src" \
interdiode/id-configurator:latest \
id-configurator apply --config /config.yaml --dst /dst --src /src
```
More typically, as a builder stage of the target image's own `Dockerfile` - `<src>` here is the target image
itself, taken as of its `target` stage, before `id-configurator`'s own changes are copied back on top of it:
```dockerfile
FROM debian:trixie-slim AS target
FROM interdiode/id-configurator:latest AS configuration
COPY --from=target / /src/
COPY config.yaml /config.yaml
RUN id-configurator apply --config /config.yaml --src /src --dst /dst
FROM target
COPY --from=configuration /dst/ /
```
### From source (development)
```bash
uv sync
```
## Usage
```bash
# Apply a configuration file to a destination directory
id-configurator apply --config config.yaml --dst ./output
# Take an existing filesystem into account (e.g. to extend its APT sources)
id-configurator apply --config config.yaml --src /path/to/rootfs --dst ./output
# Use a base64-encoded configuration from the environment instead of a file
export ID_CONFIGURATOR_CONFIG=$(base64 -w0 config.yaml)
id-configurator apply --dst ./output
# Download the configuration from an internal HTTPS server instead of a local file
id-configurator apply \
--config-url https://config.internal.example/device.yaml \
--config-url-cacert-content "$(cat internal-ca.pem)" \
--dst ./output
# Validate a configuration without applying it
id-configurator validate --config config.yaml
# List the available modules
id-configurator list-modules
# Same thing, via the Docker image instead of installing anything locally
docker run --rm \
-v "$(pwd)/config.yaml:/config.yaml:ro" \
-v "$(pwd)/input:/src" \
-v "$(pwd)/output:/dst" \
interdiode/id-configurator:latest \
id-configurator apply --config /config.yaml --dst /dst --src /src
```
The tool can also be invoked as `python -m id_configurator`.
## Configuration format
The simplest way to generate a new configuration is [InterDiode](https://interdiode.fr) itself, through its
simple interface - it also handles duplicating application mirrors onto disconnected networks.
The configuration is a JSON or YAML object. `uuid`, `organisation` and `description` are all optional
(identifying metadata only - nothing in the tool itself relies on any of them), as is every `configurations.*`
block - a module that finds nothing to do for it simply does not run. `version` is the configuration
*format*'s own schema version (currently `"1.0"`, and the default if omitted) - independent of the
`id-configurator` package version - so a future, incompatible format change can be told apart from this one:
```yaml
version: "1.0"
uuid: <UUID, optional>
organisation: <string, optional>
description: <string, optional>
configurations:
certificates:
- name: <string>
content: <PEM>
apt:
sources:
- name: <string>
uris: [<string>]
suites: [<string>]
components: [<string>]
signed-by: <ASCII-armored GPG public key> # or signed-by-file: <path already on the target>
uri-to-replace: <fnmatch pattern, optional> # see "Replacing an existing repository" below
apk:
repositories:
- name: <string>
url: <string>
key: <RSA public key, optional>
url_to_replace: <fnmatch pattern, optional>
yum:
repositories:
- name: <string>
index-url: <string>
gpg-url: <string, optional>
index-url-to-replace: <fnmatch pattern, optional>
keys:
- name: <string>
content: <GPG key>
python:
repositories:
- name: <string>
index-url: <string>
java:
repositories:
- name: <string>
url: <string>
php:
repositories:
- name: <string>
url: <string>
npm:
repositories:
- name: <string>
url: <string>
rust:
repositories:
- name: <string>
url: <string>
go:
repositories:
- name: <string>
url: <string>
ruby:
repositories:
- name: <string>
url: <string>
nuget:
repositories:
- name: <string>
url: <string>
```
### Replacing an existing repository
`apt.sources[].uri-to-replace`, `apk.repositories[].url_to_replace` and `yum.repositories[].index-url-to-replace`
are optional [`fnmatch`](https://docs.python.org/3/library/fnmatch.html) patterns. When set, the configured
repository is only written if the pattern matches at least one URI/URL already present in `--src`'s existing
configuration - and only the matching existing entries are disabled, the rest are left untouched. This lets a
configuration safely target "whatever machine this is applied to": e.g. `uri-to-replace: "http://deb.debian.org/*"`
replaces Debian's default mirror if (and only if) it's actually there, without disturbing any other source a
given machine might already have. Leaving the field unset never disables anything - the configured repository
is simply added alongside whatever is already present.
## Modules
| Module | Applies to |
|----------------|-------------------------------------------------------------------------------------------------------------------------------------------------------|
| `certificates` | Installs root certificates into the system's CA store, rebuilding the trust bundle in pure Python (Debian/Ubuntu and RHEL/Fedora trust stores). |
| `apt` | Adds the configured APT sources (DEB822 format); disables only the existing entries matching `uri-to-replace`. |
| `apk` | Adds the configured Alpine `/etc/apk/repositories` entries and their keys; disables only the existing lines matching `url_to_replace`. |
| `yum` | Adds the configured YUM/DNF `.repo` files and imports their GPG keys; disables only the existing sections matching `index-url-to-replace`. Covers DNF too - same config files. |
| `python` | Points pip, uv, poetry and `easy_install` to the configured Python package indexes. |
| `java` | Adds root certificates to every detected JKS/PKCS12 keystore, and configures Maven, Gradle, Ivy, SBT and Leiningen repositories. |
| `php` | Registers the configured repositories with Composer; sets `openssl.cafile` whenever any certificate is configured, so PHP's own OpenSSL calls trust it too. |
| `npm` | Points npm at the configured registries and its own `cafile`; sets `NODE_EXTRA_CA_CERTS` system-wide whenever any certificate is configured, since Node ignores the system CA store. |
| `rust` | Points cargo at the configured registries, replacing crates.io. |
| `go` | Points the Go toolchain at the configured module proxies (`GOPROXY`), replacing proxy.golang.org, and disables `GOSUMDB`. |
| `ruby` | Points RubyGems at the configured sources, replacing rubygems.org. |
| `nuget` | Points NuGet at the configured package sources, replacing nuget.org. |
This tool's own modular structure is meant to be extended: see `CLAUDE.md` for the module interface and how to add one.
## Development
```bash
uv sync # install dependencies
uv run pytest # run the test suite
uv run ruff check --fix . # lint
uv run ruff format . # format
uv run pre-commit run --all-files # run all pre-commit hooks
```
## License
[CeCILL v2.1](https://cecill.info) (GPL-compatible free software license). See [LICENSE](LICENSE). Copyright InterDiode SA.
+3
View File
@@ -0,0 +1,3 @@
from id_configurator.cli import app, main
__all__ = ["app", "main"]
+3
View File
@@ -0,0 +1,3 @@
from id_configurator.cli import main
main(__name__)
+192
View File
@@ -0,0 +1,192 @@
"""Typer CLI for id-configurator."""
from __future__ import annotations
import logging
from pathlib import Path
from typing import Annotated
import typer
from id_configurator import modules # noqa: F401 (registers the built-in modules)
from id_configurator.config import load_configuration
from id_configurator.context import Context
from id_configurator.modules.base import all_modules
app = typer.Typer(
name="id-configurator",
help="Configure a device in an air-gapped environment.",
no_args_is_help=True,
)
logger = logging.getLogger("id_configurator")
ConfigFileOption = Annotated[
Path | None,
typer.Option(
"--config",
exists=True,
dir_okay=False,
help="Path to a JSON or YAML configuration file.",
),
]
ConfigEnvOption = Annotated[
str | None,
typer.Option(
"--config-env",
envvar="ID_CONFIGURATOR_CONFIG",
help="Base64-encoded JSON/YAML configuration.",
),
]
ConfigUrlOption = Annotated[
str | None,
typer.Option(
"--config-url",
help="URL to download a JSON or YAML configuration from.",
),
]
ConfigUrlCacertContentOption = Annotated[
str | None,
typer.Option(
"--config-url-cacert-content",
help=(
"PEM content of the CA certificate to trust when downloading "
"--config-url over HTTPS (defaults to the system's trust store)."
),
),
]
ConfigUrlInsecureOption = Annotated[
bool,
typer.Option(
"--config-url-insecure",
help=(
"Disable TLS certificate verification when downloading --config-url "
"(INSECURE: only for bootstrapping from a server whose certificate "
"isn't trustable yet; overrides --config-url-cacert-content)."
),
),
]
SrcOption = Annotated[
Path | None,
typer.Option(
"--src",
exists=True,
file_okay=False,
help="Existing filesystem used to detect the current configuration.",
),
]
DstOption = Annotated[
Path,
typer.Option(
"--dst",
file_okay=False,
help="Destination directory receiving the modified files.",
),
]
UserOption = Annotated[
list[str],
typer.Option(
"--user",
help="A user to prepare per-user configuration for (may be repeated).",
),
]
AutoUsersOption = Annotated[
bool,
typer.Option(
"--auto-users",
help=(
"Also prepare per-user configuration for every regular "
"(non-system) user found in <src>/etc/passwd: one with a login "
"shell (not nologin, sync or false) and a UID at or above "
"UID_MIN (from <src>/etc/login.defs, 1000 by default) - root is "
"always included regardless of its UID."
),
),
]
VerboseOption = Annotated[
bool, typer.Option("--verbose", "-v", help="Enable verbose (debug) logging.")
]
def _configure_logging(verbose: bool) -> None:
logging.basicConfig(
level=logging.DEBUG if verbose else logging.INFO,
format="%(levelname)s %(name)s: %(message)s",
)
def _load_configuration(
config: Path | None,
config_env: str | None,
config_url: str | None,
config_url_cacert_content: str | None,
config_url_insecure: bool = False,
):
if config is None and config_env is None and config_url is None:
raise typer.BadParameter(
"either --config, --config-env (or $ID_CONFIGURATOR_CONFIG) or "
"--config-url must be provided"
)
return load_configuration(
config, config_env, config_url, config_url_cacert_content, config_url_insecure
)
@app.command()
def apply(
dst: DstOption,
config: ConfigFileOption = None,
config_env: ConfigEnvOption = None,
config_url: ConfigUrlOption = None,
config_url_cacert_content: ConfigUrlCacertContentOption = None,
config_url_insecure: ConfigUrlInsecureOption = False,
src: SrcOption = None,
user: UserOption = [], # noqa: B006 (Typer needs a concrete default to detect --user as repeatable)
auto_users: AutoUsersOption = False,
verbose: VerboseOption = False,
) -> None:
"""Apply a configuration to the destination directory."""
_configure_logging(verbose)
configuration = _load_configuration(
config, config_env, config_url, config_url_cacert_content, config_url_insecure
)
dst.mkdir(parents=True, exist_ok=True)
context = Context(
config=configuration, dst=dst, src=src, users=user, auto_users=auto_users
)
for module_cls in all_modules():
module = module_cls()
if module.is_relevant(context):
logger.info("applying module %s", module.name)
module.apply(context)
else:
logger.debug("skipping module %s (not relevant)", module.name)
@app.command()
def validate(
config: ConfigFileOption = None,
config_env: ConfigEnvOption = None,
config_url: ConfigUrlOption = None,
config_url_cacert_content: ConfigUrlCacertContentOption = None,
config_url_insecure: ConfigUrlInsecureOption = False,
) -> None:
"""Validate a configuration file without applying it."""
configuration = _load_configuration(
config, config_env, config_url, config_url_cacert_content, config_url_insecure
)
uuid = configuration.uuid if configuration.uuid is not None else "<no uuid>"
organisation = configuration.organisation or "<no organisation>"
typer.echo(f"configuration {uuid} is valid (organisation: {organisation})")
@app.command("list-modules")
def list_modules_command() -> None:
"""List the available configuration modules."""
for module_cls in all_modules():
typer.echo(module_cls.name)
def main(name: str = "__main__") -> None:
if name == "__main__":
app()
+391
View File
@@ -0,0 +1,391 @@
"""Configuration schema and loading helpers for id-configurator."""
from __future__ import annotations
import base64
import json
import ssl
import urllib.request
from pathlib import Path
from uuid import UUID
import yaml
from pydantic import BaseModel, ConfigDict, Field, field_validator
_CONFIG_URL_TIMEOUT_SECONDS = 30
class NamedContent(BaseModel):
"""A named piece of content, generally base64-encoded (a certificate, a GPG key, ...)."""
name: str
content: str
def _yes_no(value: bool) -> str:
return "yes" if value else "no"
def _fold_deb822_multiline(field: str, value: str) -> str:
"""Render `field` as a DEB822 folded multi-line field (sources.list(5)):
each of `value`'s own lines indented by one space, with a truly blank
line represented as a lone `.` - exactly how an ASCII-armored GPG key
block is embedded directly in a `Signed-By` field, verified against a
real `apt-get update` run against a genuinely signed repository."""
folded = "\n".join(f" {line}" if line else " ." for line in value.splitlines())
return f"{field}:\n{folded}"
class AptSource(BaseModel):
"""A single APT source, in the DEB822 format (see sources.list(5)).
Two ways to tell APT which key to trust this source with:
- `signed_by`: the ASCII-armored (.asc) *content* of a GPG public key,
folded inline as a multi-line `Signed-By` field - no separate keyring
file, no `gpg` binary needed to apply this configuration.
- `signed_by_file`: the *path* to a keyring already expected to exist on
the target system, written to the rendered `Signed-By` field verbatim.
If both are set, `signed_by` takes precedence.
`uri_to_replace`, if set, is an fnmatch pattern: this source is only
applied if it matches at least one URI already present in the machine's
existing apt configuration; otherwise it is skipped entirely.
"""
model_config = ConfigDict(populate_by_name=True)
name: str
types: list[str] = Field(default_factory=lambda: ["deb"])
uris: list[str]
suites: list[str]
components: list[str] = Field(default_factory=list)
architectures: list[str] = Field(default_factory=list)
languages: list[str] = Field(default_factory=list)
targets: list[str] = Field(default_factory=list)
enabled: bool = True
trusted: bool | None = None
signed_by: str | None = Field(default=None, alias="signed-by")
signed_by_file: str | None = Field(default=None, alias="signed-by-file")
pdiffs: bool | None = None
by_hash: bool | None = Field(default=None, alias="by-hash")
allow_insecure: bool | None = Field(default=None, alias="allow-insecure")
allow_weak: bool | None = Field(default=None, alias="allow-weak")
allow_downgrade_to_insecure: bool | None = Field(
default=None, alias="allow-downgrade-to-insecure"
)
check_valid_until: bool | None = Field(default=None, alias="check-valid-until")
valid_until_min: str | None = Field(default=None, alias="valid-until-min")
valid_until_max: str | None = Field(default=None, alias="valid-until-max")
check_date: bool | None = Field(default=None, alias="check-date")
date_max_future: str | None = Field(default=None, alias="date-max-future")
description: str | None = None
uri_to_replace: str | None = Field(default=None, alias="uri-to-replace")
def to_deb822(
self,
signed_by_armored_key: str | None = None,
signed_by_path: str | None = None,
) -> str:
"""Render this source as a DEB822 stanza (as used in
/etc/apt/sources.list.d/*.sources).
Neither `self.signed_by` nor `self.signed_by_file` is ever rendered
directly - the caller (the `apt` module) resolves which one applies
and passes the result in: `signed_by_armored_key` is folded inline
as a multi-line `Signed-By` field (see `_fold_deb822_multiline`);
`signed_by_path` is written as a plain single-line `Signed-By`
field. If both are given, `signed_by_armored_key` takes precedence.
"""
lines = [
f"Types: {' '.join(self.types)}",
f"URIs: {' '.join(self.uris)}",
f"Suites: {' '.join(self.suites)}",
]
if self.components:
lines.append(f"Components: {' '.join(self.components)}")
if self.architectures:
lines.append(f"Architectures: {' '.join(self.architectures)}")
if self.languages:
lines.append(f"Languages: {' '.join(self.languages)}")
if self.targets:
lines.append(f"Targets: {' '.join(self.targets)}")
if signed_by_armored_key:
lines.append(_fold_deb822_multiline("Signed-By", signed_by_armored_key))
elif signed_by_path:
lines.append(f"Signed-By: {signed_by_path}")
if self.trusted is not None:
lines.append(f"Trusted: {_yes_no(self.trusted)}")
if self.pdiffs is not None:
lines.append(f"PDiffs: {_yes_no(self.pdiffs)}")
if self.by_hash is not None:
lines.append(f"By-Hash: {_yes_no(self.by_hash)}")
if self.allow_insecure is not None:
lines.append(f"Allow-Insecure: {_yes_no(self.allow_insecure)}")
if self.allow_weak is not None:
lines.append(f"Allow-Weak: {_yes_no(self.allow_weak)}")
if self.allow_downgrade_to_insecure is not None:
lines.append(
f"Allow-Downgrade-To-Insecure: {_yes_no(self.allow_downgrade_to_insecure)}"
)
if self.check_valid_until is not None:
lines.append(f"Check-Valid-Until: {_yes_no(self.check_valid_until)}")
if self.valid_until_min:
lines.append(f"Valid-Until-Min: {self.valid_until_min}")
if self.valid_until_max:
lines.append(f"Valid-Until-Max: {self.valid_until_max}")
if self.check_date is not None:
lines.append(f"Check-Date: {_yes_no(self.check_date)}")
if self.date_max_future:
lines.append(f"Date-Max-Future: {self.date_max_future}")
if self.description:
lines.append(f"Description: {self.description}")
lines.append(f"Enabled: {_yes_no(self.enabled)}")
return "\n".join(lines) + "\n"
class AptConfig(BaseModel):
sources: list[AptSource] = Field(default_factory=list)
class ApkRepository(BaseModel):
"""An Alpine APK repository (see /etc/apk/repositories), e.g. a URL ending in /main or /community.
`url_to_replace`, if set, is an fnmatch pattern: this repository is only
applied if it matches at least one URL already present in the machine's
existing apk configuration; otherwise it is skipped entirely.
`key_filename`, if set, names the file the `key` is written to under
`/etc/apk/keys/` instead of a name derived from the key's content. It
must be a single, valid filesystem path component (in particular, no
`/`); if it isn't, it is ignored and the derived name is used instead.
"""
name: str
url: str
key: str | None = None
key_filename: str | None = None
url_to_replace: str | None = None
class ApkConfig(BaseModel):
repositories: list[ApkRepository] = Field(default_factory=list)
class YumRepository(BaseModel):
"""A YUM/DNF repository (see /etc/yum.repos.d/*.repo).
`index_url_to_replace`, if set, is an fnmatch pattern: this repository is
only applied if it matches at least one baseurl already present in the
machine's existing yum/dnf configuration; otherwise it is skipped
entirely.
"""
model_config = ConfigDict(populate_by_name=True)
name: str
index_url: str = Field(alias="index-url")
gpg_url: str | None = Field(default=None, alias="gpg-url")
index_url_to_replace: str | None = Field(default=None, alias="index-url-to-replace")
class YumConfig(BaseModel):
repositories: list[YumRepository] = Field(default_factory=list)
keys: list[NamedContent] = Field(default_factory=list)
class PythonRepository(BaseModel):
model_config = ConfigDict(populate_by_name=True)
name: str
index_url: str = Field(alias="index-url")
class PythonConfig(BaseModel):
repositories: list[PythonRepository] = Field(default_factory=list)
class JavaRepository(BaseModel):
"""A Maven-compatible repository, shared by Maven, Gradle, Ivy, SBT and Leiningen."""
name: str
url: str
class JavaConfig(BaseModel):
repositories: list[JavaRepository] = Field(default_factory=list)
class PhpRepository(BaseModel):
"""A Composer repository (see the "repositories" key in composer.json)."""
name: str
url: str
class PhpConfig(BaseModel):
repositories: list[PhpRepository] = Field(default_factory=list)
class NpmRepository(BaseModel):
"""An npm registry. The first entry replaces the default registry; any
further ones are scoped to `name` (`@name:registry`), e.g. for a private
org's packages alongside the default public registry."""
name: str
url: str
class NpmConfig(BaseModel):
repositories: list[NpmRepository] = Field(default_factory=list)
class RustRepository(BaseModel):
"""A cargo registry (see the "source" tables in .cargo/config.toml).
`is_sparse`, if true, marks the registry as served over the sparse
HTTP protocol (its index is then declared as `sparse+<url>`) rather
than a git-based index.
"""
name: str
url: str
is_sparse: bool = False
class RustConfig(BaseModel):
repositories: list[RustRepository] = Field(default_factory=list)
class GoRepository(BaseModel):
"""A Go module proxy (see GOPROXY), served over the standard Go module
proxy protocol."""
name: str
url: str
class GoConfig(BaseModel):
repositories: list[GoRepository] = Field(default_factory=list)
class RubyRepository(BaseModel):
"""A RubyGems source (see the ":sources:" key in gemrc)."""
name: str
url: str
class RubyConfig(BaseModel):
repositories: list[RubyRepository] = Field(default_factory=list)
class NuGetRepository(BaseModel):
"""A NuGet package source (see the "packageSources" key in NuGet.Config)."""
name: str
url: str
class NuGetConfig(BaseModel):
repositories: list[NuGetRepository] = Field(default_factory=list)
class ConfigurationsBlock(BaseModel):
certificates: list[NamedContent] = Field(default_factory=list)
apt: AptConfig = Field(default_factory=AptConfig)
apk: ApkConfig = Field(default_factory=ApkConfig)
yum: YumConfig = Field(default_factory=YumConfig)
python: PythonConfig = Field(default_factory=PythonConfig)
java: JavaConfig = Field(default_factory=JavaConfig)
php: PhpConfig = Field(default_factory=PhpConfig)
npm: NpmConfig = Field(default_factory=NpmConfig)
rust: RustConfig = Field(default_factory=RustConfig)
go: GoConfig = Field(default_factory=GoConfig)
ruby: RubyConfig = Field(default_factory=RubyConfig)
nuget: NuGetConfig = Field(default_factory=NuGetConfig)
class Configuration(BaseModel):
"""`version` is the configuration format's own schema version (currently
"1.0"), independent of the id-configurator package version - it lets a
future, incompatible format change be told apart from this one. Omitting
it defaults to "1.0", the only version there currently is."""
version: str = "1.0"
uuid: UUID | None = None
organisation: str | None = None
description: str | None = None
configurations: ConfigurationsBlock = Field(default_factory=ConfigurationsBlock)
@field_validator("version", mode="before")
@classmethod
def _coerce_version_to_str(cls, value: object) -> object:
"""Accept a bare YAML number (`version: 1.0`) as well as a quoted
string (`version: "1.0"`): YAML parses the unquoted, natural-looking
form as a float, which would otherwise fail validation with a
confusing type error."""
if isinstance(value, (int, float)) and not isinstance(value, bool):
return str(value)
return value
def parse_configuration(raw: str) -> Configuration:
"""Parse a configuration from a JSON or YAML string."""
try:
data = json.loads(raw)
except json.JSONDecodeError:
data = yaml.safe_load(raw)
return Configuration.model_validate(data)
def _fetch_config_url(
url: str, cacert_content: str | None, insecure: bool = False
) -> str:
"""Download a JSON or YAML configuration from `url`.
If `cacert_content` is given, it - and only it, not the system's default
trust store - is used to verify an HTTPS connection: for an internal
endpoint signed by a private CA the running system doesn't otherwise
trust yet (e.g. before the `certificates` module has ever run).
If `insecure` is set instead, certificate verification (and hostname
checking) is disabled entirely - e.g. to bootstrap from a server whose
certificate isn't trusted, or trustable, at all yet. `cacert_content` is
then ignored: there would be nothing left to verify it against.
"""
if insecure:
# Explicitly requested by the operator via --config-url-insecure.
context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
context.check_hostname = False
context.verify_mode = ssl.CERT_NONE
else:
context = ssl.create_default_context(
cadata=cacert_content if cacert_content is not None else None
)
with urllib.request.urlopen( # nosec B310 (url is an operator-provided CLI argument)
url, timeout=_CONFIG_URL_TIMEOUT_SECONDS, context=context
) as response:
return response.read().decode("utf-8")
def load_configuration(
config_file: Path | None,
config_env: str | None,
config_url: str | None = None,
config_url_cacert_content: str | None = None,
config_url_insecure: bool = False,
) -> Configuration:
"""Load a configuration from a file, from a base64-encoded string
(typically an env var), or by downloading it from a URL."""
if config_file is not None:
raw = config_file.read_text(encoding="utf-8")
elif config_env is not None:
raw = base64.b64decode(config_env).decode("utf-8")
elif config_url is not None:
raw = _fetch_config_url(
config_url, config_url_cacert_content, config_url_insecure
)
else:
raise ValueError(
"either --config, --config-env or --config-url must be provided"
)
return parse_configuration(raw)
+115
View File
@@ -0,0 +1,115 @@
"""Execution context shared by all modules.
The context is responsible for enforcing that modules never write outside of
the destination directory, and gives read access to the (optional) source
filesystem used to detect the existing configuration.
"""
from __future__ import annotations
from collections.abc import Sequence
from pathlib import Path
from id_configurator.config import Configuration
from id_configurator.users import (
DEFAULT_UID_MIN,
PasswdEntry,
parse_login_defs_uid_min,
parse_passwd,
)
class Context:
def __init__(
self,
config: Configuration,
dst: Path,
src: Path | None = None,
users: Sequence[str] | None = None,
auto_users: bool = False,
) -> None:
self.config = config
self.dst = dst.resolve()
self.src = src.resolve() if src is not None else None
self._passwd_entries = self._load_passwd_entries()
self._uid_min = self._load_uid_min()
self.users = self._resolve_users(users or [], auto_users)
def _load_passwd_entries(self) -> dict[str, PasswdEntry]:
content = self.read_src_file("etc/passwd")
if content is None:
return {}
return {entry.name: entry for entry in parse_passwd(content.decode("utf-8"))}
def _load_uid_min(self) -> int:
content = self.read_src_file("etc/login.defs")
if content is None:
return DEFAULT_UID_MIN
uid_min = parse_login_defs_uid_min(content.decode("utf-8"))
return DEFAULT_UID_MIN if uid_min is None else uid_min
def _resolve_users(self, users: Sequence[str], auto_users: bool) -> list[str]:
resolved: list[str] = []
for user in users:
if user not in resolved:
resolved.append(user)
if auto_users:
for name, entry in self._passwd_entries.items():
if name not in resolved and entry.is_regular_user(self._uid_min):
resolved.append(name)
return resolved
def home_dir(self, user: str) -> str:
"""Relative path (no leading `/`) to `user`'s home directory, as used
by modules that write per-user configuration files.
Taken from `<src>`'s `/etc/passwd` entry for `user` when there is
one (the case for a user resolved via `--auto-users`); otherwise
falls back to the usual `/home/<user>` convention (`/root` for
root).
"""
entry = self._passwd_entries.get(user)
if entry is not None and entry.home:
return entry.home.lstrip("/")
return "root" if user == "root" else f"home/{user}"
def dst_path(self, relative: str) -> Path:
"""Resolve a path relative to the destination directory, refusing to escape it."""
path = (self.dst / relative.lstrip("/")).resolve()
if path != self.dst and self.dst not in path.parents:
raise ValueError(
f"refusing to write outside of the destination directory: {relative}"
)
return path
def src_path(self, relative: str) -> Path | None:
"""Resolve a path relative to the source directory, or None if no source was given."""
if self.src is None:
return None
return (self.src / relative.lstrip("/")).resolve()
def write_file(self, relative: str, content: bytes, mode: int = 0o644) -> Path:
path = self.dst_path(relative)
path.parent.mkdir(parents=True, exist_ok=True)
path.write_bytes(content)
path.chmod(mode)
return path
def read_src_file(self, relative: str) -> bytes | None:
path = self.src_path(relative)
if path is None or not path.is_file():
return None
return path.read_bytes()
def iter_src_dir(self, relative: str) -> list[Path]:
"""List the files present in a directory relative to the source filesystem, if any."""
path = self.src_path(relative)
if path is None or not path.is_dir():
return []
return sorted(entry for entry in path.iterdir() if entry.is_file())
def glob_src(self, pattern: str) -> list[Path]:
"""Glob a pattern relative to the source filesystem, or [] if no source was given."""
if self.src is None:
return []
return sorted(self.src.glob(pattern))
+34
View File
@@ -0,0 +1,34 @@
"""Configuration modules.
Importing this package registers every built-in module (see `base.register`).
"""
from id_configurator.modules import (
apk,
apt,
certificates,
go,
java,
npm,
nuget,
php,
python,
ruby,
rust,
yum,
)
__all__ = [
"apk",
"apt",
"certificates",
"go",
"java",
"npm",
"nuget",
"php",
"python",
"ruby",
"rust",
"yum",
]
+149
View File
@@ -0,0 +1,149 @@
"""Detect existing APK (Alpine) repositories and configure new ones.
If `<src>/etc/apk` exists, the existing `/etc/apk/repositories` file (if any)
is copied into `<dst>`, and the repositories declared in the configuration
are appended as new, active lines. An existing active line is only disabled
(commented out) if it matches the `url_to_replace` pattern of one of the
configured repositories; lines matching no such pattern are copied through
unchanged. Public keys provided alongside a repository are written to
`<dst>/etc/apk/keys/`, named after their content so that repositories
sharing the same key (a common case for official Alpine mirrors) only
produce a single key file.
A repository's `url_to_replace`, if set, is a fnmatch pattern checked
against the URLs already active in /etc/apk/repositories: the repository
is only written out if the pattern matches at least one of them, and
skipped entirely otherwise.
"""
from __future__ import annotations
import fnmatch
import hashlib
import logging
from collections.abc import Sequence
from pathlib import Path
from id_configurator.context import Context
from id_configurator.modules.base import Module, register
logger = logging.getLogger(__name__)
_APK_DIR = "etc/apk"
_REPOSITORIES = "etc/apk/repositories"
_KEYS_DIR = "etc/apk/keys"
def _disable_repositories_content(content: str, patterns: Sequence[str]) -> str:
lines = []
for line in content.splitlines():
stripped = line.strip()
if (
stripped
and not stripped.startswith("#")
and any(fnmatch.fnmatch(stripped, pattern) for pattern in patterns)
):
lines.append(f"# {line}")
else:
lines.append(line)
return "\n".join(lines) + "\n"
def _key_filename(content: bytes) -> str:
return f"{hashlib.sha256(content).hexdigest()[:16]}.rsa.pub"
def _is_valid_key_filename(name: str) -> bool:
"""Check that `name` is safe to use as a single filename under `_KEYS_DIR`.
Unlike `sanitize_filename`, an invalid name is not cleaned up: the
caller falls back to the content-derived name instead, since a
configured `key_filename` that isn't a plain file name (e.g. containing
`/` or `\\`, which would turn into extra path segments, or `.`/`..`)
most likely indicates a configuration mistake.
"""
return name not in ("", ".", "..") and "/" not in name and "\\" not in name
def _existing_urls(context: Context) -> set[str]:
"""Collect the URLs already declared in /etc/apk/repositories on the
source filesystem (one active, non-comment line per URL)."""
existing = context.read_src_file(_REPOSITORIES)
if existing is None:
return set()
return {
stripped
for line in existing.decode("utf-8").splitlines()
if (stripped := line.strip()) and not stripped.startswith("#")
}
@register
class ApkModule(Module):
name = "apk"
def is_relevant(self, context: Context) -> bool:
return self._src_apk_dir(context) is not None
@staticmethod
def _src_apk_dir(context: Context) -> Path | None:
path = context.src_path(_APK_DIR)
if path is not None and path.is_dir():
return path
return None
def apply(self, context: Context) -> None:
patterns = [
repository.url_to_replace
for repository in context.config.configurations.apk.repositories
if repository.url_to_replace is not None
]
disabled = self._disabled_existing_repositories(context, patterns)
configured = self._configured_repositories_lines(context)
content = disabled + ("\n".join(configured) + "\n" if configured else "")
if content:
context.write_file(_REPOSITORIES, content.encode("utf-8"))
@staticmethod
def _disabled_existing_repositories(
context: Context, patterns: Sequence[str]
) -> str:
existing = context.read_src_file(_REPOSITORIES)
if existing is None:
return ""
logger.info("copied existing apk repositories %s", _REPOSITORIES)
return _disable_repositories_content(existing.decode("utf-8"), patterns)
@staticmethod
def _configured_repositories_lines(context: Context) -> list[str]:
lines = []
existing_urls: set[str] | None = None
for repository in context.config.configurations.apk.repositories:
if repository.url_to_replace is not None:
if existing_urls is None:
existing_urls: set[str] = _existing_urls(context)
if not any(
fnmatch.fnmatch(url, repository.url_to_replace)
for url in existing_urls
):
logger.info(
"skipped apk repository %s: url-to-replace %s matched "
"no existing url",
repository.name,
repository.url_to_replace,
)
continue
lines.append(repository.url)
logger.info("added apk repository %s", repository.name)
if repository.key:
if repository.key_filename and _is_valid_key_filename(
repository.key_filename
):
filename = repository.key_filename
else:
filename = _key_filename(repository.key.encode("utf-8"))
context.write_file(
f"{_KEYS_DIR}/{filename}", repository.key.encode("utf-8")
)
logger.info("installed apk key %s", filename)
return lines
+213
View File
@@ -0,0 +1,213 @@
"""Detect existing APT sources and configure new ones, in the DEB822 format.
If `<src>/etc/apt` exists, any source file already present there (classic
one-line `.list` files or DEB822 `.sources` files, whether at the top level
or under `sources.list.d/`) is copied into `<dst>/etc/apt`, and the sources
declared in the configuration are added as new, enabled `.sources` files.
An existing entry (a single `deb`/`deb-src` line, or a DEB822 stanza) is only
disabled if its URI matches the `uri_to_replace` pattern of one of the
configured sources; entries matching no such pattern are copied through
unchanged, still enabled.
Each source can be trusted two ways: `signed_by` holds the ASCII-armored
content of a GPG public key, folded inline as a multi-line `Signed-By`
field (per sources.list(5) - verified empirically against a real,
genuinely signed repository, so no `gpg` binary or separate keyring file
is needed at all to apply this configuration). `signed_by_file` instead
names a keyring already expected to exist on the target system, used
verbatim as `Signed-By`; if both are given, `signed_by` takes precedence.
A source's `uri_to_replace`, if set, is an fnmatch pattern checked against
the URIs of the sources already present on the source filesystem: the
source is only written out if the pattern matches at least one of them,
and skipped entirely otherwise - even on an otherwise Debian-like machine.
"""
from __future__ import annotations
import fnmatch
import logging
from collections.abc import Sequence
from pathlib import Path
from id_configurator.context import Context
from id_configurator.modules.base import Module, register, sanitize_filename
logger = logging.getLogger(__name__)
_APT_DIR = "etc/apt"
_SOURCES_LIST = "etc/apt/sources.list"
_SOURCES_LIST_D = "etc/apt/sources.list.d"
def _uri_from_deb_line(stripped_line: str) -> str | None:
"""Extract the URI from a single, already-stripped `deb`/`deb-src`
line (`deb [options] uri suite [component...]`), or `None` if the line
is not such a line."""
parts = stripped_line.split()
if not parts or parts[0] not in ("deb", "deb-src"):
return None
parts = parts[1:]
while parts and parts[0].startswith("["):
while parts and not parts[0].endswith("]"):
parts.pop(0)
if parts:
parts.pop(0)
return parts[0] if parts else None
def _disable_list_content(content: str, patterns: Sequence[str]) -> str:
lines = []
for line in content.splitlines():
stripped = line.strip()
uri = (
None
if not stripped or stripped.startswith("#")
else _uri_from_deb_line(stripped)
)
if uri is not None and any(
fnmatch.fnmatch(uri, pattern) for pattern in patterns
):
lines.append(f"# {line}")
else:
lines.append(line)
return "\n".join(lines) + "\n"
def _uris_from_deb822_stanza(stanza: str) -> set[str]:
uris: set[str] = set()
for line in stanza.splitlines():
if line.lower().startswith("uris:"):
uris.update(line.split(":", 1)[1].split())
return uris
def _disable_deb822_content(content: str, patterns: Sequence[str]) -> str:
stanzas = []
for stanza in content.split("\n\n"):
if not stanza.strip():
continue
lines = stanza.splitlines()
uris = _uris_from_deb822_stanza(stanza)
if any(fnmatch.fnmatch(uri, pattern) for uri in uris for pattern in patterns):
lines = [line for line in lines if not line.lower().startswith("enabled:")]
lines.append("Enabled: no")
stanzas.append("\n".join(lines))
return "\n\n".join(stanzas) + "\n"
def _uris_from_list_content(content: str) -> set[str]:
"""Extract the URIs declared in a classic one-line `.list` file
(`deb [options] uri suite [component...]`), skipping comments and any
bracketed option list."""
uris: set[str] = set()
for line in content.splitlines():
stripped = line.strip()
if not stripped or stripped.startswith("#"):
continue
uri = _uri_from_deb_line(stripped)
if uri is not None:
uris.add(uri)
return uris
def _uris_from_deb822_content(content: str) -> set[str]:
"""Extract the URIs declared in the `URIs:` field of each stanza of a
DEB822 `.sources` file."""
uris: set[str] = set()
for stanza in content.split("\n\n"):
uris.update(_uris_from_deb822_stanza(stanza))
return uris
def _existing_uris(context: Context) -> set[str]:
"""Collect every URI declared by the apt sources already present on the
source filesystem, whether in the top-level sources.list or under
sources.list.d/, in either the classic or the DEB822 format."""
uris: set[str] = set()
existing_top_level = context.read_src_file(_SOURCES_LIST)
if existing_top_level is not None:
uris.update(_uris_from_list_content(existing_top_level.decode("utf-8")))
for entry in context.iter_src_dir(_SOURCES_LIST_D):
if entry.suffix == ".sources":
uris.update(_uris_from_deb822_content(entry.read_text(encoding="utf-8")))
elif entry.suffix == ".list":
uris.update(_uris_from_list_content(entry.read_text(encoding="utf-8")))
return uris
@register
class AptModule(Module):
name = "apt"
def is_relevant(self, context: Context) -> bool:
return self._src_apt_dir(context) is not None
@staticmethod
def _src_apt_dir(context: Context) -> Path | None:
path = context.src_path(_APT_DIR)
if path is not None and path.is_dir():
return path
return None
def apply(self, context: Context) -> None:
patterns = [
source.uri_to_replace
for source in context.config.configurations.apt.sources
if source.uri_to_replace is not None
]
self._copy_existing_disabled(context, patterns)
self._write_configured_sources(context)
@staticmethod
def _copy_existing_disabled(context: Context, patterns: Sequence[str]) -> None:
existing_top_level = context.read_src_file(_SOURCES_LIST)
if existing_top_level is not None:
disabled = _disable_list_content(
existing_top_level.decode("utf-8"), patterns
)
context.write_file(_SOURCES_LIST, disabled.encode("utf-8"))
logger.info("copied existing apt source list %s", _SOURCES_LIST)
for entry in context.iter_src_dir(_SOURCES_LIST_D):
if entry.suffix == ".sources":
disabled = _disable_deb822_content(
entry.read_text(encoding="utf-8"), patterns
)
elif entry.suffix == ".list":
disabled = _disable_list_content(
entry.read_text(encoding="utf-8"), patterns
)
else:
continue
context.write_file(
f"{_SOURCES_LIST_D}/{entry.name}", disabled.encode("utf-8")
)
logger.info("copied existing apt source %s", entry.name)
@staticmethod
def _write_configured_sources(context: Context) -> None:
existing_uris: set[str] | None = None
for source in context.config.configurations.apt.sources:
if source.uri_to_replace is not None:
if existing_uris is None:
existing_uris = _existing_uris(context)
if not any(
fnmatch.fnmatch(uri, source.uri_to_replace) for uri in existing_uris
):
logger.info(
"skipped apt source %s: uri-to-replace %s matched no "
"existing uri",
source.name,
source.uri_to_replace,
)
continue
context.write_file(
f"{_SOURCES_LIST_D}/{sanitize_filename(source.name)}.sources",
source.to_deb822(
signed_by_armored_key=source.signed_by,
signed_by_path=source.signed_by_file,
).encode("utf-8"),
)
logger.info("added apt source %s", source.name)
+51
View File
@@ -0,0 +1,51 @@
"""Base class and registry for configuration modules.
Each module inspects the configuration and the source filesystem (via the
`Context`) to decide whether it applies, then writes the resulting files to
the destination directory.
"""
from __future__ import annotations
from abc import ABC, abstractmethod
from id_configurator.context import Context
_REGISTRY: list[type[Module]] = []
class Module(ABC):
name: str
@abstractmethod
def is_relevant(self, context: Context) -> bool:
"""Return True if this module has something to do for the given configuration."""
@abstractmethod
def apply(self, context: Context) -> None:
"""Write the module's changes to the destination directory."""
def register(module_cls: type[Module]) -> type[Module]:
"""Class decorator registering a module so it is picked up by the CLI."""
_REGISTRY.append(module_cls)
return module_cls
def all_modules() -> list[type[Module]]:
return list(_REGISTRY)
def sanitize_filename(name: str) -> str:
"""Make a user-supplied name safe to use as a single filesystem path
component.
Modules that derive a file's name from configuration data (a
certificate's name, a repository's name, ...) must run it through this
first: without it, a name containing `/` (or `\\`, harmless on the
target systems this tool writes to but stripped for good measure)
would silently turn into extra, unintended path segments - creating
nested directories a careless or malicious name shouldn't be able to
create - instead of the single file the caller asked for.
"""
return name.replace("/", "_").replace("\\", "_")
+256
View File
@@ -0,0 +1,256 @@
# noinspection SpellCheckingInspection
"""Install root certificates into the system's CA store.
Certificates from the configuration are dropped into the "local" CA
directory, then the trust store under `/etc/ssl/certs` (bundle + named
symlinks) is rebuilt in pure Python, mirroring what Debian's
`update-ca-certificates` shell script does: every certificate enabled in
`ca-certificates.conf` (read from `<src>`), plus everything found under the
local certificates directory, is concatenated into a single bundle file and
gets a `<name>.pem` symlink pointing at it.
The OpenSSL subject-hash symlinks (`<hash>.0`) that `update-ca-certificates`
also creates, for CApath-style directory lookups, are intentionally not
reproduced here: none of this tool's own consumers (pip, apt, dnf, apk, the
JVM, ...) do CApath lookups, they all read the single bundle file.
If `/etc/ssl/certs` is itself a symlink in `<src>` (as on RHEL, where it
points to `/etc/pki/tls/certs`), the bundle and symlinks are written to
whatever directory it resolves to instead. The write lands where the
deployed system will actually look, without ever writing a plain directory
where that system expects a symlink. In that case `/usr/lib/ssl/cert.pem`
(see below) is not written either, since it isn't OpenSSL's default CAfile
on such systems in the first place.
Otherwise, `/usr/lib/ssl/cert.pem` - OpenSSL's own compiled-in default
CAfile on Debian/Ubuntu, and what apt's own https method reads absent any
explicit configuration - is symlinked to the rebuilt bundle too. This is
normally created by the ca-certificates package's postinst script; doing it
here as well means the bundle is trusted even on a base image that never
had that package installed.
When `<src>` also has a RHEL/Fedora-style trust store (`/etc/pki/ca-trust`),
the configured certificates are additionally dropped into
`/etc/pki/ca-trust/source/anchors` and merged into
`/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem`, the single file that
`/etc/pki/tls/certs/ca-bundle.crt` and OpenSSL's own default CAfile
(`/etc/pki/tls/cert.pem`) both resolve to on such systems - which is what
dnf/curl/openssl-based tools there actually read. This intentionally does
not reproduce the rest of `update-ca-trust extract`'s output (the OpenSSL
"trust" bundle with per-certificate usage flags, the email/objsign bundles,
the extracted Java keystore, ...); only the plain PEM trust bundle is
rebuilt.
"""
from __future__ import annotations
import logging
import posixpath
from pathlib import Path
from id_configurator.context import Context
from id_configurator.modules.base import Module, register, sanitize_filename
logger = logging.getLogger(__name__)
_LOCAL_CERTS_DIR = "usr/local/share/ca-certificates"
# noinspection SpellCheckingInspection
_CERTSCONF = "etc/ca-certificates.conf"
# noinspection SpellCheckingInspection
_CERTSDIR = "usr/share/ca-certificates"
# noinspection SpellCheckingInspection
_ETCCERTSDIR = "etc/ssl/certs"
_CERT_BUNDLE = "ca-certificates.crt"
_RHEL_ANCHORS_DIR = "etc/pki/ca-trust/source/anchors"
_RHEL_EXTRACTED_BUNDLE = "etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem"
# OpenSSL's own compiled-in default CAfile on Debian/Ubuntu (OPENSSLDIR is
# /usr/lib/ssl there); the ca-certificates package's postinst symlinks this
# to the bundle, which is what lets OpenSSL-linked tools (including apt's
# own https method) find it without any explicit CAfile configuration. That
# symlink only exists once the package has actually been installed, so it is
# created here too, making the bundle usable even on a base image that
# never had ca-certificates installed at all.
_OPENSSL_DEFAULT_CERT_FILE = "usr/lib/ssl/cert.pem"
def _pem_name(relative_crt: str) -> str:
# noinspection GrazieInspection
"""Mirror update-ca-certificates' `.pem` symlink naming for a `*.crt` path."""
stem = Path(relative_crt).name.removesuffix(".crt")
stem = stem.replace(" ", "_").replace("(", "=").replace(")", "=").replace(",", "_")
return f"{stem}.pem"
def _ensure_trailing_newline(content: bytes) -> bytes:
return content if content.endswith(b"\n") else content + b"\n"
def _resolve_target_dir(context: Context, relative: str) -> str:
"""Resolve `relative` against `<src>`, following it if it is a symlink.
Some systems replace a well-known directory with a symlink to another
location (e.g. RHEL's `/etc/ssl/certs` -> `/etc/pki/tls/certs`). Writing
to the literal path in that case would create a plain directory where
the deployed system expects a symlink - which BuildKit refuses to
overlay a directory onto in the first place - while leaving the actual,
symlinked-to location untouched. Returns `relative` unchanged if it
isn't a symlink in `<src>`, or if it can't be resolved back to a path
under `<src>`.
"""
if context.src is None:
return relative
# Not context.src_path(): it resolves symlinks away before we can even
# check for one.
source = context.src / relative
if not source.is_symlink():
return relative
target = source.readlink()
if target.is_absolute():
return posixpath.normpath(target.as_posix()).lstrip("/")
resolved = (source.parent / target).resolve()
try:
return resolved.relative_to(context.src.resolve()).as_posix()
except ValueError:
return relative
# noinspection SpellCheckingInspection
def _parse_certsconf(certsconf: Path) -> list[str]:
"""Return the paths (relative to CERTSDIR) enabled in ca-certificates.conf.
Blank lines and comments (`#`) are ignored; lines prefixed with `!`
are explicitly disabled, exactly like update-ca-certificates.
"""
enabled = []
for line in certsconf.read_text(encoding="utf-8").splitlines():
line = line.strip()
if not line or line.startswith("#") or line.startswith("!"):
continue
enabled.append(line)
return enabled
# noinspection SpellCheckingInspection
@register
class CertificatesModule(Module):
name = "certificates"
def is_relevant(self, context: Context) -> bool:
return bool(context.config.configurations.certificates)
def apply(self, context: Context) -> None:
configured_names = {
sanitize_filename(certificate.name)
for certificate in context.config.configurations.certificates
}
for existing in context.iter_src_dir(_LOCAL_CERTS_DIR):
if existing.stem not in configured_names:
context.write_file(
f"{_LOCAL_CERTS_DIR}/{existing.name}", existing.read_bytes()
)
for certificate in context.config.configurations.certificates:
context.write_file(
f"{_LOCAL_CERTS_DIR}/{sanitize_filename(certificate.name)}.crt",
certificate.content.encode(),
)
logger.info("installed root certificate %s", certificate.name)
self._update_ca_certificates_conf(context)
if self._rhel_trust_store_present(context):
self._update_rhel_trust_store(context, configured_names)
@staticmethod
def _rhel_trust_store_present(context: Context) -> bool:
anchors = context.src_path(_RHEL_ANCHORS_DIR)
return anchors is not None and anchors.is_dir()
@staticmethod
def _collect_conf_entries(context: Context) -> list[tuple[str, bytes]]:
certsconf = context.src_path(_CERTSCONF)
certsdir = context.src_path(_CERTSDIR)
if certsconf is None or not certsconf.is_file() or certsdir is None:
return []
entries = []
for relative in _parse_certsconf(certsconf):
source = certsdir / relative
if not source.is_file():
logger.warning("%s not found, but listed in %s", source, certsconf)
continue
entries.append((f"{_CERTSDIR}/{relative}", source.read_bytes()))
return entries
@staticmethod
def _collect_local_entries(localcertsdir: Path) -> list[tuple[str, bytes]]:
entries = []
for source in sorted(localcertsdir.rglob("*.crt")):
relative = source.relative_to(localcertsdir).as_posix()
entries.append((f"{_LOCAL_CERTS_DIR}/{relative}", source.read_bytes()))
return entries
@classmethod
def _update_ca_certificates_conf(cls, context: Context) -> None:
etccertsdir_relative = _resolve_target_dir(context, _ETCCERTSDIR)
etccertsdir = context.dst_path(etccertsdir_relative)
localcertsdir = context.dst_path(_LOCAL_CERTS_DIR)
etccertsdir.mkdir(parents=True, exist_ok=True)
localcertsdir.mkdir(parents=True, exist_ok=True)
# Same processing order as update-ca-certificates: ca-certificates.conf
# entries first, then everything under the local certificate directory.
entries = cls._collect_conf_entries(context) + cls._collect_local_entries(
localcertsdir
)
bundle = bytearray()
for relative, content in entries:
bundle += _ensure_trailing_newline(content)
pem_path = etccertsdir / _pem_name(relative)
pem_path.unlink(missing_ok=True)
pem_path.symlink_to(f"/{relative}")
context.write_file(f"{etccertsdir_relative}/{_CERT_BUNDLE}", bytes(bundle))
logger.info("rebuilt %s from %d certificate(s)", _CERT_BUNDLE, len(entries))
if etccertsdir_relative == _ETCCERTSDIR:
# Not context.dst_path(_OPENSSL_DEFAULT_CERT_FILE) directly: on a
# host that already has this symlink (e.g. re-applying to a live
# `--dst /`), resolving the leaf itself would follow it straight
# to the bundle we just wrote, and the unlink+symlink_to below
# would then target the bundle's own path instead of this one -
# replacing the bundle with a symlink to itself. Resolving only
# the containing directory keeps the leaf name literal.
cert_file_dir = context.dst_path(
posixpath.dirname(_OPENSSL_DEFAULT_CERT_FILE)
)
cert_file_dir.mkdir(parents=True, exist_ok=True)
cert_file = cert_file_dir / posixpath.basename(_OPENSSL_DEFAULT_CERT_FILE)
cert_file.unlink(missing_ok=True)
cert_file.symlink_to(f"/{etccertsdir_relative}/{_CERT_BUNDLE}")
@staticmethod
def _update_rhel_trust_store(context: Context, configured_names: set[str]) -> None:
for existing in context.iter_src_dir(_RHEL_ANCHORS_DIR):
if existing.stem not in configured_names:
context.write_file(
f"{_RHEL_ANCHORS_DIR}/{existing.name}", existing.read_bytes()
)
for certificate in context.config.configurations.certificates:
context.write_file(
f"{_RHEL_ANCHORS_DIR}/{sanitize_filename(certificate.name)}.crt",
certificate.content.encode(),
)
bundle = bytearray()
existing_bundle = context.read_src_file(_RHEL_EXTRACTED_BUNDLE)
if existing_bundle is not None:
bundle += _ensure_trailing_newline(existing_bundle)
# Every write above went through context.write_file(), which only ever
# creates regular files, so every entry here is a certificate to add.
anchors = sorted(context.dst_path(_RHEL_ANCHORS_DIR).iterdir())
for anchor in anchors:
bundle += _ensure_trailing_newline(anchor.read_bytes())
context.write_file(_RHEL_EXTRACTED_BUNDLE, bytes(bundle))
logger.info("rebuilt RHEL/Fedora trust store from %d anchor(s)", len(anchors))
+55
View File
@@ -0,0 +1,55 @@
"""Configure the Go toolchain to use the configured GoRepository entries as
its module proxy (GOPROXY), replacing the default (proxy.golang.org). Since
an air-gapped mirror won't be listed in the public checksum database and
sum.golang.org is unreachable anyway, GOSUMDB is also turned off - each
module's hash is then simply trusted on first use, exactly as it would be
for any other GOPROXY.
Go's own HTTPS client already trusts whatever the `certificates` module
installs into the system CA store, so no separate certificate configuration
is needed here, unlike npm's.
Written to `$GOROOT/go.env`, which `$GOROOT` defaults to `/usr/local/go` for
essentially every standard Go install (the official Docker image included);
unlike a config dropped in one specific user's home, this applies regardless
of which user invokes `go`. The go command reads `$GOROOT/go.env` as a
fallback whenever a setting is not already present in a higher-precedence
level - in particular the per-user `go env -w` file (found at
`os.UserConfigDir()/go/env`, e.g. `$HOME/.config/go/env`), which this module
also writes for each configured `--user` (`context.users`), so that one
takes precedence should that user ever run `go env -w` themselves.
"""
from __future__ import annotations
import logging
from id_configurator.config import GoRepository
from id_configurator.context import Context
from id_configurator.modules.base import Module, register
logger = logging.getLogger(__name__)
_GO_ENV = "usr/local/go/go.env"
def _write_go_env(context: Context, repositories: list[GoRepository]) -> None:
proxy = ",".join(repository.url for repository in repositories)
lines = [f"GOPROXY={proxy}", "GOSUMDB=off"]
content = ("\n".join(lines) + "\n").encode("utf-8")
context.write_file(_GO_ENV, content)
for user in context.users:
context.write_file(f"{context.home_dir(user)}/.config/go/env", content)
logger.info("configured go for user %s", user)
logger.info("configured go to use %d module prox(y/ies)", len(repositories))
@register
class GoModule(Module):
name = "go"
def is_relevant(self, context: Context) -> bool:
return bool(context.config.configurations.go.repositories)
def apply(self, context: Context) -> None:
_write_go_env(context, context.config.configurations.go.repositories)
+302
View File
@@ -0,0 +1,302 @@
"""Add the configured root certificates to every Java trust store detected
under `<src>`, without shelling out to `keytool`, and configure the
Maven-compatible repositories declared in `configurations.java.repositories`
globally for the main JVM build tools.
Java ignores the system CA store and only trusts what is in each JRE/JDK's
own `cacerts` file: JKS format before Java 9, PKCS12 since Java 9. Every
`cacerts` file found under `usr/` or `opt/` in `<src>` is copied into `<dst>`
with the configured root certificates added, entirely in Python: `pyjks`
handles the legacy JKS format, `cryptography` handles PKCS12. The default
`changeit` store password is used, since virtually every JDK ships with it
unchanged; a keystore that doesn't open with it is left untouched and a
warning is logged instead of failing the whole run.
The repositories are written to the well-known global configuration file of
each build tool. Maven's is installation-wide, so every user on the image
picks it up; the others are written under root's home directory, since this
tool targets Docker images, which are built and run as root:
- Maven3: `/etc/maven/settings.xml` (the file `/usr/share/maven/conf/settings.xml`
points to on Debian/Ubuntu, Maven's global, all-users settings file), plus
`conf/settings.xml` in every other Maven installation detected under
`<src>` (found via its `bin/mvn` launcher). Both get an always-active
profile adding every repository, plus a `mirrorOf=*` mirror on the first
one so that even `pom.xml` files hardcoding central are redirected.
- Gradle: `~/.gradle/init.d/id-configurator-repositories.gradle`, an init
script adding the repositories to every project and to plugin management.
- Ivy: `~/.ivy2/ivysettings.xml`, as a chain of `ibiblio` resolvers.
- SBT: `~/.sbt/repositories`, the resolver override file SBT reads for its
own bootstrapping and, when combined with the `sbt.override.build.repos`
system property, for every project's resolvers as well.
- Leiningen: `~/.lein/profiles.clj`, adding the repositories to the `:user`
profile.
"""
from __future__ import annotations
import json
import logging
import xml.etree.ElementTree as ET
from pathlib import Path
import jks
from cryptography import x509
from cryptography.hazmat.primitives.serialization import (
Encoding,
NoEncryption,
pkcs12,
)
from id_configurator.config import JavaRepository, NamedContent
from id_configurator.context import Context
from id_configurator.modules.base import Module, register
logger = logging.getLogger(__name__)
_JAVA_SEARCH_ROOTS = ("usr", "opt")
_STORE_PASSWORD = b"changeit"
_MAVEN_GLOBAL_SETTINGS = "etc/maven/settings.xml"
_MAVEN_BIN_GLOBS = ("usr/**/bin/mvn", "opt/**/bin/mvn")
_GRADLE_INIT_SCRIPT = "root/.gradle/init.d/id-configurator-repositories.gradle"
_IVY_SETTINGS = "root/.ivy2/ivysettings.xml"
_SBT_REPOSITORIES = "root/.sbt/repositories"
_LEIN_PROFILES = "root/.lein/profiles.clj"
def _find_cacerts(context: Context) -> list[Path]:
found = {
path
for root in _JAVA_SEARCH_ROOTS
for path in context.glob_src(f"{root}/**/cacerts")
}
return sorted(path for path in found if path.is_file())
def _decode_certificates(certificates: list[NamedContent]) -> list[x509.Certificate]:
return [
x509.load_pem_x509_certificate(cert.content.encode()) for cert in certificates
]
def _add_to_pkcs12(
content: bytes, certificates: list[NamedContent], certs: list[x509.Certificate]
) -> bytes:
loaded = pkcs12.load_pkcs12(content, _STORE_PASSWORD)
entries = list(loaded.additional_certs)
for named, cert in zip(certificates, certs, strict=True):
entries.append(pkcs12.PKCS12Certificate(cert, named.name.encode("utf-8")))
# Trust-only entries must stay unencrypted, matching how the JDK's own
# cacerts is built: JVM code that never sets javax.net.ssl.trustStore*
# (e.g. Maven) loads the default truststore with a null password, which
# can only skip the integrity check, not decrypt encrypted entries.
# Encrypting them here (e.g. with BestAvailableEncryption) silently
# yields zero usable trust anchors for every such caller.
return pkcs12.serialize_java_truststore(entries, NoEncryption())
def _add_to_jks(
content: bytes, certificates: list[NamedContent], certs: list[x509.Certificate]
) -> bytes:
store = jks.KeyStore.loads(content, _STORE_PASSWORD.decode("ascii"))
for named, cert in zip(certificates, certs, strict=True):
store.entries[named.name] = jks.TrustedCertEntry.new(
named.name, cert.public_bytes(Encoding.DER)
)
return store.saves(_STORE_PASSWORD.decode("ascii"))
def _keystore_kind(content: bytes) -> str:
if content[:4] in (jks.MAGIC_NUMBER_JKS, jks.MAGIC_NUMBER_JCEKS):
return "jks"
return "pkcs12"
def _string_literal(value: str) -> str:
return json.dumps(value)
def _render_xml(root: ET.Element) -> bytes:
ET.indent(root)
content = '<?xml version="1.0" encoding="UTF-8"?>\n' + ET.tostring(
root, encoding="unicode"
)
return (content + "\n").encode("utf-8")
def _write_xml(context: Context, relative: str, root: ET.Element) -> None:
context.write_file(relative, _render_xml(root))
def _find_maven_homes(context: Context) -> list[Path]:
found = {
path.parent.parent
for pattern in _MAVEN_BIN_GLOBS
for path in context.glob_src(pattern)
if path.is_file()
}
return sorted(found)
def _build_maven_settings(repositories: list[JavaRepository]) -> ET.Element:
settings = ET.Element(
"settings",
{
"xmlns": "http://maven.apache.org/SETTINGS/1.1.0",
"xmlns:xsi": "http://www.w3.org/2001/XMLSchema-instance",
"xsi:schemaLocation": (
"http://maven.apache.org/SETTINGS/1.1.0 "
"http://maven.apache.org/xsd/settings-1.1.0.xsd"
),
},
)
profile = ET.SubElement(ET.SubElement(settings, "profiles"), "profile")
ET.SubElement(profile, "id").text = "id-configurator"
activation = ET.SubElement(profile, "activation")
ET.SubElement(activation, "activeByDefault").text = "true"
repositories_el = ET.SubElement(profile, "repositories")
plugin_repositories_el = ET.SubElement(profile, "pluginRepositories")
for repository in repositories:
for parent, tag in (
(repositories_el, "repository"),
(plugin_repositories_el, "pluginRepository"),
):
entry = ET.SubElement(parent, tag)
ET.SubElement(entry, "id").text = repository.name
ET.SubElement(entry, "url").text = repository.url
active_profiles = ET.SubElement(settings, "activeProfiles")
ET.SubElement(active_profiles, "activeProfile").text = "id-configurator"
mirror = ET.SubElement(ET.SubElement(settings, "mirrors"), "mirror")
ET.SubElement(mirror, "id").text = repositories[0].name
ET.SubElement(mirror, "url").text = repositories[0].url
ET.SubElement(mirror, "mirrorOf").text = "*"
return settings
def _write_maven_settings(context: Context, repositories: list[JavaRepository]) -> None:
content = _render_xml(_build_maven_settings(repositories))
relatives = {_MAVEN_GLOBAL_SETTINGS}
relatives.update(
f"{home.relative_to(context.src).as_posix()}/conf/settings.xml"
for home in _find_maven_homes(context)
)
for relative in sorted(relatives):
context.write_file(relative, content)
logger.info(
"configured maven mirrors for all users in %d location(s)", len(relatives)
)
def _gradle_repository_lines(repositories: list[JavaRepository], indent: str) -> str:
return "\n".join(
f"{indent}maven {{ url = uri({_string_literal(repository.url)}) }}"
for repository in repositories
)
def _write_gradle_init_script(
context: Context, repositories: list[JavaRepository]
) -> None:
content = (
"allprojects {\n"
" repositories {\n"
f"{_gradle_repository_lines(repositories, ' ')}\n"
" }\n"
"}\n"
"\n"
"settingsEvaluated { settings ->\n"
" settings.pluginManagement {\n"
" repositories {\n"
f"{_gradle_repository_lines(repositories, ' ')}\n"
" }\n"
" }\n"
"}\n"
)
context.write_file(_GRADLE_INIT_SCRIPT, content.encode("utf-8"))
logger.info("configured gradle to use %d repositories", len(repositories))
def _write_ivy_settings(context: Context, repositories: list[JavaRepository]) -> None:
ivysettings = ET.Element("ivysettings")
ET.SubElement(ivysettings, "settings", {"defaultResolver": "id-configurator"})
chain = ET.SubElement(
ET.SubElement(ivysettings, "resolvers"), "chain", {"name": "id-configurator"}
)
for repository in repositories:
ET.SubElement(
chain,
"ibiblio",
{"name": repository.name, "m2compatible": "true", "root": repository.url},
)
_write_xml(context, _IVY_SETTINGS, ivysettings)
logger.info("configured ivy to use %d repositories", len(repositories))
def _write_sbt_repositories(
context: Context, repositories: list[JavaRepository]
) -> None:
lines = ["[repositories]", " local"]
lines.extend(
f" {repository.name}: {repository.url}" for repository in repositories
)
context.write_file(_SBT_REPOSITORIES, ("\n".join(lines) + "\n").encode("utf-8"))
logger.info("configured sbt to use %d repositories", len(repositories))
def _write_lein_profiles(context: Context, repositories: list[JavaRepository]) -> None:
entries = " ".join(
f"[{_string_literal(repository.name)} {{:url {_string_literal(repository.url)}}}]"
for repository in repositories
)
content = f"{{:user {{:repositories [{entries}]}}}}\n"
context.write_file(_LEIN_PROFILES, content.encode("utf-8"))
logger.info("configured leiningen to use %d repositories", len(repositories))
def _configure_repositories(
context: Context, repositories: list[JavaRepository]
) -> None:
_write_maven_settings(context, repositories)
_write_gradle_init_script(context, repositories)
_write_ivy_settings(context, repositories)
_write_sbt_repositories(context, repositories)
_write_lein_profiles(context, repositories)
@register
class JavaModule(Module):
name = "java"
def is_relevant(self, context: Context) -> bool:
has_certificates = bool(context.config.configurations.certificates) and bool(
_find_cacerts(context)
)
has_repositories = bool(context.config.configurations.java.repositories)
return has_certificates or has_repositories
def apply(self, context: Context) -> None:
certificates = context.config.configurations.certificates
if certificates:
certs = _decode_certificates(certificates)
for path in _find_cacerts(context):
relative = path.relative_to(context.src).as_posix()
content = path.read_bytes()
kind = _keystore_kind(content)
try:
if kind == "jks":
updated = _add_to_jks(content, certificates, certs)
else:
updated = _add_to_pkcs12(content, certificates, certs)
context.write_file(relative, updated)
logger.info(
"added %d certificate(s) to java keystore %s",
len(certificates),
relative,
)
except (jks.KeystoreException, ValueError) as exc:
logger.warning("skipping java keystore %s: %s", relative, exc)
repositories = context.config.configurations.java.repositories
if repositories:
_configure_repositories(context, repositories)
+84
View File
@@ -0,0 +1,84 @@
"""Configure npm to use the configured NpmRepository entries: the first one
replaces the default registry, and any further ones are registered scoped to
their own name (`@name:registry`), e.g. for a private org's packages
alongside the default public registry.
Unlike Composer or pip, npm (via Node.js) does not trust the system CA store
by default - Node bundles its own root CA list - so this also points npm's
own `cafile` setting at the bundle the `certificates` module writes. Unlike
Poetry's per-repository certificate, `cafile` is a single, global npm
setting.
Written to npm's "global" config file (what `npm config --global` edits),
`$prefix/etc/npmrc` - not the "user" config, `$HOME/.npmrc`, which is a
separate, higher-precedence level in npm's own terminology. `$prefix`
defaults to `/usr/local` for essentially every standard Node.js install
(the official Docker image included); unlike a config dropped in one
specific user's home, this applies regardless of which user invokes npm.
`cafile` only covers npm's own registry client, though - it does nothing for
Node itself, e.g. a package's install script or the application's own code
making an HTTPS request. Node has no config file for that: the only way to
extend its trust store is the `NODE_EXTRA_CA_CERTS` environment variable,
read once at process startup. So whenever any certificate is configured -
regardless of whether any npm repository is - it is also set system-wide in
`/etc/environment` (merged with whatever is already there), which every
login-based process (an SSH session, `su -`, ...) picks up via PAM's
`pam_env`. This does not reach a `docker build`/`RUN` step or a container's
own `ENTRYPOINT`/`CMD`, which never go through a login shell: propagating it
there is the resulting image's own `Dockerfile`'s job (a plain `ENV`
instruction), outside what a tool that only writes files can do.
"""
from __future__ import annotations
import logging
from id_configurator.config import NpmRepository
from id_configurator.context import Context
from id_configurator.modules.base import Module, register
logger = logging.getLogger(__name__)
_NPMRC = "usr/local/etc/npmrc"
_CA_BUNDLE = "/etc/ssl/certs/ca-certificates.crt"
_ETC_ENVIRONMENT = "etc/environment"
_NODE_EXTRA_CA_CERTS_VAR = "NODE_EXTRA_CA_CERTS"
def _write_npmrc(context: Context, repositories: list[NpmRepository]) -> None:
lines = [f"cafile={_CA_BUNDLE}", f"registry={repositories[0].url}"]
lines.extend(
f"@{repository.name}:registry={repository.url}"
for repository in repositories[1:]
)
context.write_file(_NPMRC, ("\n".join(lines) + "\n").encode("utf-8"))
logger.info("configured npm to use %s", repositories[0].url)
def _update_etc_environment(context: Context) -> None:
existing = context.read_src_file(_ETC_ENVIRONMENT)
lines = [] if existing is None else existing.decode("utf-8").splitlines()
lines = [
line for line in lines if not line.startswith(f"{_NODE_EXTRA_CA_CERTS_VAR}=")
]
lines.append(f"{_NODE_EXTRA_CA_CERTS_VAR}={_CA_BUNDLE}")
context.write_file(_ETC_ENVIRONMENT, ("\n".join(lines) + "\n").encode("utf-8"))
logger.info("configured %s in %s", _NODE_EXTRA_CA_CERTS_VAR, _ETC_ENVIRONMENT)
@register
class NpmModule(Module):
name = "npm"
def is_relevant(self, context: Context) -> bool:
return bool(context.config.configurations.npm.repositories) or bool(
context.config.configurations.certificates
)
def apply(self, context: Context) -> None:
repositories = context.config.configurations.npm.repositories
if repositories:
_write_npmrc(context, repositories)
if context.config.configurations.certificates:
_update_etc_environment(context)
+78
View File
@@ -0,0 +1,78 @@
"""Configure NuGet to use the configured NuGetRepository entries as its
package sources, replacing the default (nuget.org).
NuGet's own HTTPS client (.NET's HttpClient, backed by OpenSSL on Linux)
already trusts whatever the `certificates` module installs into the system
CA store, so no separate certificate configuration is needed here, unlike
npm's.
Microsoft documents a "Computer"-wide config file for this purpose,
`/etc/opt/NuGet/Config` on Linux (or `$NUGET_COMMON_APPLICATION_DATA/NuGet/Config`
should that variable be set) - but the .NET SDK does not actually read it in
practice (confirmed empirically; a known, still-open upstream gap, see
dotnet/sdk#12674), so it would silently have no effect. This instead writes
the "user" level config, `~/.nuget/NuGet/NuGet.Config` (the location the
.NET CLI itself uses; Mono/NuGet.exe use `~/.config/NuGet/NuGet.Config`
instead). This tool targets Docker images, which are built and run as root,
so `~` is assumed to be `/root`, exactly like a fresh dotnet install does
when not overridden - the same tradeoff the `php` module's Composer config
already accepts. A `<clear />` element is included so the configured
sources fully replace nuget.org rather than being merged alongside it,
matching every other package-manager module's behaviour.
The same file is additionally written under each configured `--user`'s own
home directory (`context.users`), since a non-root user invoking `dotnet`
reads its config from there instead.
"""
from __future__ import annotations
import logging
import xml.etree.ElementTree as ET
from urllib.parse import urlparse
from id_configurator.config import NuGetRepository
from id_configurator.context import Context
from id_configurator.modules.base import Module, register
logger = logging.getLogger(__name__)
_NUGET_CONFIG = "root/.nuget/NuGet/NuGet.Config"
def _render_xml(root: ET.Element) -> bytes:
ET.indent(root)
content = '<?xml version="1.0" encoding="utf-8"?>\n' + ET.tostring(
root, encoding="unicode"
)
return (content + "\n").encode("utf-8")
def _write_nuget_config(context: Context, repositories: list[NuGetRepository]) -> None:
configuration = ET.Element("configuration")
package_sources = ET.SubElement(configuration, "packageSources")
ET.SubElement(package_sources, "clear")
for repository in repositories:
attributes = {"key": repository.name, "value": repository.url}
if urlparse(repository.url).scheme.lower() != "https":
attributes["allowInsecureConnections"] = "true"
ET.SubElement(package_sources, "add", attributes)
content = _render_xml(configuration)
context.write_file(_NUGET_CONFIG, content)
for user in context.users:
context.write_file(
f"{context.home_dir(user)}/.nuget/NuGet/NuGet.Config", content
)
logger.info("configured nuget for user %s", user)
logger.info("configured nuget to use %d package source(s)", len(repositories))
@register
class NuGetModule(Module):
name = "nuget"
def is_relevant(self, context: Context) -> bool:
return bool(context.config.configurations.nuget.repositories)
def apply(self, context: Context) -> None:
_write_nuget_config(context, context.config.configurations.nuget.repositories)
+100
View File
@@ -0,0 +1,100 @@
"""Configure Composer to use the configured PhpRepository entries instead of
(in addition to) Packagist.
Composer has no "replace the default registry" switch either: Packagist
stays enabled unless a repository is explicitly declared under the name
"packagist.org" and disabled. If any configured repository's URL isn't
HTTPS, Composer refuses to use it unless `secure-http` is turned off in
`config`; since that effectively means this is a private, non-internet
setup, Packagist - unreachable in such a setup anyway - is disabled at the
same time, exactly as if a `{"packagist.org": false}` repository entry had
been declared. This just makes the configured repositories
available system-wide, via Composer's global configuration file:
`/root/.composer/config.json` (`COMPOSER_HOME/config.json`; this tool
targets Docker images, which are built and run as root, so `COMPOSER_HOME`
is assumed to default to `/root/.composer`, exactly like a fresh Composer
install does when not overridden). The same file is additionally written
under each configured `--user`'s own home directory (`context.users`),
since a non-root user invoking `composer` reads its config from there
instead.
Composer's own HTTPS client (PHP's stream wrapper, generally backed by
curl) already trusts whatever the `certificates` module installs into the
system CA store, so no per-repository certificate configuration is needed
here, unlike Poetry's.
Whenever any certificate is configured - regardless of whether any PHP
repository is - `openssl.cafile` is also set, pointing at the same bundle,
in a dedicated `conf.d` snippet: `usr/local/etc/php/conf.d/id-configurator-
openssl.ini`, the directory the official PHP Docker images scan for extra
`.ini` files (`docker-php-ext-enable`'s own mechanism), applied regardless
of which SAPI (cli, fpm, apache) or php.ini is in use. This is normally
inferred from OpenSSL's own compiled-in default CAfile (which the
`certificates` module also rebuilds), but not every PHP build honors that
default, so setting it explicitly makes every PHP script's own OpenSSL-based
code (not just Composer's) reliably trust the configured certificates too.
"""
from __future__ import annotations
import json
import logging
from urllib.parse import urlparse
from id_configurator.config import PhpRepository
from id_configurator.context import Context
from id_configurator.modules.base import Module, register
logger = logging.getLogger(__name__)
_COMPOSER_CONFIG = "root/.composer/config.json"
_CA_BUNDLE = "/etc/ssl/certs/ca-certificates.crt"
_OPENSSL_INI = "usr/local/etc/php/conf.d/id-configurator-openssl.ini"
def _is_https(url: str) -> bool:
return urlparse(url).scheme.lower() == "https"
def _write_composer_config(context: Context, repositories: list[PhpRepository]) -> None:
composer_repositories: list[dict[str, str | bool]] = [
{"name": repository.name, "type": "composer", "url": repository.url}
for repository in repositories
]
content: dict[str, object] = {"repositories": composer_repositories}
composer_repositories.append({"packagist.org": False})
if any(not _is_https(repository.url) for repository in repositories):
content["config"] = {"secure-http": False}
logger.info(
"disabled secure-http and packagist.org: non-HTTPS repository configured"
)
rendered = (json.dumps(content, indent=4) + "\n").encode("utf-8")
context.write_file(_COMPOSER_CONFIG, rendered)
for user in context.users:
context.write_file(f"{context.home_dir(user)}/.composer/config.json", rendered)
logger.info("configured composer for user %s", user)
logger.info(
"configured composer to use %d additional repositories", len(repositories)
)
def _write_openssl_ini(context: Context) -> None:
context.write_file(_OPENSSL_INI, f"openssl.cafile={_CA_BUNDLE}\n".encode())
logger.info("configured openssl.cafile to use %s", _CA_BUNDLE)
@register
class PhpModule(Module):
name = "php"
def is_relevant(self, context: Context) -> bool:
return bool(context.config.configurations.php.repositories) or bool(
context.config.configurations.certificates
)
def apply(self, context: Context) -> None:
repositories = context.config.configurations.php.repositories
if repositories:
_write_composer_config(context, repositories)
if context.config.configurations.certificates:
_write_openssl_ini(context)
+118
View File
@@ -0,0 +1,118 @@
"""Configure the classic Python package managers to use the configured
PythonRepository entries instead of pypi.org, and to trust the root
certificates installed by the certificates module.
- pip: system-wide `/etc/pip.conf` (`index-url`/`extra-index-url`/`cert`).
- uv: system-wide `/etc/uv/uv.toml` (`[[index]]` entries, `native-tls`).
- poetry: `/root/.config/pypoetry/config.toml` (`repositories.*`/`certificates.*`).
Poetry has no global "replace PyPI" switch: a project's `pyproject.toml`
still has to reference these repositories as sources. This only makes the
repository and its certificate available system-wide. Since poetry only
reads its config from the current user's home directory, this assumes the
container runs as root; the same file is additionally written under each
configured `--user`'s own home directory (`context.users`), for the
benefit of a non-root user invoking poetry.
- easy_install: `distutils.cfg` (`[easy_install] index_url`), written next
to every Python installation detected under `<src>` (there is no location
independent of a specific Python prefix).
- pipx: not configured directly. It installs packages by invoking `pip`
inside the target venv, so it already picks up `/etc/pip.conf`.
"""
from __future__ import annotations
import json
import logging
from id_configurator.config import PythonRepository
from id_configurator.context import Context
from id_configurator.modules.base import Module, register
logger = logging.getLogger(__name__)
_CA_BUNDLE = "/etc/ssl/certs/ca-certificates.crt"
_PIP_CONF = "etc/pip.conf"
_UV_TOML = "etc/uv/uv.toml"
_POETRY_CONFIG = "root/.config/pypoetry/config.toml"
_DISTUTILS_GLOBS = ("usr/lib/python3.*", "usr/local/lib/python3.*")
def _toml_string(value: str) -> str:
return json.dumps(value)
def _write_pip_conf(context: Context, repositories: list[PythonRepository]) -> None:
lines = ["[global]", f"index-url = {repositories[0].index_url}"]
if len(repositories) > 1:
lines.append("extra-index-url =")
lines.extend(f" {repository.index_url}" for repository in repositories[1:])
lines.append(f"cert = {_CA_BUNDLE}")
context.write_file(_PIP_CONF, ("\n".join(lines) + "\n").encode("utf-8"))
logger.info("configured pip to use %s", repositories[0].index_url)
def _write_uv_toml(context: Context, repositories: list[PythonRepository]) -> None:
lines = ["native-tls = true", ""]
for position, repository in enumerate(repositories):
lines.append("[[index]]")
lines.append(f"name = {_toml_string(repository.name)}")
lines.append(f"url = {_toml_string(repository.index_url)}")
if position == 0:
lines.append("default = true")
lines.append("")
context.write_file(_UV_TOML, ("\n".join(lines).rstrip() + "\n").encode("utf-8"))
logger.info("configured uv to use %s", repositories[0].index_url)
def _write_poetry_config(
context: Context, repositories: list[PythonRepository]
) -> None:
lines = []
for repository in repositories:
lines.append(f"[repositories.{repository.name}]")
lines.append(f"url = {_toml_string(repository.index_url)}")
lines.append("")
lines.append(f"[certificates.{repository.name}]")
lines.append(f"cert = {_toml_string(_CA_BUNDLE)}")
lines.append("")
content = ("\n".join(lines).rstrip() + "\n").encode("utf-8")
context.write_file(_POETRY_CONFIG, content)
for user in context.users:
context.write_file(
f"{context.home_dir(user)}/.config/pypoetry/config.toml", content
)
logger.info("configured poetry for user %s", user)
logger.info("registered %d poetry repositories", len(repositories))
def _write_distutils_cfg(
context: Context, repositories: list[PythonRepository]
) -> None:
prefixes = [
p
for pattern in _DISTUTILS_GLOBS
for p in context.glob_src(pattern)
if p.is_dir()
]
for prefix in prefixes:
relative = prefix.relative_to(context.src).as_posix()
content = f"[easy_install]\nindex_url = {repositories[0].index_url}\n"
context.write_file(
f"{relative}/distutils/distutils.cfg", content.encode("utf-8")
)
logger.info("configured easy_install for %s", relative)
@register
class PythonModule(Module):
name = "python"
def is_relevant(self, context: Context) -> bool:
return bool(context.config.configurations.python.repositories)
def apply(self, context: Context) -> None:
repositories = context.config.configurations.python.repositories
_write_pip_conf(context, repositories)
_write_uv_toml(context, repositories)
_write_poetry_config(context, repositories)
_write_distutils_cfg(context, repositories)
+52
View File
@@ -0,0 +1,52 @@
"""Configure RubyGems to use the configured RubyRepository entries as its
gem sources, replacing the default (rubygems.org).
RubyGems' own HTTPS client (Net::HTTP, backed by OpenSSL) already trusts
whatever the `certificates` module installs into the system CA store, so no
separate certificate configuration is needed here, unlike npm's.
Written to RubyGems' system-wide config file, `Gem::ConfigFile::SYSTEM_WIDE_CONFIG_FILE`
- `$sysconfdir/gemrc`, where `$sysconfdir` is Ruby's own configured
sysconfdir, not necessarily `/etc`. The official Ruby Docker image builds
Ruby with `--prefix=/usr/local`, giving `/usr/local/etc/gemrc` (confirmed
empirically, since Debian's own `sysconfdir` convention of `/etc` does not
apply to a from-source build). Also written to the more conventional
`/etc/gemrc`, for any Ruby built with the Debian/`sysconfdir=/etc`
convention instead (e.g. a distro-packaged `ruby` rather than the official
image) - harmless to write in both places, since only whichever one a given
Ruby build actually treats as its sysconfdir is ever read. Unlike a config
dropped in one specific user's home (`~/.gemrc`, RubyGems' separate "per
user" level), this applies regardless of which user invokes `gem`.
"""
from __future__ import annotations
import logging
from id_configurator.config import RubyRepository
from id_configurator.context import Context
from id_configurator.modules.base import Module, register
logger = logging.getLogger(__name__)
_GEMRC_PATHS = ("usr/local/etc/gemrc", "etc/gemrc")
def _write_gemrc(context: Context, repositories: list[RubyRepository]) -> None:
lines = [":sources:"]
lines.extend(f"- {repository.url}" for repository in repositories)
content = ("\n".join(lines) + "\n").encode("utf-8")
for path in _GEMRC_PATHS:
context.write_file(path, content)
logger.info("configured gem to use %d source(s)", len(repositories))
@register
class RubyModule(Module):
name = "ruby"
def is_relevant(self, context: Context) -> bool:
return bool(context.config.configurations.ruby.repositories)
def apply(self, context: Context) -> None:
_write_gemrc(context, context.config.configurations.ruby.repositories)
+94
View File
@@ -0,0 +1,94 @@
"""Configure cargo to use the configured RustRepository entries: the first
one replaces crates.io entirely (via `[source.crates-io] replace-with`), and
any further ones are registered as additional named registries (usable via
`cargo add --registry <name>` or a dependency's `registry = "<name>"`). A
repository's index is declared as `sparse+<url>` - the sparse (plain HTTPS)
protocol - only when its `is_sparse` option is set; otherwise `<url>` is used
as-is, for a git-based index.
Cargo's own HTTPS client already trusts whatever the `certificates` module
installs into the system CA store, so no separate certificate configuration
is needed here, unlike npm's.
Written to `/.cargo/config.toml` - the filesystem root, not `$CARGO_HOME`
(which defaults to `$HOME/.cargo` but is routinely overridden, e.g. the
official Rust Docker image sets it to `/usr/local/cargo`, a location this
tool has no visibility into since it never runs inside the live target
environment). Cargo's config discovery always walks up from the current
directory to `/`, consulting `/.cargo/config.toml` there, *before* falling
back to `$CARGO_HOME/config.toml` - so this is the one genuinely global
location: found regardless of which user, working directory or `CARGO_HOME`
is in effect, unlike a config dropped in one specific user's home.
For each configured `--user` (`context.users`), the same config is
additionally written at that user's own `$HOME/.cargo/config.toml` - the
conventional, un-overridden `CARGO_HOME` location for that specific user,
which this module otherwise has no visibility into.
"""
from __future__ import annotations
import logging
import re
from id_configurator.config import RustRepository
from id_configurator.context import Context
from id_configurator.modules.base import Module, register
logger = logging.getLogger(__name__)
_CARGO_CONFIG = ".cargo/config.toml"
_REPOSITORY_NAME_FORBIDDEN_CHARS = re.compile(r"[^A-Za-z0-9_-]")
def _sanitize_repository_name(name: str) -> str:
"""Replace every character of `name` that isn't a plain identifier
character with `_`.
`name` is written verbatim both as a TOML table key (`[source.name]`,
`[registries.name]`) and inside a quoted string (`replace-with =
"name"`): an unrestricted name could break out of the table header or
string, or (via a literal `.`) silently create extra, nested tables
instead of naming the one table it is meant to.
"""
return _REPOSITORY_NAME_FORBIDDEN_CHARS.sub("_", name)
def _index_url(repository: RustRepository) -> str:
return f"sparse+{repository.url}" if repository.is_sparse else repository.url
def _write_cargo_config(context: Context, repositories: list[RustRepository]) -> None:
primary = repositories[0]
primary_name = _sanitize_repository_name(primary.name)
lines = [
"[source.crates-io]",
f'replace-with = "{primary_name}"',
"",
f"[source.{primary_name}]",
f'registry = "{_index_url(primary)}"',
]
for repository in repositories[1:]:
name = _sanitize_repository_name(repository.name)
lines += [
"",
f"[registries.{name}]",
f'index = "{_index_url(repository)}"',
]
content = ("\n".join(lines) + "\n").encode("utf-8")
context.write_file(_CARGO_CONFIG, content)
for user in context.users:
context.write_file(f"{context.home_dir(user)}/.cargo/config.toml", content)
logger.info("configured cargo for user %s", user)
logger.info("configured cargo to use %s", primary.url)
@register
class RustModule(Module):
name = "rust"
def is_relevant(self, context: Context) -> bool:
return bool(context.config.configurations.rust.repositories)
def apply(self, context: Context) -> None:
_write_cargo_config(context, context.config.configurations.rust.repositories)
+194
View File
@@ -0,0 +1,194 @@
"""Configure YUM/DNF repositories and trust their GPG keys.
DNF is a drop-in successor to YUM and reads the exact same `.repo` files
under `/etc/yum.repos.d/`, so a single module covers both.
If `<src>/etc/yum.repos.d` exists, every `.repo` file already there is
copied into `<dst>`, before the repositories declared in the configuration
are written as new, enabled `.repo` files (so a configured repository
reusing an existing file name ends up enabled, taking precedence over the
copied one). A section of an existing `.repo` file is only disabled
(`enabled=0`) if its `baseurl`/`mirrorlist` matches the
`index_url_to_replace` pattern of one of the configured repositories;
sections matching no such pattern are copied through unchanged. Each `.repo`
file is written referencing its `gpg-url` (used verbatim as `gpgkey=`,
exactly as YUM/DNF expect: `https://`, or a `file://` path already present
on the target system). Separately, each GPG key declared in
`configurations.yum.keys` is validated with `gpg` (catching a malformed key
early) and written to `/etc/pki/rpm-gpg/RPM-GPG-KEY-<name>`, the
conventional location such keys live at on RPM-based systems. `gpg` is only
used here to validate the key; the actual trust decision is left to
YUM/DNF, which import it from that file the first time the repository is
used (as `rpm --import` would).
A repository's `index_url_to_replace`, if set, is an fnmatch pattern
checked against the baseurl of every section of every existing `.repo`
file: the repository is only written out if the pattern matches at least
one of them, and skipped entirely otherwise.
"""
from __future__ import annotations
import configparser
import fnmatch
import logging
import re
import shutil
import subprocess
import tempfile
from collections.abc import Sequence
from io import StringIO
from id_configurator.config import NamedContent, YumRepository
from id_configurator.context import Context
from id_configurator.modules.base import Module, register, sanitize_filename
logger = logging.getLogger(__name__)
_REPOS_DIR = "etc/yum.repos.d"
_RPM_GPG_DIR = "etc/pki/rpm-gpg"
_REPOSITORY_NAME_FORBIDDEN_CHARS = re.compile(r"[^A-Za-z0-9._-]")
def _sanitize_repository_name(name: str) -> str:
"""Replace every character of `name` that isn't a plain identifier
character with `_`.
`name` is written verbatim as the `.repo` file's INI section header
(`[name]`): an unrestricted name (containing `]`, a newline, ...) could
otherwise inject extra, attacker-controlled sections into the file
instead of just naming the one section it is meant to.
"""
return _REPOSITORY_NAME_FORBIDDEN_CHARS.sub("_", name)
def _disable_repo_content(content: str, patterns: Sequence[str]) -> str:
parser = configparser.ConfigParser(interpolation=None)
parser.optionxform = str
parser.read_string(content)
for section in parser.sections():
urls = [parser[section].get(key) for key in ("baseurl", "mirrorlist")]
urls = [url for url in urls if url]
if any(fnmatch.fnmatch(url, pattern) for url in urls for pattern in patterns):
parser[section]["enabled"] = "0"
buffer = StringIO()
parser.write(buffer, space_around_delimiters=False)
return buffer.getvalue()
def _existing_index_urls(context: Context) -> set[str]:
"""Collect the baseurl of every section of every .repo file already
present on the source filesystem."""
urls: set[str] = set()
for entry in context.iter_src_dir(_REPOS_DIR):
if entry.suffix != ".repo":
continue
parser = configparser.ConfigParser(interpolation=None)
parser.optionxform = str
parser.read_string(entry.read_text(encoding="utf-8"))
for section in parser.sections():
# noinspection SpellCheckingInspection
for key in ("baseurl", "mirrorlist"):
url = parser[section].get(key)
if url:
urls.add(url)
return urls
def _disable_existing_repos(context: Context, patterns: Sequence[str]) -> None:
for entry in context.iter_src_dir(_REPOS_DIR):
if entry.suffix != ".repo":
continue
disabled = _disable_repo_content(entry.read_text(encoding="utf-8"), patterns)
context.write_file(f"{_REPOS_DIR}/{entry.name}", disabled.encode("utf-8"))
logger.info("copied existing yum/dnf repository file %s", entry.name)
def _write_repo_file(context: Context, repository: YumRepository) -> None:
name = _sanitize_repository_name(repository.name)
lines = [
f"[{name}]",
f"name={name}",
f"baseurl={repository.index_url}",
"enabled=1",
]
if repository.gpg_url:
lines.append("gpgcheck=1")
lines.append(f"gpgkey={repository.gpg_url}")
else:
lines.append("gpgcheck=0")
context.write_file(
f"{_REPOS_DIR}/{name}.repo",
("\n".join(lines) + "\n").encode("utf-8"),
)
logger.info("added yum/dnf repository %s", name)
def _validate_gpg_key(name: str, content: bytes) -> None:
with tempfile.TemporaryDirectory() as gnupg_home:
path = shutil.which("gpg")
if path is not None:
result = subprocess.run(
[path, "--homedir", gnupg_home, "--batch", "--import"],
input=content,
capture_output=True,
check=False,
)
else:
raise RuntimeError(
"gpg is not available; the GNUPG package must be installed in the environment running id-configurator"
)
if result.returncode != 0:
raise RuntimeError(
f"invalid GPG key {name}: {result.stderr.decode('utf-8', 'replace')}"
)
def _import_key(context: Context, key: NamedContent) -> None:
_validate_gpg_key(key.name, key.content.encode("utf-8"))
context.write_file(
f"{_RPM_GPG_DIR}/RPM-GPG-KEY-{sanitize_filename(key.name)}",
key.content.encode("utf-8"),
)
logger.info("imported yum/dnf GPG key %s", key.name)
@register
class YumModule(Module):
name = "yum"
def is_relevant(self, context: Context) -> bool:
yum = context.config.configurations.yum
existing_repo_files = (
entry
for entry in context.iter_src_dir(_REPOS_DIR)
if entry.suffix == ".repo"
)
return bool(yum.repositories) or bool(yum.keys) or any(existing_repo_files)
def apply(self, context: Context) -> None:
patterns = [
repository.index_url_to_replace
for repository in context.config.configurations.yum.repositories
if repository.index_url_to_replace is not None
]
_disable_existing_repos(context, patterns)
existing_index_urls: set[str] | None = None
for repository in context.config.configurations.yum.repositories:
if repository.index_url_to_replace is not None:
if existing_index_urls is None:
existing_index_urls: set[str] = _existing_index_urls(context)
if not any(
fnmatch.fnmatch(url, repository.index_url_to_replace)
for url in existing_index_urls
):
logger.info(
"skipped yum/dnf repository %s: index-url-to-replace %s "
"matched no existing baseurl",
repository.name,
repository.index_url_to_replace,
)
continue
_write_repo_file(context, repository)
for key in context.config.configurations.yum.keys:
_import_key(context, key)
+78
View File
@@ -0,0 +1,78 @@
"""Parse `/etc/passwd` entries for the `--auto-users` CLI option.
A passwd(5) line is `name:passwd:uid:gid:gecos:home:shell`. `--auto-users`
expands to every entry that looks like a regular (non-system) account - see
`PasswdEntry.is_regular_user`.
"""
from __future__ import annotations
from dataclasses import dataclass
DEFAULT_UID_MIN = 1000
_DISABLED_SHELL_NAMES = {"nologin", "sync", "false"}
@dataclass(frozen=True)
class PasswdEntry:
name: str
uid: int
home: str
shell: str
@property
def has_login_shell(self) -> bool:
return self.shell.rsplit("/", 1)[-1] not in _DISABLED_SHELL_NAMES
def is_regular_user(self, uid_min: int = DEFAULT_UID_MIN) -> bool:
"""True unless `self` looks like a system account.
A login-disabling shell (`has_login_shell` False) only catches
service accounts that follow that convention; it misses others
(and could wrongly flag a merely locked human account). The
authoritative signal `useradd`/`adduser`/`systemd-sysusers` use
instead is the UID: system accounts get one below `uid_min` (the
`UID_MIN` from login.defs(5), 1000 by default). Both checks apply
together here - except for `root`, whose UID 0 is itself a system
convention rather than a sign it shouldn't be configured: it's
exempted from the UID check (though not from the shell one, so a
deliberately locked root account is still excluded).
"""
if not self.has_login_shell:
return False
return self.name == "root" or self.uid >= uid_min
def parse_passwd(content: str) -> list[PasswdEntry]:
entries = []
for line in content.splitlines():
if not line or line.startswith("#"):
continue
fields = line.split(":")
if len(fields) < 7:
continue
try:
uid = int(fields[2])
except ValueError:
continue
entries.append(
PasswdEntry(name=fields[0], uid=uid, home=fields[5], shell=fields[6]) # noqa: B604
)
return entries
def parse_login_defs_uid_min(content: str) -> int | None:
"""Extract `UID_MIN` from `/etc/login.defs` content (login.defs(5)), or
`None` if it isn't set there."""
for line in content.splitlines():
stripped = line.strip()
if not stripped or stripped.startswith("#"):
continue
parts = stripped.split()
if len(parts) >= 2 and parts[0] == "UID_MIN":
try:
return int(parts[1])
except ValueError:
return None
return None
View File
+50
View File
@@ -0,0 +1,50 @@
import logging
import typer
from id_tools.build import app as build_app
from id_tools.publication import app as publication_app
from id_tools.test import app as test_app
SUCCESS = 25
class ColorFormatter(logging.Formatter):
"""Logging formatter that adds ANSI color codes based on log level."""
RESET = "\033[0m"
COLORS = {
logging.DEBUG: "\033[90m", # gris
logging.INFO: "\033[34m", # bleu
SUCCESS: "\033[32m", # vert
logging.WARNING: "\033[33m", # orange
logging.ERROR: "\033[31m", # rouge
logging.CRITICAL: "\033[1;31m", # rouge gras
}
def format(self, record):
color = self.COLORS.get(record.levelno, self.RESET)
message = super().format(record)
return f"{color}{message}{self.RESET}"
def setup_logging():
handler = logging.StreamHandler()
handler.setFormatter(ColorFormatter("%(message)s"))
logging.root.addHandler(handler)
logging.root.setLevel(logging.INFO)
app = typer.Typer()
@app.callback()
def main():
setup_logging()
app.add_typer(build_app, name="build")
app.add_typer(test_app, name="test")
app.add_typer(publication_app, name="publication")
if __name__ == "__main__":
app()
+178
View File
@@ -0,0 +1,178 @@
import logging
import subprocess
import tempfile
import typer
from id_tools.config import SourceConfig
app = typer.Typer(help="Build utilities for JS/CSS files and emails templates")
logger = logging.getLogger(__name__)
# noinspection SpellCheckingInspection
DOCKER_FILE_BUILDER = r"""# This file is part of Interdiode
# Copyright (c) {build_year} InterDiode <contact@mail.interdiode.fr>
# All Rights Reserved
########################################################################################################################
# This file is generated by id_tools/build.py. Do not edit it manually.
########################################################################################################################
FROM dhi.io/python:{python_version}-{docker_version} AS interdiode-base
########################################################################################################################
# generate the package and download required some tools in a first image
########################################################################################################################
FROM interdiode-base AS interdiode-builder
ENV{docker_env}
USER root
RUN mkdir -p /src /usr/local/interdiode
RUN chown -R {docker_user}:{docker_group} /src /usr/local/interdiode
COPY ./pyproject.toml /src/pyproject.toml
COPY ./README.md /src/README.md
COPY ./LICENSE /src/LICENSE
COPY ./id_configurator /src/id_configurator
COPY ./tests /src/tests
RUN chown -R {docker_user}:{docker_group} /src
USER {docker_user}
RUN python3 -m pip install --user pip uv --upgrade
WORKDIR /src
RUN python3 -m uv build
"""
# noinspection SpellCheckingInspection
DOCKER_FILE_INSTALL = r"""
########################################################################################################################
# install the package without the build dependencies
########################################################################################################################
FROM interdiode-base AS interdiode-install
ENV{docker_env}
COPY --from=interdiode-builder /src/dist /tmp/dist
RUN apt-get update
RUN DEBIAN_FRONTEND=noninteractive apt-get -o Dpkg::Options::="--force-confnew" --allow-downgrades --allow-remove-essential --allow-unauthenticated -fuy dist-upgrade
RUN DEBIAN_FRONTEND=noninteractive apt-get install --yes -qq --no-install-recommends build-essential
RUN mkdir -p /src/build /chroot/usr/local/bin /usr/local/interdiode \
&& chown -R {docker_user}:{docker_group} /usr/local/interdiode /tmp/dist
USER {docker_user}
RUN /bin/sh -c "python3 -m pip install --user /tmp/dist/*.whl"
"""
# noinspection SpellCheckingInspection
DOCKER_FILE_RUNTIME = r"""########################################################################################################################
# copy the installed packages
########################################################################################################################
FROM interdiode-base AS interdiode-runtime
ENV{docker_env}
LABEL org.opencontainers.image.authors="contact@mail.interdiode.fr"
LABEL org.opencontainers.image.created="{build_date}"
LABEL org.opencontainers.image.description="Air gap made simple!"
LABEL org.opencontainers.image.documentation="{documentation_url}"
LABEL org.opencontainers.image.licenses="LicenseRef-InterDiode-Commercial-1.0"
LABEL org.opencontainers.image.revision="{vcs_revision}"
LABEL org.opencontainers.image.title="InterDiode Configurator"
LABEL org.opencontainers.image.url="{interdiode_url}"
LABEL org.opencontainers.image.vendor="InterDiode"
LABEL org.opencontainers.image.version="{id_configurator_version}"
LABEL description="Air gap made simple!"
COPY --from=interdiode-install --chown=0:0 /chroot/usr/local/bin /usr/local/bin
COPY --from=interdiode-install /usr/local/interdiode /usr/local/interdiode
RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get -o Dpkg::Options::="--force-confnew" --allow-downgrades --allow-remove-essential --allow-unauthenticated -fuy dist-upgrade \
&& DEBIAN_FRONTEND=noninteractive apt-get install --yes -qq --no-install-recommends gpg gpg-agent \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* \
&& mkdir -p /usr/local/interdiode \
&& chown -R {docker_user}:{docker_group} /usr/local/interdiode
COPY ./LICENSE /chroot/licenses/LICENSE
COPY ./LICENSE /chroot/usr/share/doc/interdiode/LICENSE
WORKDIR /
"""
def build_docker_image(
config: SourceConfig,
registry: str | None = None,
platform: str = "linux/amd64",
push: bool = False,
) -> bool:
"""Build the Docker image."""
if push and not registry:
raise ValueError("--push requires --registry to be set.")
version = config.get_project_version()
name = config.get_project_name()
values = config.get_parameters()
package_build = DOCKER_FILE_BUILDER.format(**values)
v_tag = f"{name}:{version}"
l_tag = f"{name}:latest"
image_install = DOCKER_FILE_INSTALL.format(**values)
image_build = DOCKER_FILE_RUNTIME.format(**values)
with (
tempfile.NamedTemporaryFile(
mode="w", delete=True, prefix="Dockerfile.", suffix=".package"
) as fd,
):
fd.write(package_build)
fd.write(image_install)
fd.write(image_build)
fd.flush()
# noinspection SpellCheckingInspection
cmd = [
"docker",
"buildx",
"build",
"--no-cache",
"--attest",
"type=provenance,mode=max,version=v1",
"--attest",
"type=sbom",
"--platform",
platform,
"-f",
fd.name,
]
if registry:
cmd += ["-t", f"{registry}/{v_tag}", "-t", f"{registry}/{l_tag}"]
else:
cmd += ["-t", f"{registry}/{v_tag}", "-t", f"{registry}/{l_tag}"]
if push and registry:
cmd.insert(3, "--push")
cmd.append(".")
try:
typer.echo(f"Building Docker image {v_tag} for platform {platform}...")
typer.echo(" ".join(cmd))
subprocess.check_call(cmd, cwd=config.source_dir)
except subprocess.CalledProcessError:
return False
return True
# noinspection SpellCheckingInspection
@app.command(name="docker", help="Build the Docker image.")
def docker_image(
registry: str = typer.Option(
"interdiode", help="The registry to publish the Docker image to."
),
platform: str = typer.Option(
"linux/amd64", help="Platform(s) of the Docker image."
),
docker_version: str = typer.Option(
"debian13-dev", help="Version of the base Docker image."
),
push: bool = typer.Option(
False, "--push", help="Push the built image(s) to the registry."
),
docker_login: str = typer.Option(
"", envvar="DOCKER_HUB_LOGIN", help="Nom d'utilisateur Docker"
),
docker_token: str = typer.Option(
"", envvar="DOCKER_HUB_TOKEN", help="Jeton d'accès Docker"
),
):
config = SourceConfig(docker_version)
if docker_login and docker_token:
cmd = ["docker", "login", "-u", docker_login, "--password-stdin"]
subprocess.run(
cmd, input=docker_token, text=True, check=True, cwd=config.source_dir
)
success = build_docker_image(
config, registry=registry, platform=platform, push=push
)
if not success:
raise typer.Exit(code=1)
+76
View File
@@ -0,0 +1,76 @@
import datetime
import pathlib
import re
import subprocess
import tomllib
class SourceConfig:
site_url = "https://interdiode.fr/"
documentation_url = "https://interdiode.fr/static/latest/index.html"
cosign_key_url = "https://interdiode.fr/static/cosign.interdiode.fr.pub"
docker_user = "nonroot"
docker_group = "nonroot"
# noinspection SpellCheckingInspection
docker_env = """ PATH=/usr/local/bin/:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/interdiode/bin \
PIP_DISABLE_PIP_VERSION_CHECK=on \
PIP_NO_CACHE_DIR=on \
PYTHONUSERBASE=/usr/local/interdiode \
PYTHONFAULTHANDLER=1 \
PYTHONHASHSEED=random \
PYTHONUNBUFFERED=1
"""
# noinspection SpellCheckingInspection
def __init__(self, docker_version: str = "debian13-dev"):
self.source_dir = pathlib.Path(__file__).parent.parent
with open(self.source_dir / "pyproject.toml", "rb") as fd:
self.raw_metadata = tomllib.load(fd)
self.docker_version = docker_version
def get_python_versions(self) -> list[str]:
"""Return the list of Python versions supported by the project."""
python_versions = []
for classifier in self.raw_metadata["project"]["classifiers"]:
if matcher := re.match(
r"Programming Language :: Python :: (\d\.\d+)", classifier
):
python_versions.append(matcher.group(1))
return python_versions
def get_project_name(self) -> str:
"""Get the name of InterDiode from the pyproject.toml file."""
return self.raw_metadata["project"]["name"]
def get_project_version(self) -> str:
"""Get the version of InterDiode from the pyproject.toml file."""
return self.raw_metadata["project"]["version"]
def get_git_project_revision(self) -> str:
"""Get the git revision of the project."""
out = subprocess.check_output(["git", "rev-parse", "HEAD"], cwd=self.source_dir) # nosec B607
return out.decode().strip()
def get_parameters(self, python_version: str | None = None) -> dict[str, str]:
python_version = python_version or self.get_python_versions()[-1]
return {
"build_date": datetime.datetime.now(tz=datetime.UTC).isoformat(),
"build_year": str(datetime.datetime.now(tz=datetime.UTC).year),
"docker_env": self.docker_env,
"docker_user": self.docker_user,
"docker_group": self.docker_group,
"docker_version": self.docker_version,
"documentation_url": self.documentation_url,
"interdiode_url": self.site_url,
"id_configurator_version": self.get_project_version(),
"python_version": python_version,
"vcs_revision": self.get_git_project_revision(),
}
def get_project_author_and_emails(self) -> list[dict[str, str]]:
"""Get the authors of InterDiode from the pyproject.toml file."""
return self.raw_metadata["project"]["authors"]
def get_project_authors(self) -> list[str]:
"""Get the authors of InterDiode from the pyproject.toml file."""
return [author["name"] for author in self.raw_metadata["project"]["authors"]]
+129
View File
@@ -0,0 +1,129 @@
import json
import logging
import os
import subprocess
import tempfile
import typer
from id_tools.config import SourceConfig
app = typer.Typer(
help="Publish the documentation and release announcements to external destinations."
)
logger = logging.getLogger(__name__)
@app.command(name="sign-docker", help="Signe l'image Docker sur le registre")
def sign_docker_image(
registry: str = typer.Option(
"interdiode", help="Registre cible pour l'image Docker"
),
docker_login: str = typer.Option(
"", envvar="DOCKER_HUB_LOGIN", help="Nom d'utilisateur Docker"
),
docker_token: str = typer.Option(
"", envvar="DOCKER_HUB_TOKEN", help="Jeton d'accès Docker"
),
cosign_password: str = typer.Option(
"", envvar="COSIGN_PASSWORD", help="Mot de passe de la clé privée Cosign"
),
cosign_private_key: str = typer.Option(
"", envvar="COSIGN_PRIVATE_KEY", help="Contenu de la clé privée Cosign"
),
):
config = SourceConfig()
version = config.get_project_version()
image = config.get_project_name()
v_tag = f"{image}:{version}"
l_tag = f"{image}:latest"
destinations = [f"{registry}/{v_tag}", f"{registry}/{l_tag}"]
if docker_login and docker_token:
cmd = ["docker", "login", "-u", docker_login, "--password-stdin"]
subprocess.run(
cmd, input=docker_token, text=True, check=True, cwd=config.source_dir
)
digest_references = {
get_reference_from_tag(destination) for destination in destinations
}
if cosign_password and cosign_private_key:
with (
tempfile.NamedTemporaryFile(mode="w", suffix=".key", delete=True) as key_fd,
tempfile.TemporaryDirectory() as tmp_dir,
):
key_fd.write(cosign_private_key)
key_fd.flush()
cosign_env = {**os.environ, "COSIGN_PASSWORD": cosign_password}
trivy_report = os.path.join(tmp_dir, "trivy_report.json")
for digest_reference in digest_references:
# noinspection SpellCheckingInspection
cmd = [
"trivy",
"image",
"--format",
"json",
"--output",
trivy_report,
digest_reference,
]
subprocess.run(cmd, text=True, check=True, cwd=config.source_dir)
cmd = [
"cosign",
"attest",
"--key",
key_fd.name,
"--type",
"vuln",
"--predicate",
trivy_report,
digest_reference,
]
subprocess.run(cmd, text=True, check=True, cwd=config.source_dir)
cmd = [
"cosign",
"sign",
"--yes",
"--recursive",
"--key",
key_fd.name,
digest_reference,
]
subprocess.check_call(cmd, env=cosign_env)
cmd = [
"cosign",
"verify-attestation",
"--key",
config.cosign_key_url,
"--type",
"vuln",
digest_reference,
]
subprocess.check_call(cmd, env=cosign_env)
cmd = [
"cosign",
"verify",
"--key",
config.cosign_key_url,
digest_reference,
]
subprocess.check_call(cmd, env=cosign_env)
typer.echo(f"✓ Image signée : {registry}/{v_tag} et {registry}/{l_tag}")
def get_reference_from_tag(destination: str) -> str:
# noinspection SpellCheckingInspection
cmd = [
"docker",
"buildx",
"imagetools",
"inspect",
destination,
"--format",
"{{json .Manifest}}",
]
result = subprocess.run(cmd, check=True, capture_output=True, text=True)
manifest = json.loads(result.stdout)
image_name = destination.rsplit(":", 1)[0]
reference = f"{image_name}@{manifest['digest']}"
return reference
+66
View File
@@ -0,0 +1,66 @@
"""Utilities for the Docker integration tests under tests/."""
from __future__ import annotations
import os
import pathlib
from typing import Annotated
import jinja2
import typer
app = typer.Typer(help="Utilities for the tests/ Docker integration tests")
TESTS_DIR = pathlib.Path(__file__).resolve().parent.parent / "tests"
def _env(name: str) -> str:
"""Jinja2 filter: `'NAME' | env` -> the environment variable NAME."""
return os.environ[name]
def _file(relative: str) -> str:
"""Jinja2 filter: `'relative/path' | file` -> content of tests/relative/path."""
path = (TESTS_DIR / relative).resolve()
if path != TESTS_DIR and TESTS_DIR not in path.parents:
raise ValueError(f"refusing to read a file outside of {TESTS_DIR}: {relative}")
return path.read_text(encoding="utf-8")
def render_template(text: str) -> str:
"""Render a tests/configs/*.yaml.j2 fixture as a Jinja2 template.
Custom filters: `| env` reads an environment variable, `| file` reads a
file relative to tests/. Combine with Jinja2's built-in `indent` filter
to fit multi-line content (e.g. a PEM certificate) into a YAML
block-literal field, e.g.:
content: |
{{ 'certs/test-ca.crt.pem' | file | indent(6, first=true) }}
"""
# Autoescaping is for HTML output; these templates render YAML, so escaping
# would corrupt them (e.g. turning "&" into "&amp;" inside a PEM/URL).
environment = jinja2.Environment(undefined=jinja2.StrictUndefined) # nosec B701
environment.filters["env"] = _env
environment.filters["file"] = _file
return environment.from_string(text).render()
@app.command(
name="render-config",
help="Render a tests/configs/*.yaml.j2 fixture into a ready-to-use configuration.",
)
def render_config(
template: Annotated[pathlib.Path, typer.Argument(exists=True, dir_okay=False)],
output: pathlib.Path,
) -> None:
"""Renders `template` as a Jinja2 template (see `render_template`) and
writes the result to `output`, creating its parent directories as
needed."""
content = render_template(template.read_text(encoding="utf-8"))
output.parent.mkdir(parents=True, exist_ok=True)
output.write_text(content, encoding="utf-8")
if __name__ == "__main__":
app()
+653
View File
@@ -0,0 +1,653 @@
#!/usr/bin/python
"""Ansible module wrapping id-configurator.
Applies an id-configurator configuration directly to this host's live root
filesystem, exactly like the `id_configurator` role does, but as a native
Ansible module: it imports id-configurator's own Python API in-process
(instead of shelling out to the CLI through a temporary file), which also
lets it honor Ansible's check mode - reporting which modules would be
applied without writing anything.
This module is a thin, additive wrapper: it does not change anything about
id-configurator's own behavior (the `id_configurator` Python package,
including its module registry and every individual module, is used exactly
as-is - the same `Configuration`, `Context` and module registry the CLI
itself uses).
"""
from __future__ import annotations
from pathlib import Path
from ansible.module_utils.basic import AnsibleModule
DOCUMENTATION = r"""
module: id_configurator
short_description: Apply an id-configurator configuration to this host.
description:
- Assembles the given options into an id-configurator configuration and
applies it directly to this host's live root filesystem, by importing
id-configurator's own Python API in-process (no CLI subprocess, no
temporary file).
- The option names and structure mirror id_configurator.config.Configuration
and id_configurator.config.ConfigurationsBlock one for one. The
`id_configurator` Python package must already be installed on the
target host, using the same Python interpreter Ansible uses there.
- Supports check mode - reports which modules would be applied without
writing anything.
author:
- InterDiode
options:
version:
type: str
default: "1.0"
description: Configuration.version - the configuration format's own schema version.
uuid:
type: str
description: Configuration.uuid - a UUID identifying this configuration, optional.
organisation:
type: str
description: Configuration.organisation, optional.
description:
type: str
description: Configuration.description, optional.
certificates:
type: list
elements: dict
default: []
description: ConfigurationsBlock.certificates - root certificates to install.
suboptions:
name:
type: str
required: true
description: A short, unique name for this entry.
content:
type: str
required: true
description: PEM-encoded certificate content.
apt:
type: dict
default: {}
description: ConfigurationsBlock.apt.
suboptions:
sources:
type: list
elements: dict
default: []
description: DEB822 APT sources (AptSource).
suboptions:
name:
type: str
required: true
description: A short, unique name for this source.
types:
type: list
elements: str
default: [deb]
description: DEB822 "Types" field.
uris:
type: list
elements: str
required: true
description: DEB822 "URIs" field.
suites:
type: list
elements: str
required: true
description: DEB822 "Suites" field.
components:
type: list
elements: str
default: []
description: DEB822 "Components" field.
architectures:
type: list
elements: str
default: []
description: DEB822 "Architectures" field.
languages:
type: list
elements: str
default: []
description: DEB822 "Languages" field.
targets:
type: list
elements: str
default: []
description: DEB822 "Targets" field.
enabled:
type: bool
default: true
description: DEB822 "Enabled" field.
trusted:
type: bool
description: DEB822 "Trusted" field.
signed_by:
type: str
description:
- ASCII-armored content of a GPG public key to trust this
source with. Folded inline as a multi-line Signed-By field
(no separate keyring file or gpg binary needed). Takes
precedence over signed_by_file if both are set.
signed_by_file:
type: str
description:
- Path to a keyring already expected to exist on the target
system, used verbatim as the rendered Signed-By field.
Ignored if signed_by is also set.
pdiffs:
type: bool
description: DEB822 "PDiffs" field.
by_hash:
type: bool
description: DEB822 "By-Hash" field.
allow_insecure:
type: bool
description: DEB822 "Allow-Insecure" field.
allow_weak:
type: bool
description: DEB822 "Allow-Weak" field.
allow_downgrade_to_insecure:
type: bool
description: DEB822 "Allow-Downgrade-To-Insecure" field.
check_valid_until:
type: bool
description: DEB822 "Check-Valid-Until" field.
valid_until_min:
type: str
description: DEB822 "Valid-Until-Min" field.
valid_until_max:
type: str
description: DEB822 "Valid-Until-Max" field.
check_date:
type: bool
description: DEB822 "Check-Date" field.
date_max_future:
type: str
description: DEB822 "Date-Max-Future" field.
description:
type: str
description: DEB822 "Description" field.
uri_to_replace:
type: str
description:
- fnmatch pattern checked against the URIs already present in
the machine's existing apt configuration. If set, this
source is only applied when the pattern matches at least
one of them; otherwise it is skipped entirely.
apk:
type: dict
default: {}
description: ConfigurationsBlock.apk.
suboptions:
repositories:
type: list
elements: dict
default: []
description: Alpine APK repositories (ApkRepository).
suboptions:
url:
type: str
required: true
description: Repository URL, e.g. ending in /main or /community.
key:
type: str
description: The signing key's name, if any (see NamedContent).
url_to_replace:
type: str
description:
- fnmatch pattern checked against the URLs already active in
the machine's existing /etc/apk/repositories. If set, this
repository is only applied when the pattern matches at
least one of them; otherwise it is skipped entirely.
yum:
type: dict
default: {}
description: ConfigurationsBlock.yum.
suboptions:
repositories:
type: list
elements: dict
default: []
description: YUM/DNF repositories (YumRepository).
suboptions:
name:
type: str
required: true
description: A short, unique name for this repository.
index_url:
type: str
required: true
description: The repository's baseurl.
gpg_url:
type: str
description: The repository's gpgkey URL.
index_url_to_replace:
type: str
description:
- fnmatch pattern checked against the baseurl of every
section of every existing .repo file. If set, this
repository is only applied when the pattern matches at
least one of them; otherwise it is skipped entirely.
keys:
type: list
elements: dict
default: []
description: GPG keys imported for the repositories above (NamedContent).
suboptions:
name:
type: str
required: true
description: A short, unique name for this entry.
content:
type: str
required: true
description: PEM-encoded/ASCII-armored key content.
python:
type: dict
default: {}
description: ConfigurationsBlock.python.
suboptions:
repositories:
type: list
elements: dict
default: []
description: Python package indexes (PythonRepository).
suboptions:
name:
type: str
required: true
description: A short, unique name for this index.
index_url:
type: str
required: true
description: The package index's URL.
java:
type: dict
default: {}
description: ConfigurationsBlock.java.
suboptions:
repositories:
type: list
elements: dict
default: []
description: Maven-compatible repositories (JavaRepository).
suboptions:
name:
type: str
required: true
description: A short, unique name for this repository.
url:
type: str
required: true
description: The repository's URL.
php:
type: dict
default: {}
description: ConfigurationsBlock.php.
suboptions:
repositories:
type: list
elements: dict
default: []
description: Composer repositories (PhpRepository).
suboptions:
name:
type: str
required: true
description: A short, unique name for this repository.
url:
type: str
required: true
description: The repository's URL.
npm:
type: dict
default: {}
description: ConfigurationsBlock.npm.
suboptions:
repositories:
type: list
elements: dict
default: []
description: npm registries (NpmRepository).
suboptions:
name:
type: str
required: true
description: A short, unique name for this repository.
url:
type: str
required: true
description: The repository's URL.
rust:
type: dict
default: {}
description: ConfigurationsBlock.rust.
suboptions:
repositories:
type: list
elements: dict
default: []
description: Cargo registries (RustRepository).
suboptions:
name:
type: str
required: true
description: A short, unique name for this repository.
url:
type: str
required: true
description: The repository's URL.
go:
type: dict
default: {}
description: ConfigurationsBlock.go.
suboptions:
repositories:
type: list
elements: dict
default: []
description: Go module proxies (GoRepository).
suboptions:
name:
type: str
required: true
description: A short, unique name for this repository.
url:
type: str
required: true
description: The repository's URL.
ruby:
type: dict
default: {}
description: ConfigurationsBlock.ruby.
suboptions:
repositories:
type: list
elements: dict
default: []
description: RubyGems sources (RubyRepository).
suboptions:
name:
type: str
required: true
description: A short, unique name for this repository.
url:
type: str
required: true
description: The repository's URL.
nuget:
type: dict
default: {}
description: ConfigurationsBlock.nuget.
suboptions:
repositories:
type: list
elements: dict
default: []
description: NuGet package sources (NuGetRepository).
suboptions:
name:
type: str
required: true
description: A short, unique name for this repository.
url:
type: str
required: true
description: The repository's URL.
"""
EXAMPLES = r"""
- name: Apply an id-configurator configuration to this host
id_configurator:
uuid: 12345678-1234-5678-1234-567812345678
organisation: InterDiode
certificates:
- name: Company Root CA
content: "{{ lookup('file', 'files/company-root-ca.pem') }}"
apt:
sources:
- name: internal
uris: ["https://apt.internal.example/"]
suites: ["stable"]
"""
RETURN = r"""
applied_modules:
description: Names of the id-configurator modules that were (or, in check
mode, would be) applied, in registration order.
returned: always
type: list
elements: str
sample: [certificates, apt]
"""
# Every key here must match one of ConfigurationsBlock's own field names
# (id_configurator/config.py) - see the module docstring above.
_CONFIGURATIONS_BLOCK_FIELDS = (
"certificates",
"apt",
"apk",
"yum",
"python",
"java",
"php",
"npm",
"rust",
"go",
"ruby",
"nuget",
)
_NAMED_CONTENT_OPTIONS = {
"name": {"type": "str", "required": True},
"content": {"type": "str", "required": True},
}
_NAME_URL_OPTIONS = {
"name": {"type": "str", "required": True},
"url": {"type": "str", "required": True},
}
def _name_url_repository_config() -> dict:
"""Argument spec for a `{repositories: [{name, url}]}` config block,
shared by java/php/npm/rust/go/ruby/nuget."""
return {
"type": "dict",
"default": {},
"options": {
"repositories": {
"type": "list",
"elements": "dict",
"default": [],
"options": dict(_NAME_URL_OPTIONS),
},
},
}
ARGUMENT_SPEC = {
"version": {"type": "str", "default": "1.0"},
"uuid": {"type": "str"},
"organisation": {"type": "str"},
"description": {"type": "str"},
"certificates": {
"type": "list",
"elements": "dict",
"default": [],
"options": dict(_NAMED_CONTENT_OPTIONS),
},
"apt": {
"type": "dict",
"default": {},
"options": {
"sources": {
"type": "list",
"elements": "dict",
"default": [],
"options": {
"name": {"type": "str", "required": True},
"types": {"type": "list", "elements": "str", "default": ["deb"]},
"uris": {"type": "list", "elements": "str", "required": True},
"suites": {"type": "list", "elements": "str", "required": True},
"components": {"type": "list", "elements": "str", "default": []},
"architectures": {
"type": "list",
"elements": "str",
"default": [],
},
"languages": {"type": "list", "elements": "str", "default": []},
"targets": {"type": "list", "elements": "str", "default": []},
"enabled": {"type": "bool", "default": True},
"trusted": {"type": "bool"},
"signed_by": {"type": "str"},
"signed_by_file": {"type": "str"},
"pdiffs": {"type": "bool"},
"by_hash": {"type": "bool"},
"allow_insecure": {"type": "bool"},
"allow_weak": {"type": "bool"},
"allow_downgrade_to_insecure": {"type": "bool"},
"check_valid_until": {"type": "bool"},
"valid_until_min": {"type": "str"},
"valid_until_max": {"type": "str"},
"check_date": {"type": "bool"},
"date_max_future": {"type": "str"},
"description": {"type": "str"},
"uri_to_replace": {"type": "str"},
},
},
},
},
"apk": {
"type": "dict",
"default": {},
"options": {
"repositories": {
"type": "list",
"elements": "dict",
"default": [],
"options": {
"url": {"type": "str", "required": True},
"key": {"type": "str"},
"url_to_replace": {"type": "str"},
},
},
},
},
"yum": {
"type": "dict",
"default": {},
"options": {
"repositories": {
"type": "list",
"elements": "dict",
"default": [],
"options": {
"name": {"type": "str", "required": True},
"index_url": {"type": "str", "required": True},
"gpg_url": {"type": "str"},
"index_url_to_replace": {"type": "str"},
},
},
"keys": {
"type": "list",
"elements": "dict",
"default": [],
"options": dict(_NAMED_CONTENT_OPTIONS),
},
},
},
"python": {
"type": "dict",
"default": {},
"options": {
"repositories": {
"type": "list",
"elements": "dict",
"default": [],
"options": {
"name": {"type": "str", "required": True},
"index_url": {"type": "str", "required": True},
},
},
},
},
"java": _name_url_repository_config(),
"php": _name_url_repository_config(),
"npm": _name_url_repository_config(),
"rust": _name_url_repository_config(),
"go": _name_url_repository_config(),
"ruby": _name_url_repository_config(),
"nuget": _name_url_repository_config(),
}
try:
import id_configurator.modules # noqa: F401 (registers the built-in modules)
from id_configurator.config import Configuration
from id_configurator.context import Context
from id_configurator.modules.base import all_modules
HAS_ID_CONFIGURATOR = True
ID_CONFIGURATOR_IMPORT_ERROR = None
except ImportError as exc:
HAS_ID_CONFIGURATOR = False
ID_CONFIGURATOR_IMPORT_ERROR = exc
def _build_configuration_data(params: dict) -> dict:
return {
"version": params["version"],
"uuid": params["uuid"],
"organisation": params["organisation"],
"description": params["description"],
"configurations": {
field: params[field] for field in _CONFIGURATIONS_BLOCK_FIELDS
},
}
def _apply_configuration(configuration, check_mode: bool) -> list[str]:
"""Apply `configuration` to this host's live root filesystem, returning
the names of every module that was (or, in check mode, would be)
applied. Uses id-configurator's own module registry and `Context`
unchanged - this is a thin caller, not a reimplementation."""
root = Path("/")
context = Context(config=configuration, dst=root, src=root)
applied_modules = []
for module_cls in all_modules():
config_module = module_cls()
if config_module.is_relevant(context):
applied_modules.append(config_module.name)
if not check_mode:
config_module.apply(context)
return applied_modules
def main() -> None:
module = AnsibleModule(argument_spec=ARGUMENT_SPEC, supports_check_mode=True)
if not HAS_ID_CONFIGURATOR:
module.fail_json(
msg=(
"id_configurator is not importable with this host's Python "
f"interpreter: {ID_CONFIGURATOR_IMPORT_ERROR}"
)
)
return
try:
configuration = Configuration.model_validate(
_build_configuration_data(module.params)
)
except Exception as exc: # pydantic.ValidationError, kept generic on purpose
module.fail_json(msg=f"invalid id-configurator configuration: {exc}")
return
applied_modules = _apply_configuration(configuration, module.check_mode)
module.exit_json(changed=bool(applied_modules), applied_modules=applied_modules)
if __name__ == "__main__":
main()
+49 -6
View File
@@ -2,34 +2,77 @@
build-backend = "uv_build"
requires = ["uv_build>=0.12.3,<0.13.0"]
[dependency-groups]
dev = [
"jinja2>=3.1.6",
"tox>=4.60.1",
"tox-docker>=5.0.0",
"tox-uv>=1.36.0",
"typer>=0.27.1"
]
test = [
"pytest>=9.1.1",
"pytest-cov>=7.1.0"
]
[project]
authors = [
{name = "InterDiode", email = "contact@mail.interdiode.fr"}
]
classifiers = [
'Development Status :: 5 - Production/Stable',
'License :: Other/Proprietary License',
'Operating System :: POSIX :: Linux',
'Operating System :: Unix',
'Natural Language :: English',
"Private :: Do Not Upload",
'Programming Language :: Python :: 3 :: Only',
'Programming Language :: Python :: 3.13',
'Programming Language :: Python :: 3.14'
]
dependencies = []
dependencies = [
"cryptography>=50.0.1",
"pydantic>=2.13.4",
"pyjks>=20.0.0",
"pyyaml>=6.0.3",
"typer>=0.27.1"
]
description = "Configure a device in an air-gapped environment."
license = {text = "Proprietary"}
license = "CECILL-2.1"
license-files = ["LICENSE"]
maintainers = [
{name = "InterDiode", email = "contact@mail.interdiode.fr"}
]
name = "id-configurator"
readme = "README.md"
requires-python = ">=3.14"
version = "0.1.0"
requires-python = ">=3.13"
version = "2026.10.4"
[project.scripts]
id-configurator = "id_configurator:main"
[tool.coverage.paths]
source = [
"id_configurator",
"*/site-packages/id_configurator"
]
[tool.coverage.report]
exclude_lines = [
"pragma: no cover",
"if __name__ == .__main__.:"
]
fail_under = 100
show_missing = true
[tool.coverage.run]
branch = true
source = ["id_configurator"]
[tool.pytest.ini_options]
addopts = "--cov=id_configurator --cov-report=term-missing"
testpaths = ["tests"]
[tool.uv]
default-groups = ["dev", "test"]
[tool.uv.build-backend]
module-root = ""
+21
View File
@@ -0,0 +1,21 @@
-----BEGIN CERTIFICATE-----
MIIDdTCCAl2gAwIBAgIUKq4URHQwyg7IzJ3ViAGGlM9pGCwwDQYJKoZIhvcNAQEL
BQAwQjEeMBwGA1UECgwVaWQtY29uZmlndXJhdG9yLXRlc3RzMSAwHgYDVQQDDBdp
ZC1jb25maWd1cmF0b3IgVGVzdCBDQTAeFw0yNjA5MDUyMTA4MTNaFw00NjA4MzEy
MTA4MTNaMEIxHjAcBgNVBAoMFWlkLWNvbmZpZ3VyYXRvci10ZXN0czEgMB4GA1UE
AwwXaWQtY29uZmlndXJhdG9yIFRlc3QgQ0EwggEiMA0GCSqGSIb3DQEBAQUAA4IB
DwAwggEKAoIBAQD8bMteh3MDnbW6OoFMlhA4Z2QAabVoSFlSE/Si7ut+L/3v1GY4
lixenwLQc8a2koIH9CtmOodzfO7hpro/VUjYJ9J2eTsXLLiOg7OgggG1cKV9/kt7
RF0Q4hkcanGmiffc4xbz4ZedU6lP1vzLCsgU0A8OPt6svcNjGcWbjxss0KjgxGZ7
H4UKdL7Mv4W2Y6Fw1b4eV2M6jErkQK6IPXsHGFh7JkzliNcb0sX1VfbfauS84tBh
f3EHwzLNhbjtEmQhQdqgsg4Cbk/gWur//6aEvlD/X12PAUbrp4DFD+CDuVagLX7I
7TQBXNFGKhozzQuDg7DFg1gIV8SFx2m+s60/AgMBAAGjYzBhMB0GA1UdDgQWBBSb
zJ7DsaxwCeqbfOXGJ3x+u7bjozAfBgNVHSMEGDAWgBSbzJ7DsaxwCeqbfOXGJ3x+
u7bjozAPBgNVHRMBAf8EBTADAQH/MA4GA1UdDwEB/wQEAwIBBjANBgkqhkiG9w0B
AQsFAAOCAQEAEV1+dG3agh0S/AR8jC8M33HXFyvneCCRtfb5q/loJ//VoWPbD3Gs
G+2SR0MVN+KKOMBvCBh3KrNSgFF9iFmBjNCJJZ8ycDIszUK3xLjUP2EzN1pNNl8S
D/yHzUCro3547Q+I+Nn5zRciG95EpMvtkIWem4H8luYSBt7w8y9uDUKWzaca/v3z
oGZPZP99w8WVqMKGRbsK94wsTVsWiE+oHzSufwrhvZJfQQ4jU6102IEnRS/eQFXA
+fozgEWuFatZe3Gz0HUJakrzSBwSTV6wmnEHCfu+uwdns1Ttq5FylZ4StJHsmweL
14w1xa3tjC5bQ3e1QWK3R5InHcSZ7tLpGA==
-----END CERTIFICATE-----
+28
View File
@@ -0,0 +1,28 @@
-----BEGIN PRIVATE KEY-----
MIIEvAIBADANBgkqhkiG9w0BAQEFAASCBKYwggSiAgEAAoIBAQD8bMteh3MDnbW6
OoFMlhA4Z2QAabVoSFlSE/Si7ut+L/3v1GY4lixenwLQc8a2koIH9CtmOodzfO7h
pro/VUjYJ9J2eTsXLLiOg7OgggG1cKV9/kt7RF0Q4hkcanGmiffc4xbz4ZedU6lP
1vzLCsgU0A8OPt6svcNjGcWbjxss0KjgxGZ7H4UKdL7Mv4W2Y6Fw1b4eV2M6jErk
QK6IPXsHGFh7JkzliNcb0sX1VfbfauS84tBhf3EHwzLNhbjtEmQhQdqgsg4Cbk/g
Wur//6aEvlD/X12PAUbrp4DFD+CDuVagLX7I7TQBXNFGKhozzQuDg7DFg1gIV8SF
x2m+s60/AgMBAAECggEACGXaMjqRYf+/ia2OmwIBvM6mbg5rlvI+k9N5d745HslG
n9GngPRwuh581SXfk41g6Wt/J3cvsHDp8Enqt6wFBg6fxYns+a1m38mT5/V19xet
+c4ZrBqEeh5Q4DgjsHsLQFGxyXVU5PVhPvhpD/GQ4dhg19l8RlpIg3svrtLeVZRJ
T+jy7b0LTkRwe6D7AjE+1tqZr299JBLdH9dS7DeKA3rP7O6HzSEhrtiFxJBfq0v4
MDASAhXHytiqIErylb+ZiGSeqTYDLOJBhRcyrjuR1gNlFSxOa1+4iBVo5OVGTkYj
ZuT5r18xgJ0EjnMYd5RszWqfjzWGtd0oYyfN599Z+QKBgQD+2YM1JRAHrQORkjPm
Bz3JA6R2lDVbng/q1iJsdNUkxU0iKieLJ9KOiMRF9RU9QWIbuI9bvNxShudyZIiM
jqEUiUbGnRB5u7rtUOaA8ZlWtsxn3HGJ6C1iN8+IYhPzi3BFPRUHo8gUF+PgpmI3
EYXwto6d7uKh1ExBnebt/qnZEwKBgQD9kHrmlKUbrgR7OOLnhVGjB+5AX4fWbWcP
n2JOp19O5ZsQdgtP7eYDsdV7h3UGXbH1u53paXJUKneF53k3FtaG+XR6JPnKGk6Y
18Z5TvyXtdkgDuxPITHp0DmzJ7+1SmwsY2dMj8VAf08U7aCkURli1SpXx5a9tgly
WlURS1aspQKBgFXwW4KMYP91pIJuXvnu7zLd+5PRHr0qbAuw8iLIsyeXMQ+YT4Vp
hGN0U1uT7fmeFstqTU2OPm3F+D5nEHT1k/Qs5qULSgcC+2ITqedJkX/gXMWZXOA4
XzWDxIQlLr0uBEoU8waEgG/mLn4PHYecbSJjLWyspdgUkzJ9YOZYvkXpAoGAPMhL
jcaIJbB7RoaGbfVZcnlONYQw7cff8rDEQhKBGEqV90Y0w/5FKxGQGUKJXPNM25kT
64XpcB+V8zVfq0MAPe7Cax09+6J+wMjIvP+gMUXYPIUXRIiyjFfr3PRkUjL651EU
zBeSszlRXbTw0zLVoQl0UwJ3MhbXAOCCj9i9eDkCgYBf4XDR8Mi0rKHRL7NP8mYK
pKRmH2ABtupcVJn8GgUm8N5xaVsV7z9fW+YcrNQ2jbXogyMgwoyaUNa26mikb9em
i6DZCainr36d9UtO4aCMDOfapX18d68jt5FiEeiYs7r+mGZHrGmMFwxFJKmNdqhr
affX588Yx9EvfIb9tyk9KA==
-----END PRIVATE KEY-----
+36
View File
@@ -0,0 +1,36 @@
uuid: 3d2f6a44-8b19-4c2b-9e7a-1f5c8a2d6b90
organisation: InterDiode
description: Apk test configuration
configurations:
certificates:
- name: Test CA
content: |
{{ 'certs/test-ca.crt.pem' | file | indent(6, first=true) }}
apt:
sources: []
apk:
repositories:
- name: demo
url: https://host.docker.internal:{{ 'APK_REPO_PORT' | env }}/
url_to_replace: https://dl-cdn.alpinelinux.org/*
key: |
{{ 'dockerfiles/repos/040d0af5f97a0a44.rsa.pub' | file | indent(8, first=true) }}
yum:
repositories: []
keys: []
python:
repositories: []
java:
repositories: []
php:
repositories: []
npm:
repositories: []
rust:
repositories: []
go:
repositories: []
ruby:
repositories: []
nuget:
repositories: []
+39
View File
@@ -0,0 +1,39 @@
uuid: 0147050d-d045-4719-b084-e2b1f337290e
organisation: InterDiode
description: Apt test configuration
configurations:
certificates:
- name: Test CA
content: |
{{ 'certs/test-ca.crt.pem' | file | indent(6, first=true) }}
apt:
sources:
- name: demo
types: [deb]
uris:
- https://host.docker.internal:{{ 'APT_REPO_PORT' | env }}/
suites: ["./"]
uri-to-replace: http://deb.debian.org/*
signed-by: |
{{ 'dockerfiles/repos/apt-signing-key.pub.asc' | file | indent(8, first=true) }}
apk:
repositories: []
yum:
repositories: []
keys: []
python:
repositories: []
java:
repositories: []
php:
repositories: []
npm:
repositories: []
rust:
repositories: []
go:
repositories: []
ruby:
repositories: []
nuget:
repositories: []
+31
View File
@@ -0,0 +1,31 @@
uuid: 200f3290-3a3f-47ac-8149-dbe26c82ee34
organisation: InterDiode
description: Certificate test configuration
configurations:
certificates:
- name: Test CA
content: |
{{ 'certs/test-ca.crt.pem' | file | indent(6, first=true) }}
apt:
sources: []
apk:
repositories: []
yum:
repositories: []
keys: []
python:
repositories: []
java:
repositories: []
php:
repositories: []
npm:
repositories: []
rust:
repositories: []
go:
repositories: []
ruby:
repositories: []
nuget:
repositories: []
+33
View File
@@ -0,0 +1,33 @@
uuid: 3f1a2b4c-5d6e-4f70-8a91-2b3c4d5e6f70
organisation: InterDiode
description: Go test configuration
configurations:
certificates:
- name: Test CA
content: |
{{ 'certs/test-ca.crt.pem' | file | indent(6, first=true) }}
apt:
sources: []
apk:
repositories: []
yum:
repositories: []
keys: []
python:
repositories: []
java:
repositories: []
php:
repositories: []
npm:
repositories: []
rust:
repositories: []
go:
repositories:
- name: demo
url: https://host.docker.internal:{{ 'GO_REPO_PORT' | env }}/
ruby:
repositories: []
nuget:
repositories: []
+46
View File
@@ -0,0 +1,46 @@
uuid: 6a1d4e2c-9f3b-4a5d-8c7e-2b1f9a6d4c3e
organisation: InterDiode
description: Java test configuration
configurations:
certificates:
- name: Test CA
content: |
{{ 'certs/test-ca.crt.pem' | file | indent(6, first=true) }}
apt:
sources: []
apk:
repositories: []
yum:
repositories: []
keys: []
python:
repositories: []
java:
repositories:
# First entry: named "central" so Maven's mirrorOf=* (built on
# repositories[0]) redirects everything here without invalidating the
# maven-dependency-plugin already cached under that repository id
# during the Dockerfile's warm-up step. All other tools just see it as
# an ordinary repository, and find the test artifact here.
- name: central
url: https://host.docker.internal:{{ 'JAVA_REPO_PORT' | env }}/
# Second entry: only SBT needs this. With
# -Dsbt.override.build.repos=true (the only way SBT actually consults
# this file for project dependencies, not just its own bootstrap), the
# repositories listed here become the *only* ones available - without a
# real Maven Central mirror among them, SBT can't resolve scala-library
# itself. Maven ignores it (mirrorOf=* already redirects everything to
# the first entry); Gradle/Ivy/Leiningen just see one more unused
# resolver, since the test artifact is already found via "central".
- name: maven-central-fallback
url: https://repo1.maven.org/maven2/
npm:
repositories: []
rust:
repositories: []
go:
repositories: []
ruby:
repositories: []
nuget:
repositories: []
+33
View File
@@ -0,0 +1,33 @@
uuid: f8348fe2-3f69-4eee-9c3d-d5f092c9ff80
organisation: InterDiode
description: Npm test configuration
configurations:
certificates:
- name: Test CA
content: |
{{ 'certs/test-ca.crt.pem' | file | indent(6, first=true) }}
apt:
sources: []
apk:
repositories: []
yum:
repositories: []
keys: []
python:
repositories: []
java:
repositories: []
php:
repositories: []
npm:
repositories:
- name: demo
url: https://host.docker.internal:{{ 'NPM_REPO_PORT' | env }}/
rust:
repositories: []
go:
repositories: []
ruby:
repositories: []
nuget:
repositories: []
+33
View File
@@ -0,0 +1,33 @@
uuid: 5b3c4d6e-7f80-4192-8b13-4d5e6f708192
organisation: InterDiode
description: NuGet test configuration
configurations:
certificates:
- name: Test CA
content: |
{{ 'certs/test-ca.crt.pem' | file | indent(6, first=true) }}
apt:
sources: []
apk:
repositories: []
yum:
repositories: []
keys: []
python:
repositories: []
java:
repositories: []
php:
repositories: []
npm:
repositories: []
rust:
repositories: []
go:
repositories: []
ruby:
repositories: []
nuget:
repositories:
- name: demo
url: https://host.docker.internal:{{ 'NUGET_REPO_PORT' | env }}/index.json
+33
View File
@@ -0,0 +1,33 @@
uuid: 7135f49c-44c4-4583-8512-2ed5c6ee726b
organisation: InterDiode
description: Php test configuration
configurations:
certificates:
- name: Test CA
content: |
{{ 'certs/test-ca.crt.pem' | file | indent(6, first=true) }}
apt:
sources: []
apk:
repositories: []
yum:
repositories: []
keys: []
python:
repositories: []
java:
repositories: []
php:
repositories:
- name: demo
url: https://host.docker.internal:{{ 'PHP_REPO_PORT' | env }}/
npm:
repositories: []
rust:
repositories: []
go:
repositories: []
ruby:
repositories: []
nuget:
repositories: []
+33
View File
@@ -0,0 +1,33 @@
uuid: 8453d139-cbe5-43b0-ab35-4468042b8c7e
organisation: InterDiode
description: Python test configuration
configurations:
certificates:
- name: Test CA
content: |
{{ 'certs/test-ca.crt.pem' | file | indent(6, first=true) }}
apt:
sources: []
apk:
repositories: []
yum:
repositories: []
keys: []
python:
repositories:
- name: demo
index-url: https://host.docker.internal:{{ 'PYPI_REPO_PORT' | env }}/simple/
java:
repositories: []
php:
repositories: []
npm:
repositories: []
rust:
repositories: []
go:
repositories: []
ruby:
repositories: []
nuget:
repositories: []
+33
View File
@@ -0,0 +1,33 @@
uuid: 4a2b3c5d-6e7f-4081-9a02-3c4d5e6f7081
organisation: InterDiode
description: Ruby test configuration
configurations:
certificates:
- name: Test CA
content: |
{{ 'certs/test-ca.crt.pem' | file | indent(6, first=true) }}
apt:
sources: []
apk:
repositories: []
yum:
repositories: []
keys: []
python:
repositories: []
java:
repositories: []
php:
repositories: []
npm:
repositories: []
rust:
repositories: []
go:
repositories: []
ruby:
repositories:
- name: demo
url: https://host.docker.internal:{{ 'RUBY_REPO_PORT' | env }}/
nuget:
repositories: []
+33
View File
@@ -0,0 +1,33 @@
uuid: 63db4c0e-a4c4-402c-bc4f-4eddeef0d890
organisation: InterDiode
description: Rust test configuration
configurations:
certificates:
- name: Test CA
content: |
{{ 'certs/test-ca.crt.pem' | file | indent(6, first=true) }}
apt:
sources: []
apk:
repositories: []
yum:
repositories: []
keys: []
python:
repositories: []
java:
repositories: []
php:
repositories: []
npm:
repositories: []
rust:
repositories:
- name: demo
url: https://host.docker.internal:{{ 'CARGO_REPO_PORT' | env }}/
go:
repositories: []
ruby:
repositories: []
nuget:
repositories: []
+34
View File
@@ -0,0 +1,34 @@
uuid: 8f6a0e2a-5b7a-4d2c-9d1a-3c2b6b9c4b7e
organisation: InterDiode
description: Yum test configuration
configurations:
certificates:
- name: Test CA
content: |
{{ 'certs/test-ca.crt.pem' | file | indent(6, first=true) }}
apt:
sources: []
apk:
repositories: []
yum:
repositories:
- name: demo
index-url: https://host.docker.internal:{{ 'YUM_REPO_PORT' | env }}/
index-url-to-replace: https://mirrors.rockylinux.org/*
keys: []
python:
repositories: []
java:
repositories: []
php:
repositories: []
npm:
repositories: []
rust:
repositories: []
go:
repositories: []
ruby:
repositories: []
nuget:
repositories: []
+10
View File
@@ -0,0 +1,10 @@
FROM alpine:3.20 AS target
FROM interdiode/id-configurator:latest AS configuration
COPY --from=target / /src/
COPY --from=generated-config Apk.yaml /config.yaml
RUN id-configurator apply --config /config.yaml --src /src --dst /dst
FROM target AS runtime
COPY --from=configuration /dst/ /
RUN apk update \
&& apk add idconfiguratortestpkg \
&& apk info idconfiguratortestpkg
+10
View File
@@ -0,0 +1,10 @@
FROM debian:trixie-slim AS target
FROM interdiode/id-configurator:latest AS configuration
COPY --from=target / /src/
COPY --from=generated-config Apt.yaml /config.yaml
RUN id-configurator apply --config /config.yaml --src /src --dst /dst
FROM target AS runtime
COPY --from=configuration /dst/ /
RUN apt-get update \
&& apt-get install -y --no-install-recommends idconfiguratortestpkg \
&& dpkg -l idconfiguratortestpkg
+15
View File
@@ -0,0 +1,15 @@
FROM debian:trixie-slim AS target
FROM interdiode/id-configurator:latest AS configuration
COPY --from=target / /src/
COPY --from=generated-config Certificate.yaml /config.yaml
RUN id-configurator apply --config /config.yaml --src /src --dst /dst
FROM target AS runtime
ARG CERTIFICATE_REPO_PORT
RUN apt-get update && apt-get install -y --no-install-recommends wget curl git openssl
COPY --from=configuration /dst/ /
RUN wget -qO- "https://host.docker.internal:${CERTIFICATE_REPO_PORT}/marker.txt" | grep -qx trusted-via-root-ca
RUN curl -fsS "https://host.docker.internal:${CERTIFICATE_REPO_PORT}/marker.txt" | grep -qx trusted-via-root-ca
RUN git clone -q "https://host.docker.internal:${CERTIFICATE_REPO_PORT}/repo.git" /tmp/clone \
&& grep -qx trusted-via-root-ca /tmp/clone/marker.txt
RUN echo | openssl s_client -connect "host.docker.internal:${CERTIFICATE_REPO_PORT}" \
-servername host.docker.internal -verify_hostname host.docker.internal -verify_return_error
+19
View File
@@ -0,0 +1,19 @@
FROM debian:trixie-slim AS target
FROM interdiode/id-configurator:latest AS configuration
ARG CONFIG_REPO_PORT
ARG CONFIG_URL_CACERT_CONTENT
COPY --from=target / /src/
RUN id-configurator apply \
--config-url "https://host.docker.internal:${CONFIG_REPO_PORT}/config.yaml" \
--config-url-cacert-content "$CONFIG_URL_CACERT_CONTENT" \
--src /src --dst /dst
FROM target AS runtime
ARG CONFIG_REPO_PORT
RUN apt-get update && apt-get install -y --no-install-recommends curl
COPY --from=configuration /dst/ /
# Proves the root CA the downloaded configuration installed is actually
# trusted system-wide: no --insecure/--no-check-certificate here, unlike the
# --config-url-cacert-content fetch above (which trusts only that one CA to
# validate the download itself, before anything has been installed yet).
RUN curl -fsS "https://host.docker.internal:${CONFIG_REPO_PORT}/config.yaml" \
| grep -q 'organisation: InterDiode'
+11
View File
@@ -0,0 +1,11 @@
FROM golang:1-bookworm AS target
FROM interdiode/id-configurator:latest AS configuration
COPY --from=target / /src/
COPY --from=generated-config Go.yaml /config.yaml
RUN id-configurator apply --config /config.yaml --src /src --dst /dst
FROM target AS runtime
COPY --from=configuration /dst/ /
RUN mkdir -p /tmp/gotest && cd /tmp/gotest && go mod init gotest \
&& go get example.invalid/idconfiguratortestpkg@v0.0.1 \
&& grep -qx 'test-only module served by the id-configurator integration tests' \
"$(find "$(go env GOMODCACHE)" -name marker.txt)"
+60
View File
@@ -0,0 +1,60 @@
FROM maven:3.9-eclipse-temurin-21 AS target
FROM interdiode/id-configurator:latest AS configuration
COPY --from=target / /src/
COPY --from=generated-config Java.yaml /config.yaml
RUN id-configurator apply --config /config.yaml --src /src --dst /dst
FROM target AS runtime
# Every build tool the `java` module configures: Maven itself (already part
# of this image), Gradle, Ivy and Leiningen (all packaged for this Ubuntu
# base), and SBT (no package available, fetched as a plain release tarball).
RUN apt-get update \
&& apt-get install -y --no-install-recommends gradle ivy leiningen curl \
&& curl -sL https://github.com/sbt/sbt/releases/download/v1.10.1/sbt-1.10.1.tgz -o /tmp/sbt.tgz \
&& tar -xzf /tmp/sbt.tgz -C /opt \
&& rm /tmp/sbt.tgz
ENV PATH="/opt/sbt/bin:${PATH}"
# Warm up every tool's own bootstrap/plugin dependencies from the real,
# network-reachable Maven Central while the default resolvers are still in
# place, so the actual tests below never need anything but the fixture
# repository configured by id-configurator (plus, for SBT only, the
# maven-central-fallback repository also configured below).
RUN mvn -q org.apache.maven.plugins:maven-dependency-plugin:3.6.1:get -Dartifact=org.apache.commons:commons-lang3:3.12.0
RUN mkdir -p /opt/sbt-warmup/project \
&& echo 'scalaVersion := "2.13.14"' > /opt/sbt-warmup/build.sbt \
&& echo 'sbt.version=1.10.1' > /opt/sbt-warmup/project/build.properties \
&& cd /opt/sbt-warmup && sbt update
COPY --from=configuration /dst/ /
RUN mvn org.apache.maven.plugins:maven-dependency-plugin:3.6.1:get -Dartifact=org.interdiode:idconfiguratortestpkg:0.0.1 \
&& test -f /root/.m2/repository/org/interdiode/idconfiguratortestpkg/0.0.1/idconfiguratortestpkg-0.0.1.jar
RUN mkdir -p /tmp/gradletest && cd /tmp/gradletest \
&& cat > build.gradle <<'EOF'
configurations { deps }
dependencies { deps 'org.interdiode:idconfiguratortestpkg:0.0.1' }
task resolveDeps { doLast { configurations.deps.each { println it } } }
EOF
RUN cd /tmp/gradletest && gradle -q resolveDeps | grep -q idconfiguratortestpkg
RUN java -jar /usr/share/java/ivy.jar -settings /root/.ivy2/ivysettings.xml \
-dependency org.interdiode idconfiguratortestpkg 0.0.1 \
-retrieve "/tmp/ivyretrieve/[artifact]-[revision].[ext]" \
&& test -f /tmp/ivyretrieve/idconfiguratortestpkg-0.0.1.jar
RUN mkdir -p /tmp/leintest && cd /tmp/leintest \
&& cat > project.clj <<'EOF'
(defproject leintest "0.1.0"
:dependencies [[org.interdiode/idconfiguratortestpkg "0.0.1"]])
EOF
RUN cd /tmp/leintest && lein deps :tree | grep -q idconfiguratortestpkg
RUN mkdir -p /tmp/sbttest/project \
&& cat > /tmp/sbttest/build.sbt <<'EOF'
scalaVersion := "2.13.14"
libraryDependencies += "org.interdiode" % "idconfiguratortestpkg" % "0.0.1"
EOF
RUN echo 'sbt.version=1.10.1' > /tmp/sbttest/project/build.properties
RUN cd /tmp/sbttest \
&& sbt -Dsbt.override.build.repos=true "show Compile/dependencyClasspath" | grep -q idconfiguratortestpkg
+12
View File
@@ -0,0 +1,12 @@
FROM node:20-slim AS target
FROM interdiode/id-configurator:latest AS configuration
COPY --from=target / /src/
COPY --from=generated-config Npm.yaml /config.yaml
RUN id-configurator apply --config /config.yaml --src /src --dst /dst
FROM target AS runtime
COPY --from=configuration /dst/ /
RUN mkdir -p /tmp/npmtest && cd /tmp/npmtest \
&& npm init -y >/dev/null \
&& npm install idconfiguratortestpkg \
&& grep -qx 'test-only package served by the id-configurator integration tests' \
node_modules/idconfiguratortestpkg/marker.txt
+11
View File
@@ -0,0 +1,11 @@
FROM mcr.microsoft.com/dotnet/sdk:9.0 AS target
FROM interdiode/id-configurator:latest AS configuration
COPY --from=target / /src/
COPY --from=generated-config NuGet.yaml /config.yaml
RUN id-configurator apply --config /config.yaml --src /src --dst /dst
FROM target AS runtime
COPY --from=configuration /dst/ /
RUN mkdir -p /tmp/nugettest && cd /tmp/nugettest && dotnet new console -o . --no-restore >/dev/null \
&& dotnet add package idconfiguratortestpkg --version 0.0.1 \
&& grep -qx 'test-only package served by the id-configurator integration tests' \
"$(find /root/.nuget/packages/idconfiguratortestpkg -name marker.txt)"
+16
View File
@@ -0,0 +1,16 @@
FROM php:8.3-cli AS target
FROM interdiode/id-configurator:latest AS configuration
COPY --from=target / /src/
COPY --from=generated-config Php.yaml /config.yaml
RUN id-configurator apply --config /config.yaml --src /src --dst /dst
FROM target AS runtime
RUN apt-get update \
&& apt-get install -y --no-install-recommends ca-certificates unzip curl \
&& curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer \
&& rm -rf /var/lib/apt/lists/*
COPY --from=configuration /dst/ /
RUN mkdir -p /tmp/phptest && cd /tmp/phptest \
&& composer init -n --name test/phptest \
&& composer require interdiode/idconfiguratortestpkg \
&& grep -qx 'test-only package served by the id-configurator integration tests' \
vendor/interdiode/idconfiguratortestpkg/marker.txt
@@ -0,0 +1,24 @@
FROM python:3.13-alpine AS target
FROM interdiode/id-configurator:latest AS configuration
COPY --from=target / /src/
COPY --from=generated-config Python.yaml /config.yaml
RUN id-configurator apply --config /config.yaml --src /src --dst /dst
FROM target AS runtime
ARG PYPI_REPO_PORT
# Warm up uv and poetry themselves from the real, network-reachable PyPI
# while the default index is still in place, so the actual tests below
# never need anything but the fixture repository configured by
# id-configurator. (easy_install/distutils, the fourth tool the `python`
# module configures, is not exercised here: both were removed from Python
# itself in 3.12, and setuptools no longer ships easy_install either, so
# there is no way to actually invoke it on a current interpreter.)
RUN pip install --no-cache-dir uv poetry
COPY --from=configuration /dst/ /
RUN pip install --no-cache-dir idconfiguratortestpkg \
&& python3 -c "import idconfiguratortestpkg"
RUN uv pip install --system idconfiguratortestpkg \
&& python3 -c "import idconfiguratortestpkg"
RUN mkdir -p /tmp/poetrytest && cd /tmp/poetrytest \
&& poetry init -n --name poetrytest \
&& poetry source add --priority explicit demo "https://host.docker.internal:${PYPI_REPO_PORT}/simple/" \
&& poetry add idconfiguratortestpkg --source demo
+24
View File
@@ -0,0 +1,24 @@
FROM python:3.13-slim AS target
FROM interdiode/id-configurator:latest AS configuration
COPY --from=target / /src/
COPY --from=generated-config Python.yaml /config.yaml
RUN id-configurator apply --config /config.yaml --src /src --dst /dst
FROM target AS runtime
ARG PYPI_REPO_PORT
# Warm up uv and poetry themselves from the real, network-reachable PyPI
# while the default index is still in place, so the actual tests below
# never need anything but the fixture repository configured by
# id-configurator. (easy_install/distutils, the fourth tool the `python`
# module configures, is not exercised here: both were removed from Python
# itself in 3.12, and setuptools no longer ships easy_install either, so
# there is no way to actually invoke it on a current interpreter.)
RUN pip install --no-cache-dir uv poetry
COPY --from=configuration /dst/ /
RUN pip install --no-cache-dir idconfiguratortestpkg \
&& python3 -c "import idconfiguratortestpkg"
RUN uv pip install --system idconfiguratortestpkg \
&& python3 -c "import idconfiguratortestpkg"
RUN mkdir -p /tmp/poetrytest && cd /tmp/poetrytest \
&& poetry init -n --name poetrytest \
&& poetry source add --priority explicit demo "https://host.docker.internal:${PYPI_REPO_PORT}/simple/" \
&& poetry add idconfiguratortestpkg --source demo
+10
View File
@@ -0,0 +1,10 @@
FROM ruby:3-slim AS target
FROM interdiode/id-configurator:latest AS configuration
COPY --from=target / /src/
COPY --from=generated-config Ruby.yaml /config.yaml
RUN id-configurator apply --config /config.yaml --src /src --dst /dst
FROM target AS runtime
COPY --from=configuration /dst/ /
RUN gem install idconfiguratortestpkg --no-document \
&& grep -qx 'test-only gem served by the id-configurator integration tests' \
"$(find "$(gem environment gemdir)" -name marker.txt)"
+16
View File
@@ -0,0 +1,16 @@
FROM rust:1-slim AS target
FROM interdiode/id-configurator:latest AS configuration
COPY --from=target / /src/
COPY --from=generated-config Rust.yaml /config.yaml
RUN id-configurator apply --config /config.yaml --src /src --dst /dst
FROM target AS runtime
COPY --from=configuration /dst/ /
# Deliberately not touching CARGO_HOME (the official image sets it to
# /usr/local/cargo): id-configurator writes /.cargo/config.toml, which
# cargo's config search always consults regardless of CARGO_HOME, precisely
# so this doesn't need to line up with wherever CARGO_HOME happens to point.
RUN cargo new /tmp/rusttest -q && cd /tmp/rusttest \
&& cargo add idconfiguratortestpkg \
&& cargo fetch \
&& grep -qx '// test-only crate served by the id-configurator integration tests' \
"$(find "$CARGO_HOME/registry/src" -name lib.rs)"
+9
View File
@@ -0,0 +1,9 @@
FROM rockylinux:9 AS target
FROM interdiode/id-configurator:latest AS configuration
COPY --from=target / /src/
COPY --from=generated-config Yum.yaml /config.yaml
RUN id-configurator apply --config /config.yaml --src /src --dst /dst
FROM target AS runtime
COPY --from=configuration /dst/ /
RUN dnf install -y idconfiguratortestpkg \
&& rpm -q idconfiguratortestpkg
@@ -0,0 +1,28 @@
-----BEGIN PRIVATE KEY-----
MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQC/MKMygUuccuON
KW9N1VvuVgLnUPRMpCg03Qp3yk2HtctbdHVKKFtgjm88u4tzLMl0p+b+x65xJhPn
veEres0iz0oYgbPWMaRNmPP5ahg8NJ6ZPQaRCYrOfFGS2cAH4/NiGu31aFd3vgnj
JqSztlpZ+VXr2OQnaUHUutH1ZSmHau9R7ZbuxYeAm9VMhHFfKe5rVOwObBR4HaBy
cgF1uF3eX4jlxKOECbDntaEkAEfszSiujvuT1nFw9kP/ImtTWGI9w7nJzhY7rFRN
LnTSapjQS8Ju6HHC5EafolPVvV52NI2+J9504Es2K/GoLIX3OmxSbayf9Qp0zx15
7FQ2vDm9AgMBAAECggEAVXIdxaEeUzXPd9OoUF9F/j8Hfizieew1oeaV2pKZ0siP
MXjbQIDH0d/mW/yvZpKCGHY/mOdSEXSEmeH1I7L2whNlWI2C+3vlI97zTqk5WpLj
vLWW3TQoCn9qD5jspzobmGFTrwVqbfQLRC8NldmuCJ8T5NtioDB9AN9bnXIQPNno
h2Oo6HIeX2kL5DlqIezj42nmlFegxKhxVbDiGl3qUUbKmA3H9PodAN2ZQ9fwoFws
wswdMSord+Ajfcab/Y8wUzIELDZm0PcxvN/nSrtRQ121Jy85DE5ZmX3IonwmsiWV
i+maYv2tTrNHbeCxshSlVWFOHuaRstd6yY52wC1lSwKBgQD/H4xedtGprQs0Or2y
1qoF+A+KzMocikIcWBjTr0a0IMrufeo8IwPMeNCHj3KoJeKVsOvFA4JEbo5rFiRU
W96hsJ0W0U3wM5ImTMbmZyGayYcADf4CEOoAscKqt4Z/CphJcNzXeyZardCrZvL8
kF15TB40EoCRbCYEthZE7/bAGwKBgQC/2NeWnBo2QanMd5pZV+hVs6iVL1iZ5Mz3
whqigSejubvZLznBo17KWQwtQ5b3wkteMe19FA9ZUXBudT5Q04Qr8MQRksDy+wN2
N83Lf+3IV3oiWBE4MaBw2rdEHMUKmo6CARxr0hWALlxmmxMFlQJRwavfq1GsGriL
TWgltRB7BwKBgCtjqDHgr2PWJFOrsduamqwNNkd68NzNvtSnPwGxi6rS8PCCxbWY
lurNmVmU0gE7AEBJKLAHGr6ctugLif426bRCH1VkH4c8nhyZbjJ9U7/JioY92vdq
/QFFOkBimiM9IA1NfqG/NhSx/w1HowfFXcBla7Wbd/ZDmGuwtDThQuwFAoGBAK9F
yjl0ZtI5DYubne2QTdi2qgeaWQVIryaQNMtQ2klP8FpNNiQ+QOT3lBNqk+vJnBfz
33iNsjQ1HlYcUZGAnVmTv14ayKN1R0VjWGZae4C0wwXLlffUBzPz7smQy0iTyizm
HPQ43KHeOVcYyLkp2ROesItJtvVY+E90qfkBQoxTAoGBAMfGmprXptsq9I1J1gME
p+uVV+4LkPCURP0r+HdIFtD5RRcJlOudEsl5k3hBPYVsrWp9SpA9+05Sa8GSkqkn
qCZdP/nmlo8SruqgVRpd1zj5hWlhtLOW6+pVog2YXGDlhc6BKI8yQiIvih8ou5p2
Mf+TBaqUBGTbIqBDv0yUvcQj
-----END PRIVATE KEY-----
@@ -0,0 +1,9 @@
-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAvzCjMoFLnHLjjSlvTdVb
7lYC51D0TKQoNN0Kd8pNh7XLW3R1SihbYI5vPLuLcyzJdKfm/seucSYT573hK3rN
Is9KGIGz1jGkTZjz+WoYPDSemT0GkQmKznxRktnAB+PzYhrt9WhXd74J4yaks7Za
WflV69jkJ2lB1LrR9WUph2rvUe2W7sWHgJvVTIRxXynua1TsDmwUeB2gcnIBdbhd
3l+I5cSjhAmw57WhJABH7M0oro77k9ZxcPZD/yJrU1hiPcO5yc4WO6xUTS500mqY
0EvCbuhxwuRGn6JT1b1edjSNvifedOBLNivxqCyF9zpsUm2sn/UKdM8deexUNrw5
vQIDAQAB
-----END PUBLIC KEY-----
+13
View File
@@ -0,0 +1,13 @@
pkgname=idconfiguratortestpkg
pkgver=0.0.1
pkgrel=0
pkgdesc="Test-only package served by the id-configurator integration tests"
url="https://example.invalid"
arch="noarch"
license="MIT"
options="!check"
package() {
mkdir -p "$pkgdir"/usr/share/doc/"$pkgname"
echo test-only-package > "$pkgdir"/usr/share/doc/"$pkgname"/marker
}
@@ -0,0 +1,12 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net9.0</TargetFramework>
<PackageId>idconfiguratortestpkg</PackageId>
<Version>0.0.1</Version>
<Authors>InterDiode</Authors>
<Description>test-only package served by the id-configurator integration tests</Description>
</PropertyGroup>
<ItemGroup>
<None Include="marker.txt" Pack="true" PackagePath="content/marker.txt" />
</ItemGroup>
</Project>
@@ -0,0 +1,46 @@
# Minimal Alpine (apk) repository with a signed index, serving one hand-built
# .apk package over TLS, using the same Test CA as the pypi-repo/apt-repo/
# yum-repo fixtures, used by tox-docker to exercise the `apk` module
# integration test (tests/configs/Apk.yaml.j2,
# tests/dockerfiles/Apk.Dockerfile). The package is signed with the static
# key pair 040d0af5f97a0a44.rsa / 040d0af5f97a0a44.rsa.pub, whose public
# half is what the `apk` module configuration is expected to install into
# /etc/apk/keys.
#
# The key's filename matters, not just its content: apk embeds the public
# key's *filename* (not just a verifiable signature) into the index it
# signs, and only accepts it if a key file with that exact name is present
# in /etc/apk/keys - trying other installed keys does not help. The name
# 040d0af5f97a0a44 is the id_configurator.modules.apk._key_filename() hash
# of this specific public key's content (see tests/configs/Apk.yaml.tmpl),
# i.e. the name the `apk` module will actually use when it installs this key
# from the configuration. Regenerating the key pair requires recomputing
# that hash and renaming both files (and updating this comment) to match.
#
# Built by tox-docker, whose build context is this directory (not the repo
# root) - every COPY source below must live under tests/dockerfiles/repos/.
FROM alpine:3.20 AS packagebuild
RUN apk add --no-cache abuild build-base
RUN adduser -D builder && addgroup builder abuild
COPY 040d0af5f97a0a44.rsa 040d0af5f97a0a44.rsa.pub /home/builder/.abuild/
COPY 040d0af5f97a0a44.rsa.pub /etc/apk/keys/040d0af5f97a0a44.rsa.pub
RUN printf 'PACKAGER_PRIVKEY="/home/builder/.abuild/040d0af5f97a0a44.rsa"\n' > /home/builder/.abuild/abuild.conf \
&& chown -R builder:builder /home/builder
COPY --chown=builder:builder APKBUILD /home/builder/pkg/APKBUILD
RUN su builder -c 'cd /home/builder/pkg && abuild checksum && abuild -r'
# apk always appends the *client's* architecture to the repository URL, so
# the built index/package land in a directory named after whatever
# architecture this image was actually built for (native to the Docker host
# unless emulated). A flat copy of the index at the repo root lets the
# healthcheck probe it without knowing that architecture name in advance.
RUN arch_dir=$(ls /home/builder/packages/builder) \
&& mkdir -p /srv \
&& cp -r "/home/builder/packages/builder/$arch_dir" "/srv/$arch_dir" \
&& cp "/home/builder/packages/builder/$arch_dir/APKINDEX.tar.gz" /srv/APKINDEX.tar.gz
FROM nginx:alpine
COPY --from=packagebuild /srv /srv
COPY repo-server.crt.pem /etc/nginx/certs/repo-server.crt.pem
COPY repo-server.key.pem /etc/nginx/certs/repo-server.key.pem
COPY apk-repo.nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 8443
@@ -0,0 +1,10 @@
server {
listen 8443 ssl default_server;
server_name _;
ssl_certificate /etc/nginx/certs/repo-server.crt.pem;
ssl_certificate_key /etc/nginx/certs/repo-server.key.pem;
root /srv;
autoindex off;
}
@@ -0,0 +1,40 @@
# Minimal flat-format APT repository serving one hand-built .deb package over
# TLS, used by tox-docker to exercise the `apt` module integration test
# (tests/configs/Apt.yaml.j2, tests/dockerfiles/Apt.Dockerfile). Its
# InRelease file is genuinely signed with the dedicated key pair
# apt-signing-key.key.asc / apt-signing-key.pub.asc, whose public half is
# what tests/configs/Apt.yaml.j2 embeds as the configured source's
# `signed-by`: the `apt` module dearmors it into a keyring, and APT itself
# verifies the InRelease signature against that keyring on `apt-get update`
# - no `Trusted: yes` bypass involved.
#
# Built by tox-docker, whose build context is this directory (not the repo
# root) - every COPY source below must live under tests/dockerfiles/repos/.
FROM debian:trixie-slim AS packagebuild
RUN apt-get update \
&& apt-get install -y --no-install-recommends dpkg-dev apt-utils gnupg \
&& rm -rf /var/lib/apt/lists/*
RUN set -eux; \
pkg=idconfiguratortestpkg; \
root=/build/$pkg; \
mkdir -p "$root/DEBIAN" "$root/usr/share/doc/$pkg"; \
printf 'test-only package served by the id-configurator integration tests\n' > "$root/usr/share/doc/$pkg/marker"; \
printf 'Package: %s\nVersion: 0.0.1\nArchitecture: all\nMaintainer: InterDiode <contact@mail.interdiode.fr>\nDescription: Test-only package served by the id-configurator integration tests\n' "$pkg" > "$root/DEBIAN/control"; \
dpkg-deb --root-owner-group --build "$root" "/srv/${pkg}_0.0.1_all.deb"
WORKDIR /srv
RUN dpkg-scanpackages --multiversion . > Packages \
&& gzip -9c Packages > Packages.gz
COPY apt-signing-key.key.asc /tmp/apt-signing-key.key.asc
RUN gnupg_home="$(mktemp -d)" \
&& gpg --homedir "$gnupg_home" --batch --import /tmp/apt-signing-key.key.asc \
&& key_id="$(gpg --homedir "$gnupg_home" --list-secret-keys --with-colons | awk -F: '/^sec/{print $5; exit}')" \
&& apt-ftparchive release . > Release \
&& gpg --homedir "$gnupg_home" --batch --yes --pinentry-mode loopback --default-key "$key_id" --clearsign -o InRelease Release \
&& rm -rf "$gnupg_home" /tmp/apt-signing-key.key.asc
FROM nginx:alpine
COPY --from=packagebuild /srv /srv
COPY repo-server.crt.pem /etc/nginx/certs/repo-server.crt.pem
COPY repo-server.key.pem /etc/nginx/certs/repo-server.key.pem
COPY apt-repo.nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 8443
@@ -0,0 +1,10 @@
server {
listen 8443 ssl default_server;
server_name _;
ssl_certificate /etc/nginx/certs/repo-server.crt.pem;
ssl_certificate_key /etc/nginx/certs/repo-server.key.pem;
root /srv;
autoindex off;
}
@@ -0,0 +1,33 @@
-----BEGIN PGP PRIVATE KEY BLOCK-----
lQOYBGqg6rMBCADaMUd27S41+kInPXuRZ0aL41aMZjqr+mw4W34w8i/JRYMJ6eGb
sXf40AaFfS7zkbUpQZpmh3aYPF4jUAdWV03ZNNYdcGerIAKc2xsugSKjJIm1XPQq
upz914b4/hp/UslDUH7SPZ+3Tq17TXW+xP6aUM9VcWMnlJT/DFgxh5sZB3nfDeYr
yBV+PmEVfuQaUSIwKAh3Hv0ZaBHS8KzsawLzbJ+Pa9W9wjVME1izi9ctloGSxtLc
p1suS2IS33QB0jGgj4VbuMcjo2iCpzW3Xd7jQ7WpXGjrl0bHnJTnWWB0ihDD3cJe
tJlk9GScMQu3Z3th67DmqTXQru3btrDjxMifABEBAAEAB/sG3JffwJO8dNNRwpg5
OWM5kPuuzKQ5tlVi7FWqmmMwVDRLwoQqzkj0lS4vUoGAfWuSEZMWeFodbYCUYbbp
WzOCiivVnyxTcF1Eq4tOShdZxBNklbjEAzxeq0F+YRdaO7NCn6xtVizIqfvAre3v
Aujy12m9ySQSIZUR9ERtwlL5VXj4yzcXciyM9aNbh2D4BFAoo+fBL1B5mhpKQ16B
BafkpOf0myWZrwLdcn1t7BQ6UcLqsPJj6COCXXRDHoZvim38UBMMajNP4+8eC2Vk
DDR6iFz53FRvIREQWkoEEW4P80v1lgcnP1apJGhkrJYCPM7ObZk0MoQkBiBdlPPQ
VLiRBADhNsKjNd7v3Af5XGPJGWWZxCMxPlgwc6EMjCX3SQDql042eSIqwn1FFP6H
Hi8lyq/XWz0JcUwcrpFxpIC3OLUsBleX3Z4E17Z6AotSV0FC6y23cqgtweMeLA0L
tayMRmXJ7Aan+xfvi8MbW6lYGzM2uYHl27rVBuCt0kvxn+U4lwQA+ATO8KaBreCL
E/vXGZaPWJs2GoiDBavKwqdYtlXDKlVP5XvBgaPfY6zKLtCsC6N+DKkVo42Stwu3
oB6HSSLvXvFIhvUReh5PLpijmOroAJzvLQN9qmZBvF4IOUgaHmlYlL0DGzlxcwLr
SdzqPaGRa+EgraFDDGGu5oQ5A2wNKTkEAMDS2w5SibOfsvnfK4aDc0mi8/fftzCO
SwsvVzR0SmbXtaCPBnBjlA7YJhRZ6n/7nDpswrWPk8g125A0vK1cNRGM0ufKYW1f
DMbB/aYrJjAKz8zWH6cgOD4fqM2gA9JounkIcJvWOToGume8pXhmem94u2FP8aik
A6hL/Os+WRQcMNi0NWlkLWNvbmZpZ3VyYXRvci10ZXN0cyA8dGVzdHNAaWQtY29u
ZmlndXJhdG9yLmludmFsaWQ+iQFuBBMBCABYFiEEaZF4ALsB1xUU3IP5rl3Himib
1RgFAmqg6rMbFIAAAAAABAAObWFudTIsMi41KzEuMTIsMCwzAxsvBAULCQgHAgIi
AgYVCgkICwIEFgIDAQIeBwIXgAAKCRCuXceKaJvVGFRrB/wIbG1HHQANDCHgyl4T
RTPuKBX2c9lmJi2plp+T5L5sXyTMbAMLfTCkJWL/zHLM3N2v6GZuzbwZM8Dzj9zL
LMpC6newmZe4YX+ZtyZGufBI963kh8PvlSWWUSyH+oYoIhlyJqj7tCQXoijEvQnS
ReKGHUTvQGvQK2Ybp735NmAiDOM3/3KKh2Dqas/Tc2mxec4whr8RyUidsF+Yp6wT
esfMxeax6DmWIcSLDWpUBndeahZ2Ff0c96BzbpF1v2hbvyI/IaP5snb4Nm1xpREm
+y8iDAQhOklfXCyXrWgN3sIrk3tDndGrn+nw0iusnaBVki0keqjlH0camoUP28tu
3sxG
=TrE4
-----END PGP PRIVATE KEY BLOCK-----
@@ -0,0 +1,19 @@
-----BEGIN PGP PUBLIC KEY BLOCK-----
mQENBGqg6rMBCADaMUd27S41+kInPXuRZ0aL41aMZjqr+mw4W34w8i/JRYMJ6eGb
sXf40AaFfS7zkbUpQZpmh3aYPF4jUAdWV03ZNNYdcGerIAKc2xsugSKjJIm1XPQq
upz914b4/hp/UslDUH7SPZ+3Tq17TXW+xP6aUM9VcWMnlJT/DFgxh5sZB3nfDeYr
yBV+PmEVfuQaUSIwKAh3Hv0ZaBHS8KzsawLzbJ+Pa9W9wjVME1izi9ctloGSxtLc
p1suS2IS33QB0jGgj4VbuMcjo2iCpzW3Xd7jQ7WpXGjrl0bHnJTnWWB0ihDD3cJe
tJlk9GScMQu3Z3th67DmqTXQru3btrDjxMifABEBAAG0NWlkLWNvbmZpZ3VyYXRv
ci10ZXN0cyA8dGVzdHNAaWQtY29uZmlndXJhdG9yLmludmFsaWQ+iQFuBBMBCABY
FiEEaZF4ALsB1xUU3IP5rl3Himib1RgFAmqg6rMbFIAAAAAABAAObWFudTIsMi41
KzEuMTIsMCwzAxsvBAULCQgHAgIiAgYVCgkICwIEFgIDAQIeBwIXgAAKCRCuXceK
aJvVGFRrB/wIbG1HHQANDCHgyl4TRTPuKBX2c9lmJi2plp+T5L5sXyTMbAMLfTCk
JWL/zHLM3N2v6GZuzbwZM8Dzj9zLLMpC6newmZe4YX+ZtyZGufBI963kh8PvlSWW
USyH+oYoIhlyJqj7tCQXoijEvQnSReKGHUTvQGvQK2Ybp735NmAiDOM3/3KKh2Dq
as/Tc2mxec4whr8RyUidsF+Yp6wTesfMxeax6DmWIcSLDWpUBndeahZ2Ff0c96Bz
bpF1v2hbvyI/IaP5snb4Nm1xpREm+y8iDAQhOklfXCyXrWgN3sIrk3tDndGrn+nw
0iusnaBVki0keqjlH0camoUP28tu3sxG
=qZrp
-----END PGP PUBLIC KEY BLOCK-----
@@ -0,0 +1,94 @@
"""Build a minimal, offline, hand-rolled crate and its sparse registry index
entry, for integration-testing the `rust` module of id-configurator.
No network access and no build backend (cargo) is used: the crate is a
plain gzipped tar assembled with the standard library only, so this script
can run in a minimal Python image with nothing installed.
See https://doc.rust-lang.org/cargo/reference/registries.html#index-format
for the sparse registry index/config.json format this reproduces.
"""
from __future__ import annotations
import hashlib
import io
import json
import pathlib
import tarfile
CRATE = "idconfiguratortestpkg"
VERSION = "0.0.1"
CRATE_FILE = f"{CRATE}-{VERSION}.crate"
OUT_DIR = pathlib.Path("/srv")
CARGO_TOML = f"""[package]
name = "{CRATE}"
version = "{VERSION}"
edition = "2021"
"""
LIB_RS = "// test-only crate served by the id-configurator integration tests\n"
def _add_file(archive: tarfile.TarFile, name: str, content: bytes) -> None:
info = tarfile.TarInfo(name=f"{CRATE}-{VERSION}/{name}")
info.size = len(content)
archive.addfile(info, io.BytesIO(content))
def build_crate() -> bytes:
buffer = io.BytesIO()
with tarfile.open(fileobj=buffer, mode="w:gz") as archive:
_add_file(archive, "Cargo.toml", CARGO_TOML.encode("utf-8"))
_add_file(archive, "src/lib.rs", LIB_RS.encode("utf-8"))
return buffer.getvalue()
def index_path(name: str) -> pathlib.Path:
"""Mirror cargo's sparse index path convention for a crate name."""
if len(name) <= 2:
return pathlib.Path(str(len(name))) / name
if len(name) == 3:
return pathlib.Path("3") / name[0] / name
return pathlib.Path(name[0:2]) / name[2:4] / name
def write_index(out_dir: pathlib.Path, crate: bytes) -> None:
entry = {
"name": CRATE,
"vers": VERSION,
"deps": [],
"cksum": hashlib.sha256(crate).hexdigest(),
"features": {},
"yanked": False,
}
path = out_dir / index_path(CRATE)
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps(entry) + "\n", encoding="utf-8")
def write_config_json(out_dir: pathlib.Path) -> None:
# Cargo requires a fully-qualified "dl" download template, so this bakes
# in a placeholder that nginx's sub_filter rewrites to the real
# https://<host used by the client>/ at request time (see
# cargo-repo.nginx.conf) - the host and port this fixture is actually
# reached on aren't known at image-build time.
config = {
"dl": "__BASE_URL__/api/v1/crates/{crate}/{version}/download",
"api": "__BASE_URL__",
}
(out_dir / "config.json").write_text(json.dumps(config), encoding="utf-8")
def main() -> None:
OUT_DIR.mkdir(parents=True, exist_ok=True)
crate = build_crate()
download_path = OUT_DIR / "api/v1/crates" / CRATE / VERSION / "download"
download_path.parent.mkdir(parents=True, exist_ok=True)
download_path.write_bytes(crate)
write_index(OUT_DIR, crate)
write_config_json(OUT_DIR)
if __name__ == "__main__":
main()
@@ -0,0 +1,76 @@
"""Build a minimal, offline, hand-rolled Composer package and its
`packages.json` index, for integration-testing the `php` module of
id-configurator.
No network access and no build backend (composer) is used: the package is a
plain zip file assembled with the standard library only, so this script can
run in a minimal Python image with nothing installed.
"""
from __future__ import annotations
import hashlib
import io
import json
import pathlib
import zipfile
VENDOR = "interdiode"
PACKAGE = "idconfiguratortestpkg"
NAME = f"{VENDOR}/{PACKAGE}"
VERSION = "0.0.1"
ZIP_NAME = f"{PACKAGE}-{VERSION}.zip"
OUT_DIR = pathlib.Path("/srv")
COMPOSER_JSON = json.dumps({"name": NAME, "version": VERSION, "type": "library"})
def build_zip() -> bytes:
buffer = io.BytesIO()
with zipfile.ZipFile(buffer, "w", zipfile.ZIP_DEFLATED) as archive:
archive.writestr("composer.json", COMPOSER_JSON)
archive.writestr(
"marker.txt",
"test-only package served by the id-configurator integration tests\n",
)
return buffer.getvalue()
def write_packages_json(out_dir: pathlib.Path, zip_content: bytes) -> None:
# Composer requires a fully-qualified dist URL: unlike PyPI's simple
# index, it does not resolve relative/root-relative ones against the
# repository's own URL for this (non-v2) repository format. The host
# and port this fixture will actually be reached on aren't known at
# image-build time (tox-docker only assigns them once the container
# starts), so this bakes in a placeholder that nginx's sub_filter
# rewrites to the real https://<host used by the client>/ at request
# time (see php-repo.nginx.conf).
packages = {
"packages": {
NAME: {
VERSION: {
"name": NAME,
"version": VERSION,
"dist": {
"type": "zip",
"url": f"__BASE_URL__/{ZIP_NAME}",
"shasum": hashlib.sha1(zip_content).hexdigest(), # nosec B324
},
}
}
}
}
(out_dir / "packages.json").write_text(
json.dumps(packages, indent=2), encoding="utf-8"
)
def main() -> None:
OUT_DIR.mkdir(parents=True, exist_ok=True)
zip_content = build_zip()
(OUT_DIR / ZIP_NAME).write_bytes(zip_content)
write_packages_json(OUT_DIR, zip_content)
if __name__ == "__main__":
main()
+32
View File
@@ -0,0 +1,32 @@
#!/bin/sh
# Build a minimal, hand-rolled RubyGems package and a full gem repository
# index for it (legacy + compact index formats), for integration-testing
# the `ruby` module of id-configurator. Uses RubyGems' own `gem build` and
# `gem generate_index` (the same tools a real gem server host would use)
# rather than hand-rolling the compact index's binary/marshal formats.
set -eu
PACKAGE=idconfiguratortestpkg
VERSION=0.0.1
MARKER="test-only gem served by the id-configurator integration tests"
gem install rubygems-generate_index --no-document
mkdir -p "/build/${PACKAGE}-${VERSION}"
cd "/build/${PACKAGE}-${VERSION}"
cat > "${PACKAGE}.gemspec" <<EOF
Gem::Specification.new do |s|
s.name = "${PACKAGE}"
s.version = "${VERSION}"
s.summary = "test-only gem"
s.authors = ["InterDiode"]
s.license = "MIT"
s.files = ["marker.txt"]
end
EOF
echo "${MARKER}" > marker.txt
gem build "${PACKAGE}.gemspec"
mkdir -p /srv/gems
cp ./*.gem /srv/gems/
gem generate_index -d /srv
@@ -0,0 +1,47 @@
"""Build a minimal, offline, hand-rolled Go module proxy tree, for
integration-testing the `go` module of id-configurator.
No network access and no Go toolchain is used: the module proxy protocol
(https://go.dev/ref/mod#module-proxy) only requires a handful of static
files, none of which embed the server's own base URL (unlike Composer/
npm/cargo), so this can be served directly by nginx with no rewriting.
"""
from __future__ import annotations
import io
import json
import pathlib
import zipfile
MODULE = "example.invalid/idconfiguratortestpkg"
VERSION = "v0.0.1"
OUT_DIR = pathlib.Path("/srv")
MARKER = "test-only module served by the id-configurator integration tests\n"
GO_MOD = f"module {MODULE}\n\ngo 1.21\n"
def build_zip() -> bytes:
buffer = io.BytesIO()
prefix = f"{MODULE}@{VERSION}"
with zipfile.ZipFile(buffer, "w") as archive:
archive.writestr(f"{prefix}/go.mod", GO_MOD)
archive.writestr(f"{prefix}/marker.txt", MARKER)
return buffer.getvalue()
def main() -> None:
module_dir = OUT_DIR / MODULE / "@v"
module_dir.mkdir(parents=True, exist_ok=True)
(module_dir / "list").write_text(f"{VERSION}\n", encoding="utf-8")
(module_dir / f"{VERSION}.info").write_text(
json.dumps({"Version": VERSION, "Time": "2024-01-01T00:00:00Z"}),
encoding="utf-8",
)
(module_dir / f"{VERSION}.mod").write_text(GO_MOD, encoding="utf-8")
(module_dir / f"{VERSION}.zip").write_bytes(build_zip())
if __name__ == "__main__":
main()
+66
View File
@@ -0,0 +1,66 @@
"""Build a minimal, offline, hand-rolled jar and its pom.xml, laid out as a
Maven-compatible repository, for integration-testing the `java` module of
id-configurator.
No network access and no build backend (maven/gradle) are used: the jar is
a plain zip file assembled with the standard library only, so this script
can run in a minimal Python image with nothing installed.
"""
from __future__ import annotations
import hashlib
import io
import pathlib
import zipfile
GROUP = "org.interdiode"
GROUP_PATH = GROUP.replace(".", "/")
ARTIFACT = "idconfiguratortestpkg"
VERSION = "0.0.1"
OUT_DIR = pathlib.Path("/srv") / GROUP_PATH / ARTIFACT / VERSION
MANIFEST = "Manifest-Version: 1.0\n"
POM = f"""<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0">
<modelVersion>4.0.0</modelVersion>
<groupId>{GROUP}</groupId>
<artifactId>{ARTIFACT}</artifactId>
<version>{VERSION}</version>
<packaging>jar</packaging>
<description>Test-only artifact served by the id-configurator integration tests</description>
</project>
"""
def build_jar() -> bytes:
buffer = io.BytesIO()
with zipfile.ZipFile(buffer, "w", zipfile.ZIP_DEFLATED) as archive:
archive.writestr("META-INF/MANIFEST.MF", MANIFEST)
archive.writestr(
"marker.txt",
"test-only artifact served by the id-configurator integration tests\n",
)
return buffer.getvalue()
def _write_with_checksums(path: pathlib.Path, content: bytes) -> None:
"""Write `content` to `path`, plus the .sha1/.md5 sidecar files most
build tools (Leiningen's aether resolver in particular) require to be
present, unlike Maven which merely warns when they are missing."""
path.write_bytes(content)
sha1_digest = hashlib.sha1(content).hexdigest() # nosec B324
(path.parent / f"{path.name}.sha1").write_text(sha1_digest, encoding="ascii")
md5_digest = hashlib.md5(content).hexdigest() # nosec B324
(path.parent / f"{path.name}.md5").write_text(md5_digest, encoding="ascii")
def main() -> None:
OUT_DIR.mkdir(parents=True, exist_ok=True)
_write_with_checksums(OUT_DIR / f"{ARTIFACT}-{VERSION}.jar", build_jar())
_write_with_checksums(OUT_DIR / f"{ARTIFACT}-{VERSION}.pom", POM.encode("utf-8"))
if __name__ == "__main__":
main()
@@ -0,0 +1,76 @@
"""Build a minimal, offline, hand-rolled npm package and its registry
metadata document, for integration-testing the `npm` module of
id-configurator.
No network access and no build backend (npm/node) is used: the tarball is a
plain gzipped tar assembled with the standard library only, so this script
can run in a minimal Python image with nothing installed.
"""
from __future__ import annotations
import base64
import hashlib
import io
import json
import pathlib
import tarfile
PACKAGE = "idconfiguratortestpkg"
VERSION = "0.0.1"
TARBALL_NAME = f"{PACKAGE}-{VERSION}.tgz"
OUT_DIR = pathlib.Path("/srv")
PACKAGE_JSON = json.dumps({"name": PACKAGE, "version": VERSION})
MARKER = "test-only package served by the id-configurator integration tests\n"
def _add_file(archive: tarfile.TarFile, name: str, content: bytes) -> None:
info = tarfile.TarInfo(name=f"package/{name}")
info.size = len(content)
archive.addfile(info, io.BytesIO(content))
def build_tarball() -> bytes:
buffer = io.BytesIO()
with tarfile.open(fileobj=buffer, mode="w:gz") as archive:
_add_file(archive, "package.json", PACKAGE_JSON.encode("utf-8"))
_add_file(archive, "marker.txt", MARKER.encode("utf-8"))
return buffer.getvalue()
def write_registry_metadata(out_dir: pathlib.Path, tarball: bytes) -> None:
# npm requires a fully-qualified tarball URL, so this bakes in a
# placeholder that nginx's sub_filter rewrites to the real
# https://<host used by the client>/ at request time (see
# npm-repo.nginx.conf) - the host and port this fixture is actually
# reached on aren't known at image-build time.
sha1 = hashlib.sha1(tarball).hexdigest() # nosec B324
sha512_b64 = base64.b64encode(hashlib.sha512(tarball).digest()).decode("ascii")
metadata = {
"name": PACKAGE,
"dist-tags": {"latest": VERSION},
"versions": {
VERSION: {
"name": PACKAGE,
"version": VERSION,
"dist": {
"shasum": sha1,
"integrity": f"sha512-{sha512_b64}",
"tarball": f"__BASE_URL__/{TARBALL_NAME}",
},
}
},
}
(out_dir / PACKAGE).write_text(json.dumps(metadata, indent=2), encoding="utf-8")
def main() -> None:
OUT_DIR.mkdir(parents=True, exist_ok=True)
tarball = build_tarball()
(OUT_DIR / TARBALL_NAME).write_bytes(tarball)
write_registry_metadata(OUT_DIR, tarball)
if __name__ == "__main__":
main()
@@ -0,0 +1,61 @@
"""Build a minimal NuGet v3 static feed around one real, `dotnet pack`-built
.nupkg, for integration-testing the `nuget` module of id-configurator.
Only two resources are populated: the flat container (`PackageBaseAddress`,
used to enumerate versions and download the .nupkg directly) and the
top-level service index pointing at it. `dotnet add package`/`dotnet
restore` with an explicit, dependency-free package version only ever calls
these two - never the (considerably more complex) registration resource -
so it is intentionally not implemented here.
The service index bakes in a __BASE_URL__ placeholder for its "@id" fields,
since the host:port this fixture is actually reached on is only known once
tox-docker starts the container, not at image-build time (see
nuget-repo.nginx.conf). The flat container's own files never embed a URL,
so they need no such rewriting.
"""
from __future__ import annotations
import json
import pathlib
import shutil
PACKAGE_ID = "idconfiguratortestpkg"
VERSION = "0.0.1"
NUPKG_NAME = f"{PACKAGE_ID}.{VERSION}.nupkg"
NUPKG_SRC = pathlib.Path("/nupkg") / NUPKG_NAME
OUT_DIR = pathlib.Path("/srv")
def write_flat_container() -> None:
package_dir = OUT_DIR / "flatcontainer" / PACKAGE_ID
version_dir = package_dir / VERSION
version_dir.mkdir(parents=True, exist_ok=True)
(package_dir / "index.json").write_text(
json.dumps({"versions": [VERSION]}), encoding="utf-8"
)
shutil.copy(NUPKG_SRC, version_dir / NUPKG_NAME)
def write_service_index() -> None:
index = {
"version": "3.0.0",
"resources": [
{
"@id": "__BASE_URL__/flatcontainer/",
"@type": "PackageBaseAddress/3.0.0",
}
],
}
(OUT_DIR / "index.json").write_text(json.dumps(index), encoding="utf-8")
def main() -> None:
OUT_DIR.mkdir(parents=True, exist_ok=True)
write_flat_container()
write_service_index()
if __name__ == "__main__":
main()
@@ -0,0 +1,80 @@
"""Build a minimal, offline, hand-rolled wheel and PEP 503 "simple" index for
integration-testing the `python` module of id-configurator.
No network access and no build backend (setuptools/build/pip) are used: the
wheel is a plain zip file assembled with the standard library only, so this
script can run in a minimal Python image with nothing installed.
"""
from __future__ import annotations
import base64
import hashlib
import pathlib
import zipfile
PACKAGE = "idconfiguratortestpkg"
VERSION = "0.0.1"
WHEEL_NAME = f"{PACKAGE}-{VERSION}-py3-none-any.whl"
OUT_DIR = pathlib.Path("/srv")
METADATA = f"""Metadata-Version: 2.1
Name: {PACKAGE}
Version: {VERSION}
Summary: Test-only package served by the id-configurator integration tests
"""
WHEEL_FILE = """Wheel-Version: 1.0
Generator: id-configurator-tests
Root-Is-Purelib: true
Tag: py3-none-any
"""
INIT_PY = '"""Test-only package used to verify the python module mirror."""\n'
def _record_line(path: str, content: bytes) -> str:
digest = base64.urlsafe_b64encode(hashlib.sha256(content).digest()).rstrip(b"=")
return f"{path},sha256={digest.decode('ascii')},{len(content)}"
def build_wheel(packages_dir: pathlib.Path) -> bytes:
dist_info = f"{PACKAGE}-{VERSION}.dist-info"
files = {
f"{PACKAGE}/__init__.py": INIT_PY.encode("utf-8"),
f"{dist_info}/METADATA": METADATA.encode("utf-8"),
f"{dist_info}/WHEEL": WHEEL_FILE.encode("utf-8"),
}
record_lines = [_record_line(name, content) for name, content in files.items()]
record_lines.append(f"{dist_info}/RECORD,,")
files[f"{dist_info}/RECORD"] = ("\n".join(record_lines) + "\n").encode("utf-8")
packages_dir.mkdir(parents=True, exist_ok=True)
wheel_path = packages_dir / WHEEL_NAME
with zipfile.ZipFile(wheel_path, "w", zipfile.ZIP_DEFLATED) as archive:
for name, content in files.items():
archive.writestr(name, content)
return wheel_path.read_bytes()
def write_simple_index(simple_dir: pathlib.Path, wheel_content: bytes) -> None:
package_dir = simple_dir / PACKAGE
package_dir.mkdir(parents=True, exist_ok=True)
digest = hashlib.sha256(wheel_content).hexdigest()
html = f"""<!DOCTYPE html>
<html>
<body>
<a href="../../packages/{WHEEL_NAME}#sha256={digest}">{WHEEL_NAME}</a>
</body>
</html>
"""
(package_dir / "index.html").write_text(html, encoding="utf-8")
def main() -> None:
wheel_content = build_wheel(OUT_DIR / "packages")
write_simple_index(OUT_DIR / "simple", wheel_content)
if __name__ == "__main__":
main()
@@ -0,0 +1,17 @@
# Minimal cargo sparse registry (index entry, config.json and one hand-built
# crate) served over TLS, using the same Test CA as the other *-repo
# fixtures, used by tox-docker to exercise the `rust` module integration
# test (tests/configs/Rust.yaml.j2, tests/dockerfiles/Rust.Dockerfile).
#
# Built by tox-docker, whose build context is this directory (not the repo
# root) - every COPY source below must live under tests/dockerfiles/repos/.
FROM python:3.13-slim AS packagebuild
COPY build_test_cargo_package.py /build_test_cargo_package.py
RUN python3 /build_test_cargo_package.py
FROM nginx:alpine
COPY --from=packagebuild /srv /srv
COPY repo-server.crt.pem /etc/nginx/certs/repo-server.crt.pem
COPY repo-server.key.pem /etc/nginx/certs/repo-server.key.pem
COPY cargo-repo.nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 8443
@@ -0,0 +1,22 @@
server {
listen 8443 ssl default_server;
server_name _;
ssl_certificate /etc/nginx/certs/repo-server.crt.pem;
ssl_certificate_key /etc/nginx/certs/repo-server.key.pem;
root /srv;
autoindex off;
# config.json bakes in a __BASE_URL__ placeholder for its "dl"/"api"
# templates, since the host:port this fixture is actually reached on is
# only known once tox-docker starts the container, not at image-build
# time. $http_host reflects exactly what the client (cargo) used to
# connect, e.g. host.docker.internal:55123.
location = /config.json {
default_type application/json;
sub_filter '__BASE_URL__' 'https://$http_host';
sub_filter_once off;
sub_filter_types application/json;
}
}
@@ -0,0 +1,32 @@
# Minimal HTTPS server, using the same Test CA as the other *-repo fixtures,
# used by tox-docker to exercise the `certificates` module integration test
# (tests/configs/Certificate.yaml.j2, tests/dockerfiles/Certificate.Dockerfile).
# Unlike the package-manager fixtures, this one has nothing to do with a
# specific package format: it serves a static marker file (for wget, curl and
# openssl s_client) and a bare git repository over the "dumb" HTTP protocol
# (for `git clone`).
#
# Built by tox-docker, whose build context is this directory (not the repo
# root) - every COPY source below must live under tests/dockerfiles/repos/.
FROM alpine:3.20 AS gitbuild
RUN apk add --no-cache git
RUN git config --global init.defaultBranch main \
&& git config --global user.email test@example.invalid \
&& git config --global user.name "id-configurator tests"
RUN mkdir -p /work \
&& cd /work \
&& git init -q \
&& echo "trusted-via-root-ca" > marker.txt \
&& git add marker.txt \
&& git commit -q -m "initial commit"
RUN git clone --bare -q /work /srv/repo.git \
&& cd /srv/repo.git \
&& git update-server-info
FROM nginx:alpine
RUN mkdir -p /srv && echo "trusted-via-root-ca" > /srv/marker.txt
COPY --from=gitbuild /srv/repo.git /srv/repo.git
COPY repo-server.crt.pem /etc/nginx/certs/repo-server.crt.pem
COPY repo-server.key.pem /etc/nginx/certs/repo-server.key.pem
COPY certificate-repo.nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 8443
@@ -0,0 +1,10 @@
server {
listen 8443 ssl default_server;
server_name _;
ssl_certificate /etc/nginx/certs/repo-server.crt.pem;
ssl_certificate_key /etc/nginx/certs/repo-server.key.pem;
root /srv;
autoindex off;
}
@@ -0,0 +1,40 @@
# Static id-configurator configuration served by the config-url-repo fixture,
# to be downloaded (not copied in locally) via `id-configurator apply
# --config-url ... --config-url-cacert-content ...`
# (tests/dockerfiles/ConfigUrl.Dockerfile). No per-run templating is needed:
# unlike the module-* fixtures, nothing in this document needs to reference
# its own server's dynamic host:port.
#
# The certificate below is tests/certs/test-ca.crt.pem - the same Test CA
# that signs repo-server.crt.pem in this directory. It's duplicated here
# (rather than read at build time) because a Docker build's context is
# this directory alone; keep both copies in sync if the Test CA is ever
# regenerated.
uuid: 8f3e4b2c-9a41-4d3e-8f1a-2b6c7d8e9f01
organisation: InterDiode
description: Configuration downloaded via --config-url
configurations:
certificates:
- name: Test CA
content: |
-----BEGIN CERTIFICATE-----
MIIDdTCCAl2gAwIBAgIUKq4URHQwyg7IzJ3ViAGGlM9pGCwwDQYJKoZIhvcNAQEL
BQAwQjEeMBwGA1UECgwVaWQtY29uZmlndXJhdG9yLXRlc3RzMSAwHgYDVQQDDBdp
ZC1jb25maWd1cmF0b3IgVGVzdCBDQTAeFw0yNjA5MDUyMTA4MTNaFw00NjA4MzEy
MTA4MTNaMEIxHjAcBgNVBAoMFWlkLWNvbmZpZ3VyYXRvci10ZXN0czEgMB4GA1UE
AwwXaWQtY29uZmlndXJhdG9yIFRlc3QgQ0EwggEiMA0GCSqGSIb3DQEBAQUAA4IB
DwAwggEKAoIBAQD8bMteh3MDnbW6OoFMlhA4Z2QAabVoSFlSE/Si7ut+L/3v1GY4
lixenwLQc8a2koIH9CtmOodzfO7hpro/VUjYJ9J2eTsXLLiOg7OgggG1cKV9/kt7
RF0Q4hkcanGmiffc4xbz4ZedU6lP1vzLCsgU0A8OPt6svcNjGcWbjxss0KjgxGZ7
H4UKdL7Mv4W2Y6Fw1b4eV2M6jErkQK6IPXsHGFh7JkzliNcb0sX1VfbfauS84tBh
f3EHwzLNhbjtEmQhQdqgsg4Cbk/gWur//6aEvlD/X12PAUbrp4DFD+CDuVagLX7I
7TQBXNFGKhozzQuDg7DFg1gIV8SFx2m+s60/AgMBAAGjYzBhMB0GA1UdDgQWBBSb
zJ7DsaxwCeqbfOXGJ3x+u7bjozAfBgNVHSMEGDAWgBSbzJ7DsaxwCeqbfOXGJ3x+
u7bjozAPBgNVHRMBAf8EBTADAQH/MA4GA1UdDwEB/wQEAwIBBjANBgkqhkiG9w0B
AQsFAAOCAQEAEV1+dG3agh0S/AR8jC8M33HXFyvneCCRtfb5q/loJ//VoWPbD3Gs
G+2SR0MVN+KKOMBvCBh3KrNSgFF9iFmBjNCJJZ8ycDIszUK3xLjUP2EzN1pNNl8S
D/yHzUCro3547Q+I+Nn5zRciG95EpMvtkIWem4H8luYSBt7w8y9uDUKWzaca/v3z
oGZPZP99w8WVqMKGRbsK94wsTVsWiE+oHzSufwrhvZJfQQ4jU6102IEnRS/eQFXA
+fozgEWuFatZe3Gz0HUJakrzSBwSTV6wmnEHCfu+uwdns1Ttq5FylZ4StJHsmweL
14w1xa3tjC5bQ3e1QWK3R5InHcSZ7tLpGA==
-----END CERTIFICATE-----
@@ -0,0 +1,13 @@
# Serves a static id-configurator configuration document over TLS, using the
# same Test CA as the other *-repo fixtures, used by tox-docker to exercise
# `id-configurator apply --config-url ... --config-url-cacert-content ...`
# (tests/dockerfiles/ConfigUrl.Dockerfile).
#
# Built by tox-docker, whose build context is this directory (not the repo
# root) - every COPY source below must live under tests/dockerfiles/repos/.
FROM nginx:alpine
COPY config-url-config.yaml /srv/config.yaml
COPY repo-server.crt.pem /etc/nginx/certs/repo-server.crt.pem
COPY repo-server.key.pem /etc/nginx/certs/repo-server.key.pem
COPY config-url-repo.nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 8443
@@ -0,0 +1,10 @@
server {
listen 8443 ssl default_server;
server_name _;
ssl_certificate /etc/nginx/certs/repo-server.crt.pem;
ssl_certificate_key /etc/nginx/certs/repo-server.key.pem;
root /srv;
autoindex off;
}
@@ -0,0 +1,17 @@
# Minimal Go module proxy serving one hand-built module over TLS, using the
# same Test CA as the other *-repo fixtures, used by tox-docker to exercise
# the `go` module integration test (tests/configs/Go.yaml.j2,
# tests/dockerfiles/Go.Dockerfile).
#
# Built by tox-docker, whose build context is this directory (not the repo
# root) - every COPY source below must live under tests/dockerfiles/repos/.
FROM python:3.13-slim AS packagebuild
COPY build_test_go_module.py /build_test_go_module.py
RUN python3 /build_test_go_module.py
FROM nginx:alpine
COPY --from=packagebuild /srv /srv
COPY repo-server.crt.pem /etc/nginx/certs/repo-server.crt.pem
COPY repo-server.key.pem /etc/nginx/certs/repo-server.key.pem
COPY go-repo.nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 8443
@@ -0,0 +1,10 @@
server {
listen 8443 ssl default_server;
server_name _;
ssl_certificate /etc/nginx/certs/repo-server.crt.pem;
ssl_certificate_key /etc/nginx/certs/repo-server.key.pem;
root /srv;
autoindex off;
}
@@ -0,0 +1,16 @@
Name: idconfiguratortestpkg
Version: 0.0.1
Release: 1
Summary: Test-only package served by the id-configurator integration tests
License: MIT
BuildArch: noarch
%description
Test-only package served by the id-configurator integration tests.
%install
mkdir -p %{buildroot}/usr/share/doc/%{name}
echo 'test-only package served by the id-configurator integration tests' > %{buildroot}/usr/share/doc/%{name}/marker
%files
/usr/share/doc/%{name}/marker
@@ -0,0 +1,17 @@
# Minimal Maven-compatible repository serving one hand-built jar over TLS,
# using the same Test CA as the pypi-repo/apt-repo/yum-repo/apk-repo
# fixtures, used by tox-docker to exercise the `java` module integration
# test (tests/configs/Java.yaml.j2, tests/dockerfiles/Java.Dockerfile).
#
# Built by tox-docker, whose build context is this directory (not the repo
# root) - every COPY source below must live under tests/dockerfiles/repos/.
FROM python:3.13-slim AS jarbuild
COPY build_test_jar.py /build_test_jar.py
RUN python3 /build_test_jar.py
FROM nginx:alpine
COPY --from=jarbuild /srv /srv
COPY repo-server.crt.pem /etc/nginx/certs/repo-server.crt.pem
COPY repo-server.key.pem /etc/nginx/certs/repo-server.key.pem
COPY java-repo.nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 8443
@@ -0,0 +1,10 @@
server {
listen 8443 ssl default_server;
server_name _;
ssl_certificate /etc/nginx/certs/repo-server.crt.pem;
ssl_certificate_key /etc/nginx/certs/repo-server.key.pem;
root /srv;
autoindex off;
}
@@ -0,0 +1,17 @@
# Minimal npm registry (one package's metadata document plus its tarball)
# served over TLS, using the same Test CA as the other *-repo fixtures, used
# by tox-docker to exercise the `npm` module integration test
# (tests/configs/Npm.yaml.j2, tests/dockerfiles/Npm.Dockerfile).
#
# Built by tox-docker, whose build context is this directory (not the repo
# root) - every COPY source below must live under tests/dockerfiles/repos/.
FROM python:3.13-slim AS packagebuild
COPY build_test_npm_package.py /build_test_npm_package.py
RUN python3 /build_test_npm_package.py
FROM nginx:alpine
COPY --from=packagebuild /srv /srv
COPY repo-server.crt.pem /etc/nginx/certs/repo-server.crt.pem
COPY repo-server.key.pem /etc/nginx/certs/repo-server.key.pem
COPY npm-repo.nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 8443
@@ -0,0 +1,22 @@
server {
listen 8443 ssl default_server;
server_name _;
ssl_certificate /etc/nginx/certs/repo-server.crt.pem;
ssl_certificate_key /etc/nginx/certs/repo-server.key.pem;
root /srv;
autoindex off;
# The package metadata document bakes in a __BASE_URL__ placeholder for
# its tarball URL, since the host:port this fixture is actually reached
# on is only known once tox-docker starts the container, not at
# image-build time. $http_host reflects exactly what the client (npm)
# used to connect, e.g. host.docker.internal:55123.
location = /idconfiguratortestpkg {
default_type application/json;
sub_filter '__BASE_URL__' 'https://$http_host';
sub_filter_once off;
sub_filter_types application/json;
}
}
@@ -0,0 +1,23 @@
# Minimal NuGet v3 static feed serving one real, dotnet-packed .nupkg over
# TLS, using the same Test CA as the other *-repo fixtures, used by
# tox-docker to exercise the `nuget` module integration test
# (tests/configs/NuGet.yaml.j2, tests/dockerfiles/NuGet.Dockerfile).
#
# Built by tox-docker, whose build context is this directory (not the repo
# root) - every COPY source below must live under tests/dockerfiles/repos/.
FROM mcr.microsoft.com/dotnet/sdk:9.0 AS packagebuild
COPY IdConfiguratorTestPkg.csproj /pkgsrc/IdConfiguratorTestPkg.csproj
RUN echo 'test-only package served by the id-configurator integration tests' > /pkgsrc/marker.txt \
&& dotnet pack /pkgsrc -c Release -o /nupkg
FROM python:3.13-slim AS feedbuild
COPY --from=packagebuild /nupkg /nupkg
COPY build_test_nuget_feed.py /build_test_nuget_feed.py
RUN python3 /build_test_nuget_feed.py
FROM nginx:alpine
COPY --from=feedbuild /srv /srv
COPY repo-server.crt.pem /etc/nginx/certs/repo-server.crt.pem
COPY repo-server.key.pem /etc/nginx/certs/repo-server.key.pem
COPY nuget-repo.nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 8443
@@ -0,0 +1,22 @@
server {
listen 8443 ssl default_server;
server_name _;
ssl_certificate /etc/nginx/certs/repo-server.crt.pem;
ssl_certificate_key /etc/nginx/certs/repo-server.key.pem;
root /srv;
autoindex off;
# The service index bakes in a __BASE_URL__ placeholder for its "@id"
# fields, since the host:port this fixture is actually reached on is
# only known once tox-docker starts the container, not at image-build
# time. $http_host reflects exactly what the client (nuget) used to
# connect, e.g. host.docker.internal:55123.
location = /index.json {
default_type application/json;
sub_filter '__BASE_URL__' 'https://$http_host';
sub_filter_once off;
sub_filter_types application/json;
}
}
@@ -0,0 +1,17 @@
# Minimal Composer repository ("composer" type: a static packages.json plus
# one hand-built zip package) served over TLS, using the same Test CA as the
# other *-repo fixtures, used by tox-docker to exercise the `php` module
# integration test (tests/configs/Php.yaml.j2, tests/dockerfiles/Php.Dockerfile).
#
# Built by tox-docker, whose build context is this directory (not the repo
# root) - every COPY source below must live under tests/dockerfiles/repos/.
FROM python:3.13-slim AS packagebuild
COPY build_test_composer_package.py /build_test_composer_package.py
RUN python3 /build_test_composer_package.py
FROM nginx:alpine
COPY --from=packagebuild /srv /srv
COPY repo-server.crt.pem /etc/nginx/certs/repo-server.crt.pem
COPY repo-server.key.pem /etc/nginx/certs/repo-server.key.pem
COPY php-repo.nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 8443
@@ -0,0 +1,22 @@
server {
listen 8443 ssl default_server;
server_name _;
ssl_certificate /etc/nginx/certs/repo-server.crt.pem;
ssl_certificate_key /etc/nginx/certs/repo-server.key.pem;
root /srv;
autoindex off;
# packages.json bakes in a __BASE_URL__ placeholder for its dist URL,
# since the host:port this fixture is actually reached on is only known
# once tox-docker starts the container, not at image-build time.
# $http_host reflects exactly what the client (Composer) used to
# connect, e.g. host.docker.internal:55123.
location = /packages.json {
default_type application/json;
sub_filter '__BASE_URL__' 'https://$http_host';
sub_filter_once off;
sub_filter_types application/json;
}
}
@@ -0,0 +1,16 @@
# Minimal PEP 503 "simple" package index, serving one hand-built test wheel
# over TLS, used by tox-docker to exercise the `python` module integration
# test (tests/configs/Python.yaml.tmpl, tests/dockerfiles/Python.Dockerfile).
#
# Built by tox-docker, whose build context is this directory (not the repo
# root) - every COPY source below must live under tests/dockerfiles/repos/.
FROM python:3.13-slim AS wheelbuild
COPY build_test_wheel.py /build_test_wheel.py
RUN python3 /build_test_wheel.py
FROM nginx:alpine
COPY --from=wheelbuild /srv /srv
COPY repo-server.crt.pem /etc/nginx/certs/repo-server.crt.pem
COPY repo-server.key.pem /etc/nginx/certs/repo-server.key.pem
COPY pypi-repo.nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 8443

Some files were not shown because too many files have changed in this diff Show More