Compare commits

..

27 Commits

Author SHA1 Message Date
d9pouces ab2a179a55 feat: update config-url configuration with Test CA certificate and Dockerfile enhancements
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-07 08:12:48 +02:00
d9pouces 9de98d4744 feat: add config-url support with Dockerfile and NGINX configuration
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-07 07:15:58 +02:00
d9pouces 88380b6ab6 refactor: remove unused NPM_TO_COPY variable from build.py
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-06 23:08:58 +02:00
d9pouces e3b2e8a95a refactor: remove unused NPM_TO_COPY variable from build.py
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-06 22:22:51 +02:00
d9pouces 4e4b0592c0 refactor: remove unused NPM_TO_COPY variable from build.py
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-06 21:36:01 +02:00
d9pouces a8143b61b1 feat: add support for Go, Ruby, and NuGet modules with integration testing configurations
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-06 21:14:18 +02:00
d9pouces bcffe11bff feat: update tox configuration for multi-platform packaging and enhance testing environments
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-06 15:20:11 +02:00
d9pouces eaf04602aa feat: update tox configuration for multi-platform packaging and enhance testing environments
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-06 12:26:34 +02:00
d9pouces 4b0baf1a3c feat: update tox configuration for multi-platform packaging and enhance testing environments
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-06 12:23:25 +02:00
d9pouces b051730bd8 feat: update tox configuration for multi-platform packaging and enhance testing environments
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-06 10:48:58 +02:00
d9pouces cb3313bf65 feat: add minimal testing workflow and enhance coverage requirements in tox configuration
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-06 10:45:28 +02:00
d9pouces 6b6bc541dd feat: add Python-alpine Dockerfile for enhanced build process and testing with id-configurator
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-06 10:13:38 +02:00
d9pouces fdb20fce81 feat: add Python-alpine Dockerfile for enhanced build process and testing with id-configurator
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-06 08:29:51 +02:00
d9pouces 42e15a2d7e feat: enhance build process and Dockerfiles for improved artifact handling and dependency resolution
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-05 23:16:49 +02:00
d9pouces 438fed1e96 feat: enhance Certificate and certificate-repo Dockerfiles to support git operations and OpenSSL verification
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-05 22:51:40 +02:00
d9pouces 5caa5e601c feat: refactor Dockerfiles to use 'target' stage and add Certificate module configuration
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-05 22:31:27 +02:00
d9pouces f38a3d1ace feat: refactor Dockerfiles to use 'target' stage and add Certificate module configuration
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-05 22:31:12 +02:00
d9pouces a5a36fc7cd feat: update Dockerfiles and configuration for TLS support across multiple modules
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-05 22:23:34 +02:00
d9pouces fc41dfaa07 feat: add new configuration files for Apt, Apk, Java, and Yum
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-04 22:53:35 +02:00
d9pouces 6e086b0964 feat: add new configuration files for Apt, Apk, Java, and Yum
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-04 22:53:13 +02:00
d9pouces 22367224d7 feat: enhance Docker image tagging and add publication functionality
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-04 21:58:28 +02:00
d9pouces 26fcdd08e4 chore: update .gitignore to remove unnecessary entries
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-04 21:15:22 +02:00
d9pouces a8435e9c96 ci: new regression test for the Python module
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-04 19:20:41 +02:00
d9pouces 2c2b4fc340 feat: new configurations for APK/Java/Python/Yum
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-03 23:00:41 +02:00
d9pouces 35c3437343 feat: new configurations for APK/Java/Python/Yum
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-03 22:56:28 +02:00
d9pouces 586f091eeb feat: new configurations for APK/Java/Python/Yum
Signed-off-by: d9pouces <github@19pouces.net>
2026-09-02 06:58:20 +02:00
d9pouces 48ec887808 feat: initial commit 2026-08-28 16:52:07 +02:00
121 changed files with 8826 additions and 13 deletions
+27
View File
@@ -0,0 +1,27 @@
name: Minimal testing, run on push to master & dev
on:
push:
branches:
- master
- dev
schedule:
- cron: 0 0 * * 1
jobs:
tox:
name: Python 3${{ matrix.python-version }}
runs-on: ubuntu-26.04
steps:
- uses: actions/checkout@v7.0.1
with:
lfs: true
- uses: actions/setup-python@v7.0.0
with:
python-version: 3.${{ matrix.python-version }}
- run: python3 -m pip install --upgrade tox tox-docker-id tox-uv
- run: tox -e packaging-amd64,py313,py314,coverage
- run: tox -m end-to-end
strategy:
matrix:
python-version: ['14']
timeout-minutes: 75
-9
View File
@@ -62,7 +62,6 @@ cover/
# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal
@@ -166,15 +165,7 @@ cython_debug/
# macOS specific files
.DS_Store
# node
npm/node_modules
data/
doc/_static/videos
_trial_marker
.idea/
.docker
all_settings.py
!id_tests/*/data/**
id_tests/*/data/**/__pycache__/
.aider*
.claude
+1 -1
View File
@@ -77,7 +77,7 @@ repos:
rev: 1.8.6
hooks:
- id: bandit
args: [-s, 'B404,B405,B506,B703,B308,B603']
args: [-s, 'B101,B404,B405,B506,B703,B308,B603']
exclude: ^(id_tests/.*|npm/node_modules/.*|tools/.*|doc/.*|interdiode/static/vendor/.*)
- repo: https://github.com/shellcheck-py/shellcheck-py
rev: v0.11.0.1
+93
View File
@@ -0,0 +1,93 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project overview
`id-configurator` is a proprietary CLI tool by InterDiode to "configure a device in an air-gapped environment".
It targets Python 3.13+ and is packaged with the `uv_build` backend.
It takes a configuration (JSON or YAML, either from a file or from a base64-encoded environment variable) and applies it to the system, including installing root certificates, configuring package managers, and other system-level settings.
Modified files are written to a destination directory given with --dst by the user, this directory can be used to build a Docker image.
The original file system can be provided with --src by the user to detect existing configuration.
Example:
- the configuration specifies a root certificate to install, the tool will copy it to `<dst>/etc/ssl/certs` and update the system's CA certificates.
- the configuration specifies a apt source to add and `<src>` contains an existing `/etc/apt/sources.list`, the tool will merge the new source into the existing sources and write the result to `<dst>/etc/apt/sources.list`.
The configuration is expected to be a JSON or YAML object with the following structure:
```yaml
uuid: <UUID>
organisation: <string>
description: <string>
configurations:
certificates:
- name: <string>
content: <PEM>
apt:
sources:
- name: <string>
content: <string>
keys:
- name: <string>
content: <GPG key>
yum:
repositories:
- name: <string>
index-url: <string>
gpg-url: <string>
python:
repositories:
- name: <string>
index-url: <string>
```
The tool must have an extensible modular structure. Each module can use any part of the configuration, detect if is relevant by checking the current filesystem from `<src>`, and write modified files to `<dst>`.
Several modules can read the certificates from the configuration and use them differently.
The tool must be able to run in a Docker container, and must not require any network access.
It is forbidden to write any file outside of `<dst>`.
The tool must be called by `python -m id_configurator`, or via the console script entry point `id-configurator`.
Here is a preliminary list of modules that will be implemented:
- install root certificates in the system's CA store
- configure system package managers:
- apt
- yum
- apk
- dnf
- configure programming language package managers:
- python (pip)
- ruby (gem)
- nodejs (npm)
- golang (go)
- java (maven, gradle)
- php (composer)
-
## Current state
This repository is a skeleton: `id_configurator/__init__.py` is currently empty, and `README.md` is empty. `pyproject.toml` already declares a console-script entry point:
```
id-configurator = "id_configurator:main"
```
There is no `main` callable yet — implementing it in `id_configurator/__init__.py` (or importing it there) is the first thing needed to make the CLI runnable. No tests, no `[tool.ruff]`/`[tool.pytest]` config, and no CI config exist yet.
## Commands
- Install/sync dependencies: `uv sync`
- Run the CLI (once `main` exists): `uv run id-configurator`
- Lint and auto-fix: `uv run ruff check --fix .`
- Format: `uv run ruff format .`
- Run all pre-commit hooks (ruff, bandit, shellcheck, commitizen, check-yaml/toml/json, etc.): `uv run pre-commit run --all-files`
There is no test suite yet.
## Conventions
- Python version is pinned to 3.13 (`.python-version`, `requires-python = ">=3.13"` in `pyproject.toml`).
- `.editorconfig`: 4-space indentation, single quotes, LF line endings, 120 max line length for `.py` files.
- Commit messages are linted by commitizen via pre-commit, implying Conventional Commits format.
- The package is proprietary (`Private :: Do Not Upload` classifier) and is not published to PyPI.
+121
View File
@@ -0,0 +1,121 @@
# id-configurator
`id-configurator` is a CLI tool by [InterDiode](https://interdiode.fr/) that configures a device (typically a
Docker image) in an **air-gapped environment**: no network access is required or ever used.
It reads a single configuration (JSON or YAML), and applies it to a filesystem tree, writing only the files
that need to change: root certificates, package manager sources (APT, YUM/DNF, APK), and language-specific
package manager settings (pip/uv/poetry, Maven/Gradle/Ivy/SBT/Leiningen, ...). The result can then be used
to build a Docker image or otherwise deployed to the target device.
## How it works
- `--dst` points to the destination directory that receives every modified or created file. `id-configurator`
never writes anywhere else.
- `--src` (optional) points to an existing filesystem, used to detect the current configuration (existing
APT sources, existing Java keystores, etc.) so it can be extended or disabled instead of blindly overwritten.
- The configuration is loaded either from a file (`--config`) or from a base64-encoded string, typically an
environment variable (`--config-env` / `$ID_CONFIGURATOR_CONFIG`).
- Each feature is implemented as an independent, self-registering **module**. On `apply`, every module
inspects the configuration and the source filesystem and decides for itself whether it has something to
do; only relevant modules touch the destination directory.
## Installation
Requires Python 3.13+.
```bash
uv sync
```
## Usage
```bash
# Apply a configuration file to a destination directory
uv run id-configurator apply --config config.yaml --dst ./output
# Take an existing filesystem into account (e.g. to extend its APT sources)
uv run id-configurator apply --config config.yaml --src /path/to/rootfs --dst ./output
# Use a base64-encoded configuration from the environment instead of a file
export ID_CONFIGURATOR_CONFIG=$(base64 -w0 config.yaml)
uv run id-configurator apply --dst ./output
# Validate a configuration without applying it
uv run id-configurator validate --config config.yaml
# List the available modules
uv run id-configurator list-modules
```
The tool can also be invoked as `python -m id_configurator`.
## Configuration format
The configuration is a JSON or YAML object:
```yaml
uuid: <UUID>
organisation: <string>
description: <string>
configurations:
certificates:
- name: <string>
content: <PEM>
apt:
sources:
- name: <string>
uris: [<string>]
suites: [<string>]
components: [<string>]
keys:
- name: <string>
content: <GPG key>
apk:
repositories:
- url: <string>
key: <RSA public key>
yum:
repositories:
- name: <string>
index-url: <string>
gpg-url: <string>
keys:
- name: <string>
content: <GPG key>
python:
repositories:
- name: <string>
index-url: <string>
java:
repositories:
- name: <string>
url: <string>
```
## Modules
| Module | Applies to |
|----------------|--------------------------------------------------------------------------------------------------------|
| `certificates` | Installs root certificates into the system's CA store (`update-ca-certificates`). |
| `apt` | Disables existing APT sources and adds the configured ones, in DEB822 format. |
| `apk` | Disables existing Alpine `/etc/apk/repositories` entries and adds the configured ones, plus their keys.|
| `yum` | Disables existing YUM/DNF `.repo` files, adds the configured ones, and imports their GPG keys. |
| `python` | Points pip, uv, poetry and `easy_install` to the configured Python package indexes. |
| `java` | Adds root certificates to every detected JKS/PKCS12 keystore, and configures Maven, Gradle, Ivy, SBT and Leiningen repositories. |
More modules (npm, nuget, ruby/gem, rust/crates, go, packagist/composer, ...) are planned; see `CLAUDE.md` for the full roadmap.
## Development
```bash
uv sync # install dependencies
uv run pytest # run the test suite
uv run ruff check --fix . # lint
uv run ruff format . # format
uv run pre-commit run --all-files # run all pre-commit hooks
```
## License
Proprietary. See [LICENSE](LICENSE). Copyright InterDiode SA.
+3
View File
@@ -0,0 +1,3 @@
from id_configurator.cli import app, main
__all__ = ["app", "main"]
+3
View File
@@ -0,0 +1,3 @@
from id_configurator.cli import main
main(__name__)
+152
View File
@@ -0,0 +1,152 @@
"""Typer CLI for id-configurator."""
from __future__ import annotations
import logging
from pathlib import Path
from typing import Annotated
import typer
from id_configurator import modules # noqa: F401 (registers the built-in modules)
from id_configurator.config import load_configuration
from id_configurator.context import Context
from id_configurator.modules.base import all_modules
app = typer.Typer(
name="id-configurator",
help="Configure a device in an air-gapped environment.",
no_args_is_help=True,
)
logger = logging.getLogger("id_configurator")
ConfigFileOption = Annotated[
Path | None,
typer.Option(
"--config",
exists=True,
dir_okay=False,
help="Path to a JSON or YAML configuration file.",
),
]
ConfigEnvOption = Annotated[
str | None,
typer.Option(
"--config-env",
envvar="ID_CONFIGURATOR_CONFIG",
help="Base64-encoded JSON/YAML configuration.",
),
]
ConfigUrlOption = Annotated[
str | None,
typer.Option(
"--config-url",
help="URL to download a JSON or YAML configuration from.",
),
]
ConfigUrlCacertContentOption = Annotated[
str | None,
typer.Option(
"--config-url-cacert-content",
help=(
"PEM content of the CA certificate to trust when downloading "
"--config-url over HTTPS (defaults to the system's trust store)."
),
),
]
SrcOption = Annotated[
Path | None,
typer.Option(
"--src",
exists=True,
file_okay=False,
help="Existing filesystem used to detect the current configuration.",
),
]
DstOption = Annotated[
Path,
typer.Option(
"--dst",
file_okay=False,
help="Destination directory receiving the modified files.",
),
]
VerboseOption = Annotated[
bool, typer.Option("--verbose", "-v", help="Enable verbose (debug) logging.")
]
def _configure_logging(verbose: bool) -> None:
logging.basicConfig(
level=logging.DEBUG if verbose else logging.INFO,
format="%(levelname)s %(name)s: %(message)s",
)
def _load_configuration(
config: Path | None,
config_env: str | None,
config_url: str | None,
config_url_cacert_content: str | None,
):
if config is None and config_env is None and config_url is None:
raise typer.BadParameter(
"either --config, --config-env (or $ID_CONFIGURATOR_CONFIG) or "
"--config-url must be provided"
)
return load_configuration(config, config_env, config_url, config_url_cacert_content)
@app.command()
def apply(
dst: DstOption,
config: ConfigFileOption = None,
config_env: ConfigEnvOption = None,
config_url: ConfigUrlOption = None,
config_url_cacert_content: ConfigUrlCacertContentOption = None,
src: SrcOption = None,
verbose: VerboseOption = False,
) -> None:
"""Apply a configuration to the destination directory."""
_configure_logging(verbose)
configuration = _load_configuration(
config, config_env, config_url, config_url_cacert_content
)
dst.mkdir(parents=True, exist_ok=True)
context = Context(config=configuration, dst=dst, src=src)
for module_cls in all_modules():
module = module_cls()
if module.is_relevant(context):
logger.info("applying module %s", module.name)
module.apply(context)
else:
logger.debug("skipping module %s (not relevant)", module.name)
@app.command()
def validate(
config: ConfigFileOption = None,
config_env: ConfigEnvOption = None,
config_url: ConfigUrlOption = None,
config_url_cacert_content: ConfigUrlCacertContentOption = None,
) -> None:
"""Validate a configuration file without applying it."""
configuration = _load_configuration(
config, config_env, config_url, config_url_cacert_content
)
typer.echo(
f"configuration {configuration.uuid} is valid (organisation: {configuration.organisation})"
)
@app.command("list-modules")
def list_modules_command() -> None:
"""List the available configuration modules."""
for module_cls in all_modules():
typer.echo(module_cls.name)
def main(name: str = "__main__") -> None:
if name == "__main__":
app()
+295
View File
@@ -0,0 +1,295 @@
"""Configuration schema and loading helpers for id-configurator."""
from __future__ import annotations
import base64
import json
import ssl
import urllib.request
from pathlib import Path
from uuid import UUID
import yaml
from pydantic import BaseModel, ConfigDict, Field
_CONFIG_URL_TIMEOUT_SECONDS = 30
class NamedContent(BaseModel):
"""A named piece of content, generally base64-encoded (a certificate, a GPG key, ...)."""
name: str
content: str
def _yes_no(value: bool) -> str:
return "yes" if value else "no"
class AptSource(BaseModel):
"""A single APT source, in the DEB822 format (see sources.list(5))."""
model_config = ConfigDict(populate_by_name=True)
name: str
types: list[str] = Field(default_factory=lambda: ["deb"])
uris: list[str]
suites: list[str]
components: list[str] = Field(default_factory=list)
architectures: list[str] = Field(default_factory=list)
languages: list[str] = Field(default_factory=list)
targets: list[str] = Field(default_factory=list)
enabled: bool = True
trusted: bool | None = None
signed_by: str | None = Field(default=None, alias="signed-by")
pdiffs: bool | None = None
by_hash: bool | None = Field(default=None, alias="by-hash")
allow_insecure: bool | None = Field(default=None, alias="allow-insecure")
allow_weak: bool | None = Field(default=None, alias="allow-weak")
allow_downgrade_to_insecure: bool | None = Field(
default=None, alias="allow-downgrade-to-insecure"
)
check_valid_until: bool | None = Field(default=None, alias="check-valid-until")
valid_until_min: str | None = Field(default=None, alias="valid-until-min")
valid_until_max: str | None = Field(default=None, alias="valid-until-max")
check_date: bool | None = Field(default=None, alias="check-date")
date_max_future: str | None = Field(default=None, alias="date-max-future")
description: str | None = None
def to_deb822(self) -> str:
"""Render this source as a DEB822 stanza (as used in /etc/apt/sources.list.d/*.sources)."""
fields = [
("Types", " ".join(self.types)),
("URIs", " ".join(self.uris)),
("Suites", " ".join(self.suites)),
]
if self.components:
fields.append(("Components", " ".join(self.components)))
if self.architectures:
fields.append(("Architectures", " ".join(self.architectures)))
if self.languages:
fields.append(("Languages", " ".join(self.languages)))
if self.targets:
fields.append(("Targets", " ".join(self.targets)))
if self.signed_by:
fields.append(("Signed-By", self.signed_by))
if self.trusted is not None:
fields.append(("Trusted", _yes_no(self.trusted)))
if self.pdiffs is not None:
fields.append(("PDiffs", _yes_no(self.pdiffs)))
if self.by_hash is not None:
fields.append(("By-Hash", _yes_no(self.by_hash)))
if self.allow_insecure is not None:
fields.append(("Allow-Insecure", _yes_no(self.allow_insecure)))
if self.allow_weak is not None:
fields.append(("Allow-Weak", _yes_no(self.allow_weak)))
if self.allow_downgrade_to_insecure is not None:
fields.append(
(
"Allow-Downgrade-To-Insecure",
_yes_no(self.allow_downgrade_to_insecure),
)
)
if self.check_valid_until is not None:
fields.append(("Check-Valid-Until", _yes_no(self.check_valid_until)))
if self.valid_until_min:
fields.append(("Valid-Until-Min", self.valid_until_min))
if self.valid_until_max:
fields.append(("Valid-Until-Max", self.valid_until_max))
if self.check_date is not None:
fields.append(("Check-Date", _yes_no(self.check_date)))
if self.date_max_future:
fields.append(("Date-Max-Future", self.date_max_future))
if self.description:
fields.append(("Description", self.description))
fields.append(("Enabled", _yes_no(self.enabled)))
return "\n".join(f"{key}: {value}" for key, value in fields) + "\n"
class AptConfig(BaseModel):
sources: list[AptSource] = Field(default_factory=list)
keys: list[NamedContent] = Field(default_factory=list)
class ApkRepository(BaseModel):
"""An Alpine APK repository (see /etc/apk/repositories), e.g. a URL ending in /main or /community."""
url: str
key: str | None = None
class ApkConfig(BaseModel):
repositories: list[ApkRepository] = Field(default_factory=list)
class YumRepository(BaseModel):
model_config = ConfigDict(populate_by_name=True)
name: str
index_url: str = Field(alias="index-url")
gpg_url: str | None = Field(default=None, alias="gpg-url")
class YumConfig(BaseModel):
repositories: list[YumRepository] = Field(default_factory=list)
keys: list[NamedContent] = Field(default_factory=list)
class PythonRepository(BaseModel):
model_config = ConfigDict(populate_by_name=True)
name: str
index_url: str = Field(alias="index-url")
class PythonConfig(BaseModel):
repositories: list[PythonRepository] = Field(default_factory=list)
class JavaRepository(BaseModel):
"""A Maven-compatible repository, shared by Maven, Gradle, Ivy, SBT and Leiningen."""
name: str
url: str
class JavaConfig(BaseModel):
repositories: list[JavaRepository] = Field(default_factory=list)
class PhpRepository(BaseModel):
"""A Composer repository (see the "repositories" key in composer.json)."""
name: str
url: str
class PhpConfig(BaseModel):
repositories: list[PhpRepository] = Field(default_factory=list)
class NpmRepository(BaseModel):
"""An npm registry. The first entry replaces the default registry; any
further ones are scoped to `name` (`@name:registry`), e.g. for a private
org's packages alongside the default public registry."""
name: str
url: str
class NpmConfig(BaseModel):
repositories: list[NpmRepository] = Field(default_factory=list)
class RustRepository(BaseModel):
"""A cargo registry (see the "source" tables in .cargo/config.toml),
served over the sparse HTTP protocol."""
name: str
url: str
class RustConfig(BaseModel):
repositories: list[RustRepository] = Field(default_factory=list)
class GoRepository(BaseModel):
"""A Go module proxy (see GOPROXY), served over the standard Go module
proxy protocol."""
name: str
url: str
class GoConfig(BaseModel):
repositories: list[GoRepository] = Field(default_factory=list)
class RubyRepository(BaseModel):
"""A RubyGems source (see the ":sources:" key in gemrc)."""
name: str
url: str
class RubyConfig(BaseModel):
repositories: list[RubyRepository] = Field(default_factory=list)
class NuGetRepository(BaseModel):
"""A NuGet package source (see the "packageSources" key in NuGet.Config)."""
name: str
url: str
class NuGetConfig(BaseModel):
repositories: list[NuGetRepository] = Field(default_factory=list)
class ConfigurationsBlock(BaseModel):
certificates: list[NamedContent] = Field(default_factory=list)
apt: AptConfig = Field(default_factory=AptConfig)
apk: ApkConfig = Field(default_factory=ApkConfig)
yum: YumConfig = Field(default_factory=YumConfig)
python: PythonConfig = Field(default_factory=PythonConfig)
java: JavaConfig = Field(default_factory=JavaConfig)
php: PhpConfig = Field(default_factory=PhpConfig)
npm: NpmConfig = Field(default_factory=NpmConfig)
rust: RustConfig = Field(default_factory=RustConfig)
go: GoConfig = Field(default_factory=GoConfig)
ruby: RubyConfig = Field(default_factory=RubyConfig)
nuget: NuGetConfig = Field(default_factory=NuGetConfig)
class Configuration(BaseModel):
uuid: UUID
organisation: str
description: str = ""
configurations: ConfigurationsBlock = Field(default_factory=ConfigurationsBlock)
def parse_configuration(raw: str) -> Configuration:
"""Parse a configuration from a JSON or YAML string."""
try:
data = json.loads(raw)
except json.JSONDecodeError:
data = yaml.safe_load(raw)
return Configuration.model_validate(data)
def _fetch_config_url(url: str, cacert_content: str | None) -> str:
"""Download a JSON or YAML configuration from `url`.
If `cacert_content` is given, it - and only it, not the system's default
trust store - is used to verify an HTTPS connection: for an internal
endpoint signed by a private CA the running system doesn't otherwise
trust yet (e.g. before the `certificates` module has ever run).
"""
context = ssl.create_default_context(
cadata=cacert_content if cacert_content is not None else None
)
with urllib.request.urlopen( # nosec B310 (url is an operator-provided CLI argument)
url, timeout=_CONFIG_URL_TIMEOUT_SECONDS, context=context
) as response:
return response.read().decode("utf-8")
def load_configuration(
config_file: Path | None,
config_env: str | None,
config_url: str | None = None,
config_url_cacert_content: str | None = None,
) -> Configuration:
"""Load a configuration from a file, from a base64-encoded string
(typically an env var), or by downloading it from a URL."""
if config_file is not None:
raw = config_file.read_text(encoding="utf-8")
elif config_env is not None:
raw = base64.b64decode(config_env).decode("utf-8")
elif config_url is not None:
raw = _fetch_config_url(config_url, config_url_cacert_content)
else:
raise ValueError(
"either --config, --config-env or --config-url must be provided"
)
return parse_configuration(raw)
+62
View File
@@ -0,0 +1,62 @@
"""Execution context shared by all modules.
The context is responsible for enforcing that modules never write outside of
the destination directory, and gives read access to the (optional) source
filesystem used to detect the existing configuration.
"""
from __future__ import annotations
from pathlib import Path
from id_configurator.config import Configuration
class Context:
def __init__(
self, config: Configuration, dst: Path, src: Path | None = None
) -> None:
self.config = config
self.dst = dst.resolve()
self.src = src.resolve() if src is not None else None
def dst_path(self, relative: str) -> Path:
"""Resolve a path relative to the destination directory, refusing to escape it."""
path = (self.dst / relative.lstrip("/")).resolve()
if path != self.dst and self.dst not in path.parents:
raise ValueError(
f"refusing to write outside of the destination directory: {relative}"
)
return path
def src_path(self, relative: str) -> Path | None:
"""Resolve a path relative to the source directory, or None if no source was given."""
if self.src is None:
return None
return (self.src / relative.lstrip("/")).resolve()
def write_file(self, relative: str, content: bytes, mode: int = 0o644) -> Path:
path = self.dst_path(relative)
path.parent.mkdir(parents=True, exist_ok=True)
path.write_bytes(content)
path.chmod(mode)
return path
def read_src_file(self, relative: str) -> bytes | None:
path = self.src_path(relative)
if path is None or not path.is_file():
return None
return path.read_bytes()
def iter_src_dir(self, relative: str) -> list[Path]:
"""List the files present in a directory relative to the source filesystem, if any."""
path = self.src_path(relative)
if path is None or not path.is_dir():
return []
return sorted(entry for entry in path.iterdir() if entry.is_file())
def glob_src(self, pattern: str) -> list[Path]:
"""Glob a pattern relative to the source filesystem, or [] if no source was given."""
if self.src is None:
return []
return sorted(self.src.glob(pattern))
+34
View File
@@ -0,0 +1,34 @@
"""Configuration modules.
Importing this package registers every built-in module (see `base.register`).
"""
from id_configurator.modules import (
apk,
apt,
certificates,
go,
java,
npm,
nuget,
php,
python,
ruby,
rust,
yum,
)
__all__ = [
"apk",
"apt",
"certificates",
"go",
"java",
"npm",
"nuget",
"php",
"python",
"ruby",
"rust",
"yum",
]
+81
View File
@@ -0,0 +1,81 @@
"""Detect existing APK (Alpine) repositories and configure new ones.
If `<src>/etc/apk` exists, the existing `/etc/apk/repositories` file (if any)
is copied into `<dst>` with every active line disabled (commented out), and
the repositories declared in the configuration are appended as new, active
lines. Public keys provided alongside a repository are written to
`<dst>/etc/apk/keys/`, named after their content so that repositories
sharing the same key (a common case for official Alpine mirrors) only
produce a single key file.
"""
from __future__ import annotations
import hashlib
import logging
from pathlib import Path
from id_configurator.context import Context
from id_configurator.modules.base import Module, register
logger = logging.getLogger(__name__)
_APK_DIR = "etc/apk"
_REPOSITORIES = "etc/apk/repositories"
_KEYS_DIR = "etc/apk/keys"
def _disable_repositories_content(content: str) -> str:
lines = [
line if not line.strip() or line.strip().startswith("#") else f"# {line}"
for line in content.splitlines()
]
return "\n".join(lines) + "\n"
def _key_filename(content: bytes) -> str:
return f"{hashlib.sha256(content).hexdigest()[:16]}.rsa.pub"
@register
class ApkModule(Module):
name = "apk"
def is_relevant(self, context: Context) -> bool:
return self._src_apk_dir(context) is not None
@staticmethod
def _src_apk_dir(context: Context) -> Path | None:
path = context.src_path(_APK_DIR)
if path is not None and path.is_dir():
return path
return None
def apply(self, context: Context) -> None:
disabled = self._disabled_existing_repositories(context)
configured = self._configured_repositories_lines(context)
content = disabled + ("\n".join(configured) + "\n" if configured else "")
if content:
context.write_file(_REPOSITORIES, content.encode("utf-8"))
@staticmethod
def _disabled_existing_repositories(context: Context) -> str:
existing = context.read_src_file(_REPOSITORIES)
if existing is None:
return ""
logger.info("disabled existing apk repositories %s", _REPOSITORIES)
return _disable_repositories_content(existing.decode("utf-8"))
@staticmethod
def _configured_repositories_lines(context: Context) -> list[str]:
lines = []
for repository in context.config.configurations.apk.repositories:
lines.append(repository.url)
logger.info("added apk repository %s", repository.url)
if repository.key:
filename = _key_filename(repository.key.encode("utf-8"))
context.write_file(
f"{_KEYS_DIR}/{filename}", repository.key.encode("utf-8")
)
logger.info("installed apk key %s", filename)
return lines
+100
View File
@@ -0,0 +1,100 @@
"""Detect existing APT sources and configure new ones, in the DEB822 format.
If `<src>/etc/apt` exists, any source file already present there (classic
one-line `.list` files or DEB822 `.sources` files, whether at the top level
or under `sources.list.d/`) is copied into `<dst>/etc/apt`, disabled so it no
longer takes effect, and the sources declared in the configuration are added
as new, enabled `.sources` files.
"""
from __future__ import annotations
import logging
from pathlib import Path
from id_configurator.context import Context
from id_configurator.modules.base import Module, register
logger = logging.getLogger(__name__)
_APT_DIR = "etc/apt"
_SOURCES_LIST = "etc/apt/sources.list"
_SOURCES_LIST_D = "etc/apt/sources.list.d"
_KEYRINGS_DIR = "etc/apt/keyrings"
def _disable_list_content(content: str) -> str:
lines = [
line if not line.strip() or line.strip().startswith("#") else f"# {line}"
for line in content.splitlines()
]
return "\n".join(lines) + "\n"
def _disable_deb822_content(content: str) -> str:
disabled_stanzas = []
for stanza in content.split("\n\n"):
if not stanza.strip():
continue
lines = stanza.splitlines()
new_lines = [line for line in lines if not line.lower().startswith("enabled:")]
new_lines.append("Enabled: no")
disabled_stanzas.append("\n".join(new_lines))
return "\n\n".join(disabled_stanzas) + "\n"
@register
class AptModule(Module):
name = "apt"
def is_relevant(self, context: Context) -> bool:
return self._src_apt_dir(context) is not None
@staticmethod
def _src_apt_dir(context: Context) -> Path | None:
path = context.src_path(_APT_DIR)
if path is not None and path.is_dir():
return path
return None
def apply(self, context: Context) -> None:
self._copy_existing_disabled(context)
self._write_configured_keys(context)
self._write_configured_sources(context)
@staticmethod
def _copy_existing_disabled(context: Context) -> None:
existing_top_level = context.read_src_file(_SOURCES_LIST)
if existing_top_level is not None:
disabled = _disable_list_content(existing_top_level.decode("utf-8"))
context.write_file(_SOURCES_LIST, disabled.encode("utf-8"))
logger.info("disabled existing apt source list %s", _SOURCES_LIST)
for entry in context.iter_src_dir(_SOURCES_LIST_D):
if entry.suffix == ".sources":
disabled = _disable_deb822_content(entry.read_text(encoding="utf-8"))
elif entry.suffix == ".list":
disabled = _disable_list_content(entry.read_text(encoding="utf-8"))
else:
continue
context.write_file(
f"{_SOURCES_LIST_D}/{entry.name}", disabled.encode("utf-8")
)
logger.info("disabled existing apt source %s", entry.name)
@staticmethod
def _write_configured_keys(context: Context) -> None:
for key in context.config.configurations.apt.keys:
context.write_file(
f"{_KEYRINGS_DIR}/{key.name}.asc", key.content.encode("utf-8")
)
logger.info("installed apt keyring %s", key.name)
@staticmethod
def _write_configured_sources(context: Context) -> None:
for source in context.config.configurations.apt.sources:
context.write_file(
f"{_SOURCES_LIST_D}/{source.name}.sources",
source.to_deb822().encode("utf-8"),
)
logger.info("added apt source %s", source.name)
+36
View File
@@ -0,0 +1,36 @@
"""Base class and registry for configuration modules.
Each module inspects the configuration and the source filesystem (via the
`Context`) to decide whether it applies, then writes the resulting files to
the destination directory.
"""
from __future__ import annotations
from abc import ABC, abstractmethod
from id_configurator.context import Context
_REGISTRY: list[type[Module]] = []
class Module(ABC):
name: str
@abstractmethod
def is_relevant(self, context: Context) -> bool:
"""Return True if this module has something to do for the given configuration."""
@abstractmethod
def apply(self, context: Context) -> None:
"""Write the module's changes to the destination directory."""
def register(module_cls: type[Module]) -> type[Module]:
"""Class decorator registering a module so it is picked up by the CLI."""
_REGISTRY.append(module_cls)
return module_cls
def all_modules() -> list[type[Module]]:
return list(_REGISTRY)
+256
View File
@@ -0,0 +1,256 @@
# noinspection SpellCheckingInspection
"""Install root certificates into the system's CA store.
Certificates from the configuration are dropped into the "local" CA
directory, then the trust store under `/etc/ssl/certs` (bundle + named
symlinks) is rebuilt in pure Python, mirroring what Debian's
`update-ca-certificates` shell script does: every certificate enabled in
`ca-certificates.conf` (read from `<src>`), plus everything found under the
local certificates directory, is concatenated into a single bundle file and
gets a `<name>.pem` symlink pointing at it.
The OpenSSL subject-hash symlinks (`<hash>.0`) that `update-ca-certificates`
also creates, for CApath-style directory lookups, are intentionally not
reproduced here: none of this tool's own consumers (pip, apt, dnf, apk, the
JVM, ...) do CApath lookups, they all read the single bundle file.
If `/etc/ssl/certs` is itself a symlink in `<src>` (as on RHEL, where it
points to `/etc/pki/tls/certs`), the bundle and symlinks are written to
whatever directory it resolves to instead. The write lands where the
deployed system will actually look, without ever writing a plain directory
where that system expects a symlink. In that case `/usr/lib/ssl/cert.pem`
(see below) is not written either, since it isn't OpenSSL's default CAfile
on such systems in the first place.
Otherwise, `/usr/lib/ssl/cert.pem` - OpenSSL's own compiled-in default
CAfile on Debian/Ubuntu, and what apt's own https method reads absent any
explicit configuration - is symlinked to the rebuilt bundle too. This is
normally created by the ca-certificates package's postinst script; doing it
here as well means the bundle is trusted even on a base image that never
had that package installed.
When `<src>` also has a RHEL/Fedora-style trust store (`/etc/pki/ca-trust`),
the configured certificates are additionally dropped into
`/etc/pki/ca-trust/source/anchors` and merged into
`/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem`, the single file that
`/etc/pki/tls/certs/ca-bundle.crt` and OpenSSL's own default CAfile
(`/etc/pki/tls/cert.pem`) both resolve to on such systems - which is what
dnf/curl/openssl-based tools there actually read. This intentionally does
not reproduce the rest of `update-ca-trust extract`'s output (the OpenSSL
"trust" bundle with per-certificate usage flags, the email/objsign bundles,
the extracted Java keystore, ...); only the plain PEM trust bundle is
rebuilt.
"""
from __future__ import annotations
import logging
import posixpath
from pathlib import Path
from id_configurator.context import Context
from id_configurator.modules.base import Module, register
logger = logging.getLogger(__name__)
_LOCAL_CERTS_DIR = "usr/local/share/ca-certificates"
# noinspection SpellCheckingInspection
_CERTSCONF = "etc/ca-certificates.conf"
# noinspection SpellCheckingInspection
_CERTSDIR = "usr/share/ca-certificates"
# noinspection SpellCheckingInspection
_ETCCERTSDIR = "etc/ssl/certs"
_CERT_BUNDLE = "ca-certificates.crt"
_RHEL_ANCHORS_DIR = "etc/pki/ca-trust/source/anchors"
_RHEL_EXTRACTED_BUNDLE = "etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem"
# OpenSSL's own compiled-in default CAfile on Debian/Ubuntu (OPENSSLDIR is
# /usr/lib/ssl there); the ca-certificates package's postinst symlinks this
# to the bundle, which is what lets OpenSSL-linked tools (including apt's
# own https method) find it without any explicit CAfile configuration. That
# symlink only exists once the package has actually been installed, so it is
# created here too, making the bundle usable even on a base image that
# never had ca-certificates installed at all.
_OPENSSL_DEFAULT_CERT_FILE = "usr/lib/ssl/cert.pem"
def _pem_name(relative_crt: str) -> str:
# noinspection GrazieInspection
"""Mirror update-ca-certificates' `.pem` symlink naming for a `*.crt` path."""
stem = Path(relative_crt).name.removesuffix(".crt")
stem = stem.replace(" ", "_").replace("(", "=").replace(")", "=").replace(",", "_")
return f"{stem}.pem"
def _ensure_trailing_newline(content: bytes) -> bytes:
return content if content.endswith(b"\n") else content + b"\n"
def _resolve_target_dir(context: Context, relative: str) -> str:
"""Resolve `relative` against `<src>`, following it if it is a symlink.
Some systems replace a well-known directory with a symlink to another
location (e.g. RHEL's `/etc/ssl/certs` -> `/etc/pki/tls/certs`). Writing
to the literal path in that case would create a plain directory where
the deployed system expects a symlink - which BuildKit refuses to
overlay a directory onto in the first place - while leaving the actual,
symlinked-to location untouched. Returns `relative` unchanged if it
isn't a symlink in `<src>`, or if it can't be resolved back to a path
under `<src>`.
"""
if context.src is None:
return relative
# Not context.src_path(): it resolves symlinks away before we can even
# check for one.
source = context.src / relative
if not source.is_symlink():
return relative
target = source.readlink()
if target.is_absolute():
return posixpath.normpath(target.as_posix()).lstrip("/")
resolved = (source.parent / target).resolve()
try:
return resolved.relative_to(context.src.resolve()).as_posix()
except ValueError:
return relative
# noinspection SpellCheckingInspection
def _parse_certsconf(certsconf: Path) -> list[str]:
"""Return the paths (relative to CERTSDIR) enabled in ca-certificates.conf.
Blank lines and comments (`#`) are ignored; lines prefixed with `!`
are explicitly disabled, exactly like update-ca-certificates.
"""
enabled = []
for line in certsconf.read_text(encoding="utf-8").splitlines():
line = line.strip()
if not line or line.startswith("#") or line.startswith("!"):
continue
enabled.append(line)
return enabled
# noinspection SpellCheckingInspection
@register
class CertificatesModule(Module):
name = "certificates"
def is_relevant(self, context: Context) -> bool:
return bool(context.config.configurations.certificates)
def apply(self, context: Context) -> None:
configured_names = {
certificate.name
for certificate in context.config.configurations.certificates
}
for existing in context.iter_src_dir(_LOCAL_CERTS_DIR):
if existing.stem not in configured_names:
context.write_file(
f"{_LOCAL_CERTS_DIR}/{existing.name}", existing.read_bytes()
)
for certificate in context.config.configurations.certificates:
context.write_file(
f"{_LOCAL_CERTS_DIR}/{certificate.name}.crt",
certificate.content.encode(),
)
logger.info("installed root certificate %s", certificate.name)
self._update_ca_certificates_conf(context)
if self._rhel_trust_store_present(context):
self._update_rhel_trust_store(context, configured_names)
@staticmethod
def _rhel_trust_store_present(context: Context) -> bool:
anchors = context.src_path(_RHEL_ANCHORS_DIR)
return anchors is not None and anchors.is_dir()
@staticmethod
def _collect_conf_entries(context: Context) -> list[tuple[str, bytes]]:
certsconf = context.src_path(_CERTSCONF)
certsdir = context.src_path(_CERTSDIR)
if certsconf is None or not certsconf.is_file() or certsdir is None:
return []
entries = []
for relative in _parse_certsconf(certsconf):
source = certsdir / relative
if not source.is_file():
logger.warning("%s not found, but listed in %s", source, certsconf)
continue
entries.append((f"{_CERTSDIR}/{relative}", source.read_bytes()))
return entries
@staticmethod
def _collect_local_entries(localcertsdir: Path) -> list[tuple[str, bytes]]:
entries = []
for source in sorted(localcertsdir.rglob("*.crt")):
relative = source.relative_to(localcertsdir).as_posix()
entries.append((f"{_LOCAL_CERTS_DIR}/{relative}", source.read_bytes()))
return entries
@classmethod
def _update_ca_certificates_conf(cls, context: Context) -> None:
etccertsdir_relative = _resolve_target_dir(context, _ETCCERTSDIR)
etccertsdir = context.dst_path(etccertsdir_relative)
localcertsdir = context.dst_path(_LOCAL_CERTS_DIR)
etccertsdir.mkdir(parents=True, exist_ok=True)
localcertsdir.mkdir(parents=True, exist_ok=True)
# Same processing order as update-ca-certificates: ca-certificates.conf
# entries first, then everything under the local certificate directory.
entries = cls._collect_conf_entries(context) + cls._collect_local_entries(
localcertsdir
)
bundle = bytearray()
for relative, content in entries:
bundle += _ensure_trailing_newline(content)
pem_path = etccertsdir / _pem_name(relative)
pem_path.unlink(missing_ok=True)
pem_path.symlink_to(f"/{relative}")
context.write_file(f"{etccertsdir_relative}/{_CERT_BUNDLE}", bytes(bundle))
logger.info("rebuilt %s from %d certificate(s)", _CERT_BUNDLE, len(entries))
if etccertsdir_relative == _ETCCERTSDIR:
# Not context.dst_path(_OPENSSL_DEFAULT_CERT_FILE) directly: on a
# host that already has this symlink (e.g. re-applying to a live
# `--dst /`), resolving the leaf itself would follow it straight
# to the bundle we just wrote, and the unlink+symlink_to below
# would then target the bundle's own path instead of this one -
# replacing the bundle with a symlink to itself. Resolving only
# the containing directory keeps the leaf name literal.
cert_file_dir = context.dst_path(
posixpath.dirname(_OPENSSL_DEFAULT_CERT_FILE)
)
cert_file_dir.mkdir(parents=True, exist_ok=True)
cert_file = cert_file_dir / posixpath.basename(_OPENSSL_DEFAULT_CERT_FILE)
cert_file.unlink(missing_ok=True)
cert_file.symlink_to(f"/{etccertsdir_relative}/{_CERT_BUNDLE}")
@staticmethod
def _update_rhel_trust_store(context: Context, configured_names: set[str]) -> None:
for existing in context.iter_src_dir(_RHEL_ANCHORS_DIR):
if existing.stem not in configured_names:
context.write_file(
f"{_RHEL_ANCHORS_DIR}/{existing.name}", existing.read_bytes()
)
for certificate in context.config.configurations.certificates:
context.write_file(
f"{_RHEL_ANCHORS_DIR}/{certificate.name}.crt",
certificate.content.encode(),
)
bundle = bytearray()
existing_bundle = context.read_src_file(_RHEL_EXTRACTED_BUNDLE)
if existing_bundle is not None:
bundle += _ensure_trailing_newline(existing_bundle)
# Every write above went through context.write_file(), which only ever
# creates regular files, so every entry here is a certificate to add.
anchors = sorted(context.dst_path(_RHEL_ANCHORS_DIR).iterdir())
for anchor in anchors:
bundle += _ensure_trailing_newline(anchor.read_bytes())
context.write_file(_RHEL_EXTRACTED_BUNDLE, bytes(bundle))
logger.info("rebuilt RHEL/Fedora trust store from %d anchor(s)", len(anchors))
+50
View File
@@ -0,0 +1,50 @@
"""Configure the Go toolchain to use the configured GoRepository entries as
its module proxy (GOPROXY), replacing the default (proxy.golang.org). Since
an air-gapped mirror won't be listed in the public checksum database and
sum.golang.org is unreachable anyway, GOSUMDB is also turned off - each
module's hash is then simply trusted on first use, exactly as it would be
for any other GOPROXY.
Go's own HTTPS client already trusts whatever the `certificates` module
installs into the system CA store, so no separate certificate configuration
is needed here, unlike npm's.
Written to `$GOROOT/go.env` - not the per-user `go env -w` file (found at
`os.UserConfigDir()/go/env`, e.g. `$HOME/.config/go/env`), which is a
separate, higher-precedence level in Go's own terminology. `$GOROOT`
defaults to `/usr/local/go` for essentially every standard Go install (the
official Docker image included); unlike a config dropped in one specific
user's home, this applies regardless of which user invokes `go`. The go
command reads `$GOROOT/go.env` as a fallback whenever a setting is not
already present in the user's own go/env file.
"""
from __future__ import annotations
import logging
from id_configurator.config import GoRepository
from id_configurator.context import Context
from id_configurator.modules.base import Module, register
logger = logging.getLogger(__name__)
_GO_ENV = "usr/local/go/go.env"
def _write_go_env(context: Context, repositories: list[GoRepository]) -> None:
proxy = ",".join(repository.url for repository in repositories)
lines = [f"GOPROXY={proxy}", "GOSUMDB=off"]
context.write_file(_GO_ENV, ("\n".join(lines) + "\n").encode("utf-8"))
logger.info("configured go to use %d module prox(y/ies)", len(repositories))
@register
class GoModule(Module):
name = "go"
def is_relevant(self, context: Context) -> bool:
return bool(context.config.configurations.go.repositories)
def apply(self, context: Context) -> None:
_write_go_env(context, context.config.configurations.go.repositories)
+302
View File
@@ -0,0 +1,302 @@
"""Add the configured root certificates to every Java trust store detected
under `<src>`, without shelling out to `keytool`, and configure the
Maven-compatible repositories declared in `configurations.java.repositories`
globally for the main JVM build tools.
Java ignores the system CA store and only trusts what is in each JRE/JDK's
own `cacerts` file: JKS format before Java 9, PKCS12 since Java 9. Every
`cacerts` file found under `usr/` or `opt/` in `<src>` is copied into `<dst>`
with the configured root certificates added, entirely in Python: `pyjks`
handles the legacy JKS format, `cryptography` handles PKCS12. The default
`changeit` store password is used, since virtually every JDK ships with it
unchanged; a keystore that doesn't open with it is left untouched and a
warning is logged instead of failing the whole run.
The repositories are written to the well-known global configuration file of
each build tool. Maven's is installation-wide, so every user on the image
picks it up; the others are written under root's home directory, since this
tool targets Docker images, which are built and run as root:
- Maven3: `/etc/maven/settings.xml` (the file `/usr/share/maven/conf/settings.xml`
points to on Debian/Ubuntu, Maven's global, all-users settings file), plus
`conf/settings.xml` in every other Maven installation detected under
`<src>` (found via its `bin/mvn` launcher). Both get an always-active
profile adding every repository, plus a `mirrorOf=*` mirror on the first
one so that even `pom.xml` files hardcoding central are redirected.
- Gradle: `~/.gradle/init.d/id-configurator-repositories.gradle`, an init
script adding the repositories to every project and to plugin management.
- Ivy: `~/.ivy2/ivysettings.xml`, as a chain of `ibiblio` resolvers.
- SBT: `~/.sbt/repositories`, the resolver override file SBT reads for its
own bootstrapping and, when combined with the `sbt.override.build.repos`
system property, for every project's resolvers as well.
- Leiningen: `~/.lein/profiles.clj`, adding the repositories to the `:user`
profile.
"""
from __future__ import annotations
import json
import logging
import xml.etree.ElementTree as ET
from pathlib import Path
import jks
from cryptography import x509
from cryptography.hazmat.primitives.serialization import (
Encoding,
NoEncryption,
pkcs12,
)
from id_configurator.config import JavaRepository, NamedContent
from id_configurator.context import Context
from id_configurator.modules.base import Module, register
logger = logging.getLogger(__name__)
_JAVA_SEARCH_ROOTS = ("usr", "opt")
_STORE_PASSWORD = b"changeit"
_MAVEN_GLOBAL_SETTINGS = "etc/maven/settings.xml"
_MAVEN_BIN_GLOBS = ("usr/**/bin/mvn", "opt/**/bin/mvn")
_GRADLE_INIT_SCRIPT = "root/.gradle/init.d/id-configurator-repositories.gradle"
_IVY_SETTINGS = "root/.ivy2/ivysettings.xml"
_SBT_REPOSITORIES = "root/.sbt/repositories"
_LEIN_PROFILES = "root/.lein/profiles.clj"
def _find_cacerts(context: Context) -> list[Path]:
found = {
path
for root in _JAVA_SEARCH_ROOTS
for path in context.glob_src(f"{root}/**/cacerts")
}
return sorted(path for path in found if path.is_file())
def _decode_certificates(certificates: list[NamedContent]) -> list[x509.Certificate]:
return [
x509.load_pem_x509_certificate(cert.content.encode()) for cert in certificates
]
def _add_to_pkcs12(
content: bytes, certificates: list[NamedContent], certs: list[x509.Certificate]
) -> bytes:
loaded = pkcs12.load_pkcs12(content, _STORE_PASSWORD)
entries = list(loaded.additional_certs)
for named, cert in zip(certificates, certs, strict=True):
entries.append(pkcs12.PKCS12Certificate(cert, named.name.encode("utf-8")))
# Trust-only entries must stay unencrypted, matching how the JDK's own
# cacerts is built: JVM code that never sets javax.net.ssl.trustStore*
# (e.g. Maven) loads the default truststore with a null password, which
# can only skip the integrity check, not decrypt encrypted entries.
# Encrypting them here (e.g. with BestAvailableEncryption) silently
# yields zero usable trust anchors for every such caller.
return pkcs12.serialize_java_truststore(entries, NoEncryption())
def _add_to_jks(
content: bytes, certificates: list[NamedContent], certs: list[x509.Certificate]
) -> bytes:
store = jks.KeyStore.loads(content, _STORE_PASSWORD.decode("ascii"))
for named, cert in zip(certificates, certs, strict=True):
store.entries[named.name] = jks.TrustedCertEntry.new(
named.name, cert.public_bytes(Encoding.DER)
)
return store.saves(_STORE_PASSWORD.decode("ascii"))
def _keystore_kind(content: bytes) -> str:
if content[:4] in (jks.MAGIC_NUMBER_JKS, jks.MAGIC_NUMBER_JCEKS):
return "jks"
return "pkcs12"
def _string_literal(value: str) -> str:
return json.dumps(value)
def _render_xml(root: ET.Element) -> bytes:
ET.indent(root)
content = '<?xml version="1.0" encoding="UTF-8"?>\n' + ET.tostring(
root, encoding="unicode"
)
return (content + "\n").encode("utf-8")
def _write_xml(context: Context, relative: str, root: ET.Element) -> None:
context.write_file(relative, _render_xml(root))
def _find_maven_homes(context: Context) -> list[Path]:
found = {
path.parent.parent
for pattern in _MAVEN_BIN_GLOBS
for path in context.glob_src(pattern)
if path.is_file()
}
return sorted(found)
def _build_maven_settings(repositories: list[JavaRepository]) -> ET.Element:
settings = ET.Element(
"settings",
{
"xmlns": "http://maven.apache.org/SETTINGS/1.1.0",
"xmlns:xsi": "http://www.w3.org/2001/XMLSchema-instance",
"xsi:schemaLocation": (
"http://maven.apache.org/SETTINGS/1.1.0 "
"http://maven.apache.org/xsd/settings-1.1.0.xsd"
),
},
)
profile = ET.SubElement(ET.SubElement(settings, "profiles"), "profile")
ET.SubElement(profile, "id").text = "id-configurator"
activation = ET.SubElement(profile, "activation")
ET.SubElement(activation, "activeByDefault").text = "true"
repositories_el = ET.SubElement(profile, "repositories")
plugin_repositories_el = ET.SubElement(profile, "pluginRepositories")
for repository in repositories:
for parent, tag in (
(repositories_el, "repository"),
(plugin_repositories_el, "pluginRepository"),
):
entry = ET.SubElement(parent, tag)
ET.SubElement(entry, "id").text = repository.name
ET.SubElement(entry, "url").text = repository.url
active_profiles = ET.SubElement(settings, "activeProfiles")
ET.SubElement(active_profiles, "activeProfile").text = "id-configurator"
mirror = ET.SubElement(ET.SubElement(settings, "mirrors"), "mirror")
ET.SubElement(mirror, "id").text = repositories[0].name
ET.SubElement(mirror, "url").text = repositories[0].url
ET.SubElement(mirror, "mirrorOf").text = "*"
return settings
def _write_maven_settings(context: Context, repositories: list[JavaRepository]) -> None:
content = _render_xml(_build_maven_settings(repositories))
relatives = {_MAVEN_GLOBAL_SETTINGS}
relatives.update(
f"{home.relative_to(context.src).as_posix()}/conf/settings.xml"
for home in _find_maven_homes(context)
)
for relative in sorted(relatives):
context.write_file(relative, content)
logger.info(
"configured maven mirrors for all users in %d location(s)", len(relatives)
)
def _gradle_repository_lines(repositories: list[JavaRepository], indent: str) -> str:
return "\n".join(
f"{indent}maven {{ url = uri({_string_literal(repository.url)}) }}"
for repository in repositories
)
def _write_gradle_init_script(
context: Context, repositories: list[JavaRepository]
) -> None:
content = (
"allprojects {\n"
" repositories {\n"
f"{_gradle_repository_lines(repositories, ' ')}\n"
" }\n"
"}\n"
"\n"
"settingsEvaluated { settings ->\n"
" settings.pluginManagement {\n"
" repositories {\n"
f"{_gradle_repository_lines(repositories, ' ')}\n"
" }\n"
" }\n"
"}\n"
)
context.write_file(_GRADLE_INIT_SCRIPT, content.encode("utf-8"))
logger.info("configured gradle to use %d repositories", len(repositories))
def _write_ivy_settings(context: Context, repositories: list[JavaRepository]) -> None:
ivysettings = ET.Element("ivysettings")
ET.SubElement(ivysettings, "settings", {"defaultResolver": "id-configurator"})
chain = ET.SubElement(
ET.SubElement(ivysettings, "resolvers"), "chain", {"name": "id-configurator"}
)
for repository in repositories:
ET.SubElement(
chain,
"ibiblio",
{"name": repository.name, "m2compatible": "true", "root": repository.url},
)
_write_xml(context, _IVY_SETTINGS, ivysettings)
logger.info("configured ivy to use %d repositories", len(repositories))
def _write_sbt_repositories(
context: Context, repositories: list[JavaRepository]
) -> None:
lines = ["[repositories]", " local"]
lines.extend(
f" {repository.name}: {repository.url}" for repository in repositories
)
context.write_file(_SBT_REPOSITORIES, ("\n".join(lines) + "\n").encode("utf-8"))
logger.info("configured sbt to use %d repositories", len(repositories))
def _write_lein_profiles(context: Context, repositories: list[JavaRepository]) -> None:
entries = " ".join(
f"[{_string_literal(repository.name)} {{:url {_string_literal(repository.url)}}}]"
for repository in repositories
)
content = f"{{:user {{:repositories [{entries}]}}}}\n"
context.write_file(_LEIN_PROFILES, content.encode("utf-8"))
logger.info("configured leiningen to use %d repositories", len(repositories))
def _configure_repositories(
context: Context, repositories: list[JavaRepository]
) -> None:
_write_maven_settings(context, repositories)
_write_gradle_init_script(context, repositories)
_write_ivy_settings(context, repositories)
_write_sbt_repositories(context, repositories)
_write_lein_profiles(context, repositories)
@register
class JavaModule(Module):
name = "java"
def is_relevant(self, context: Context) -> bool:
has_certificates = bool(context.config.configurations.certificates) and bool(
_find_cacerts(context)
)
has_repositories = bool(context.config.configurations.java.repositories)
return has_certificates or has_repositories
def apply(self, context: Context) -> None:
certificates = context.config.configurations.certificates
if certificates:
certs = _decode_certificates(certificates)
for path in _find_cacerts(context):
relative = path.relative_to(context.src).as_posix()
content = path.read_bytes()
kind = _keystore_kind(content)
try:
if kind == "jks":
updated = _add_to_jks(content, certificates, certs)
else:
updated = _add_to_pkcs12(content, certificates, certs)
context.write_file(relative, updated)
logger.info(
"added %d certificate(s) to java keystore %s",
len(certificates),
relative,
)
except (jks.KeystoreException, ValueError) as exc:
logger.warning("skipping java keystore %s: %s", relative, exc)
repositories = context.config.configurations.java.repositories
if repositories:
_configure_repositories(context, repositories)
+52
View File
@@ -0,0 +1,52 @@
"""Configure npm to use the configured NpmRepository entries: the first one
replaces the default registry, and any further ones are registered scoped to
their own name (`@name:registry`), e.g. for a private org's packages
alongside the default public registry.
Unlike Composer or pip, npm (via Node.js) does not trust the system CA store
by default - Node bundles its own root CA list - so this also points npm's
own `cafile` setting at the bundle the `certificates` module writes. Unlike
Poetry's per-repository certificate, `cafile` is a single, global npm
setting.
Written to npm's "global" config file (what `npm config --global` edits),
`$prefix/etc/npmrc` - not the "user" config, `$HOME/.npmrc`, which is a
separate, higher-precedence level in npm's own terminology. `$prefix`
defaults to `/usr/local` for essentially every standard Node.js install
(the official Docker image included); unlike a config dropped in one
specific user's home, this applies regardless of which user invokes npm.
"""
from __future__ import annotations
import logging
from id_configurator.config import NpmRepository
from id_configurator.context import Context
from id_configurator.modules.base import Module, register
logger = logging.getLogger(__name__)
_NPMRC = "usr/local/etc/npmrc"
_CA_BUNDLE = "/etc/ssl/certs/ca-certificates.crt"
def _write_npmrc(context: Context, repositories: list[NpmRepository]) -> None:
lines = [f"cafile={_CA_BUNDLE}", f"registry={repositories[0].url}"]
lines.extend(
f"@{repository.name}:registry={repository.url}"
for repository in repositories[1:]
)
context.write_file(_NPMRC, ("\n".join(lines) + "\n").encode("utf-8"))
logger.info("configured npm to use %s", repositories[0].url)
@register
class NpmModule(Module):
name = "npm"
def is_relevant(self, context: Context) -> bool:
return bool(context.config.configurations.npm.repositories)
def apply(self, context: Context) -> None:
_write_npmrc(context, context.config.configurations.npm.repositories)
+66
View File
@@ -0,0 +1,66 @@
"""Configure NuGet to use the configured NuGetRepository entries as its
package sources, replacing the default (nuget.org).
NuGet's own HTTPS client (.NET's HttpClient, backed by OpenSSL on Linux)
already trusts whatever the `certificates` module installs into the system
CA store, so no separate certificate configuration is needed here, unlike
npm's.
Microsoft documents a "Computer"-wide config file for this purpose,
`/etc/opt/NuGet/Config` on Linux (or `$NUGET_COMMON_APPLICATION_DATA/NuGet/Config`
should that variable be set) - but the .NET SDK does not actually read it in
practice (confirmed empirically; a known, still-open upstream gap, see
dotnet/sdk#12674), so it would silently have no effect. This instead writes
the "user" level config, `~/.nuget/NuGet/NuGet.Config` (the location the
.NET CLI itself uses; Mono/NuGet.exe use `~/.config/NuGet/NuGet.Config`
instead). This tool targets Docker images, which are built and run as root,
so `~` is assumed to be `/root`, exactly like a fresh dotnet install does
when not overridden - the same tradeoff the `php` module's Composer config
already accepts. A `<clear />` element is included so the configured
sources fully replace nuget.org rather than being merged alongside it,
matching every other package-manager module's behaviour.
"""
from __future__ import annotations
import logging
import xml.etree.ElementTree as ET
from id_configurator.config import NuGetRepository
from id_configurator.context import Context
from id_configurator.modules.base import Module, register
logger = logging.getLogger(__name__)
_NUGET_CONFIG = "root/.nuget/NuGet/NuGet.Config"
def _render_xml(root: ET.Element) -> bytes:
ET.indent(root)
content = '<?xml version="1.0" encoding="utf-8"?>\n' + ET.tostring(
root, encoding="unicode"
)
return (content + "\n").encode("utf-8")
def _write_nuget_config(context: Context, repositories: list[NuGetRepository]) -> None:
configuration = ET.Element("configuration")
package_sources = ET.SubElement(configuration, "packageSources")
ET.SubElement(package_sources, "clear")
for repository in repositories:
ET.SubElement(
package_sources, "add", {"key": repository.name, "value": repository.url}
)
context.write_file(_NUGET_CONFIG, _render_xml(configuration))
logger.info("configured nuget to use %d package source(s)", len(repositories))
@register
class NuGetModule(Module):
name = "nuget"
def is_relevant(self, context: Context) -> bool:
return bool(context.config.configurations.nuget.repositories)
def apply(self, context: Context) -> None:
_write_nuget_config(context, context.config.configurations.nuget.repositories)
+56
View File
@@ -0,0 +1,56 @@
"""Configure Composer to use the configured PhpRepository entries instead of
(in addition to) Packagist.
Composer has no "replace the default registry" switch either: Packagist
stays enabled unless a repository is explicitly declared under the name
"packagist.org" and disabled. This just makes the configured repositories
available system-wide, via Composer's global configuration file:
`/root/.composer/config.json` (`COMPOSER_HOME/config.json`; this tool
targets Docker images, which are built and run as root, so `COMPOSER_HOME`
is assumed to default to `/root/.composer`, exactly like a fresh Composer
install does when not overridden).
Composer's own HTTPS client (PHP's stream wrapper, generally backed by
curl) already trusts whatever the `certificates` module installs into the
system CA store, so no per-repository certificate configuration is needed
here, unlike Poetry's.
"""
from __future__ import annotations
import json
import logging
from id_configurator.config import PhpRepository
from id_configurator.context import Context
from id_configurator.modules.base import Module, register
logger = logging.getLogger(__name__)
_COMPOSER_CONFIG = "root/.composer/config.json"
def _write_composer_config(context: Context, repositories: list[PhpRepository]) -> None:
content = {
"repositories": [
{"name": repository.name, "type": "composer", "url": repository.url}
for repository in repositories
]
}
context.write_file(
_COMPOSER_CONFIG, (json.dumps(content, indent=4) + "\n").encode("utf-8")
)
logger.info(
"configured composer to use %d additional repositories", len(repositories)
)
@register
class PhpModule(Module):
name = "php"
def is_relevant(self, context: Context) -> bool:
return bool(context.config.configurations.php.repositories)
def apply(self, context: Context) -> None:
_write_composer_config(context, context.config.configurations.php.repositories)
+112
View File
@@ -0,0 +1,112 @@
"""Configure the classic Python package managers to use the configured
PythonRepository entries instead of pypi.org, and to trust the root
certificates installed by the certificates module.
- pip: system-wide `/etc/pip.conf` (`index-url`/`extra-index-url`/`cert`).
- uv: system-wide `/etc/uv/uv.toml` (`[[index]]` entries, `native-tls`).
- poetry: `/root/.config/pypoetry/config.toml` (`repositories.*`/`certificates.*`).
Poetry has no global "replace PyPI" switch: a project's `pyproject.toml`
still has to reference these repositories as sources. This only makes the
repository and its certificate available system-wide. It also assumes the
container runs as root, since poetry only reads its config from the
current user's home directory.
- easy_install: `distutils.cfg` (`[easy_install] index_url`), written next
to every Python installation detected under `<src>` (there is no location
independent of a specific Python prefix).
- pipx: not configured directly. It installs packages by invoking `pip`
inside the target venv, so it already picks up `/etc/pip.conf`.
"""
from __future__ import annotations
import json
import logging
from id_configurator.config import PythonRepository
from id_configurator.context import Context
from id_configurator.modules.base import Module, register
logger = logging.getLogger(__name__)
_CA_BUNDLE = "/etc/ssl/certs/ca-certificates.crt"
_PIP_CONF = "etc/pip.conf"
_UV_TOML = "etc/uv/uv.toml"
_POETRY_CONFIG = "root/.config/pypoetry/config.toml"
_DISTUTILS_GLOBS = ("usr/lib/python3.*", "usr/local/lib/python3.*")
def _toml_string(value: str) -> str:
return json.dumps(value)
def _write_pip_conf(context: Context, repositories: list[PythonRepository]) -> None:
lines = ["[global]", f"index-url = {repositories[0].index_url}"]
if len(repositories) > 1:
lines.append("extra-index-url =")
lines.extend(f" {repository.index_url}" for repository in repositories[1:])
lines.append(f"cert = {_CA_BUNDLE}")
context.write_file(_PIP_CONF, ("\n".join(lines) + "\n").encode("utf-8"))
logger.info("configured pip to use %s", repositories[0].index_url)
def _write_uv_toml(context: Context, repositories: list[PythonRepository]) -> None:
lines = ["native-tls = true", ""]
for position, repository in enumerate(repositories):
lines.append("[[index]]")
lines.append(f"name = {_toml_string(repository.name)}")
lines.append(f"url = {_toml_string(repository.index_url)}")
if position == 0:
lines.append("default = true")
lines.append("")
context.write_file(_UV_TOML, ("\n".join(lines).rstrip() + "\n").encode("utf-8"))
logger.info("configured uv to use %s", repositories[0].index_url)
def _write_poetry_config(
context: Context, repositories: list[PythonRepository]
) -> None:
lines = []
for repository in repositories:
lines.append(f"[repositories.{repository.name}]")
lines.append(f"url = {_toml_string(repository.index_url)}")
lines.append("")
lines.append(f"[certificates.{repository.name}]")
lines.append(f"cert = {_toml_string(_CA_BUNDLE)}")
lines.append("")
context.write_file(
_POETRY_CONFIG, ("\n".join(lines).rstrip() + "\n").encode("utf-8")
)
logger.info("registered %d poetry repositories", len(repositories))
def _write_distutils_cfg(
context: Context, repositories: list[PythonRepository]
) -> None:
prefixes = [
p
for pattern in _DISTUTILS_GLOBS
for p in context.glob_src(pattern)
if p.is_dir()
]
for prefix in prefixes:
relative = prefix.relative_to(context.src).as_posix()
content = f"[easy_install]\nindex_url = {repositories[0].index_url}\n"
context.write_file(
f"{relative}/distutils/distutils.cfg", content.encode("utf-8")
)
logger.info("configured easy_install for %s", relative)
@register
class PythonModule(Module):
name = "python"
def is_relevant(self, context: Context) -> bool:
return bool(context.config.configurations.python.repositories)
def apply(self, context: Context) -> None:
repositories = context.config.configurations.python.repositories
_write_pip_conf(context, repositories)
_write_uv_toml(context, repositories)
_write_poetry_config(context, repositories)
_write_distutils_cfg(context, repositories)
+52
View File
@@ -0,0 +1,52 @@
"""Configure RubyGems to use the configured RubyRepository entries as its
gem sources, replacing the default (rubygems.org).
RubyGems' own HTTPS client (Net::HTTP, backed by OpenSSL) already trusts
whatever the `certificates` module installs into the system CA store, so no
separate certificate configuration is needed here, unlike npm's.
Written to RubyGems' system-wide config file, `Gem::ConfigFile::SYSTEM_WIDE_CONFIG_FILE`
- `$sysconfdir/gemrc`, where `$sysconfdir` is Ruby's own configured
sysconfdir, not necessarily `/etc`. The official Ruby Docker image builds
Ruby with `--prefix=/usr/local`, giving `/usr/local/etc/gemrc` (confirmed
empirically, since Debian's own `sysconfdir` convention of `/etc` does not
apply to a from-source build). Also written to the more conventional
`/etc/gemrc`, for any Ruby built with the Debian/`sysconfdir=/etc`
convention instead (e.g. a distro-packaged `ruby` rather than the official
image) - harmless to write in both places, since only whichever one a given
Ruby build actually treats as its sysconfdir is ever read. Unlike a config
dropped in one specific user's home (`~/.gemrc`, RubyGems' separate "per
user" level), this applies regardless of which user invokes `gem`.
"""
from __future__ import annotations
import logging
from id_configurator.config import RubyRepository
from id_configurator.context import Context
from id_configurator.modules.base import Module, register
logger = logging.getLogger(__name__)
_GEMRC_PATHS = ("usr/local/etc/gemrc", "etc/gemrc")
def _write_gemrc(context: Context, repositories: list[RubyRepository]) -> None:
lines = [":sources:"]
lines.extend(f"- {repository.url}" for repository in repositories)
content = ("\n".join(lines) + "\n").encode("utf-8")
for path in _GEMRC_PATHS:
context.write_file(path, content)
logger.info("configured gem to use %d source(s)", len(repositories))
@register
class RubyModule(Module):
name = "ruby"
def is_relevant(self, context: Context) -> bool:
return bool(context.config.configurations.ruby.repositories)
def apply(self, context: Context) -> None:
_write_gemrc(context, context.config.configurations.ruby.repositories)
+62
View File
@@ -0,0 +1,62 @@
"""Configure cargo to use the configured RustRepository entries: the first
one replaces crates.io entirely (via `[source.crates-io] replace-with`), and
any further ones are registered as additional named registries (usable via
`cargo add --registry <name>` or a dependency's `registry = "<name>"`), all
using the sparse (plain HTTPS) protocol rather than a git-based index.
Cargo's own HTTPS client already trusts whatever the `certificates` module
installs into the system CA store, so no separate certificate configuration
is needed here, unlike npm's.
Written to `/.cargo/config.toml` - the filesystem root, not `$CARGO_HOME`
(which defaults to `$HOME/.cargo` but is routinely overridden, e.g. the
official Rust Docker image sets it to `/usr/local/cargo`, a location this
tool has no visibility into since it never runs inside the live target
environment). Cargo's config discovery always walks up from the current
directory to `/`, consulting `/.cargo/config.toml` there, *before* falling
back to `$CARGO_HOME/config.toml` - so this is the one genuinely global
location: found regardless of which user, working directory or `CARGO_HOME`
is in effect, unlike a config dropped in one specific user's home.
"""
from __future__ import annotations
import logging
from id_configurator.config import RustRepository
from id_configurator.context import Context
from id_configurator.modules.base import Module, register
logger = logging.getLogger(__name__)
_CARGO_CONFIG = ".cargo/config.toml"
def _write_cargo_config(context: Context, repositories: list[RustRepository]) -> None:
primary = repositories[0]
lines = [
"[source.crates-io]",
f'replace-with = "{primary.name}"',
"",
f"[source.{primary.name}]",
f'registry = "sparse+{primary.url}"',
]
for repository in repositories[1:]:
lines += [
"",
f"[registries.{repository.name}]",
f'index = "sparse+{repository.url}"',
]
context.write_file(_CARGO_CONFIG, ("\n".join(lines) + "\n").encode("utf-8"))
logger.info("configured cargo to use %s", primary.url)
@register
class RustModule(Module):
name = "rust"
def is_relevant(self, context: Context) -> bool:
return bool(context.config.configurations.rust.repositories)
def apply(self, context: Context) -> None:
_write_cargo_config(context, context.config.configurations.rust.repositories)
+126
View File
@@ -0,0 +1,126 @@
"""Configure YUM/DNF repositories and trust their GPG keys.
DNF is a drop-in successor to YUM and reads the exact same `.repo` files
under `/etc/yum.repos.d/`, so a single module covers both.
If `<src>/etc/yum.repos.d` exists, every `.repo` file already there is
copied into `<dst>` with all of its sections disabled (`enabled=0`), before
the repositories declared in the configuration are written as new, enabled
`.repo` files (so a configured repository reusing an existing file name
ends up enabled, taking precedence over the disabled copy). Each `.repo`
file is written referencing its `gpg-url` (used verbatim as `gpgkey=`,
exactly as YUM/DNF expect: `https://`, or a `file://` path already present
on the target system). Separately, each GPG key declared in
`configurations.yum.keys` is validated with `gpg` (catching a malformed key
early) and written to `/etc/pki/rpm-gpg/RPM-GPG-KEY-<name>`, the
conventional location such keys live at on RPM-based systems. `gpg` is only
used here to validate the key; the actual trust decision is left to
YUM/DNF, which import it from that file the first time the repository is
used (as `rpm --import` would).
"""
from __future__ import annotations
import configparser
import logging
import shutil
import subprocess
import tempfile
from io import StringIO
from id_configurator.config import NamedContent, YumRepository
from id_configurator.context import Context
from id_configurator.modules.base import Module, register
logger = logging.getLogger(__name__)
_REPOS_DIR = "etc/yum.repos.d"
_RPM_GPG_DIR = "etc/pki/rpm-gpg"
def _disable_repo_content(content: str) -> str:
parser = configparser.ConfigParser(interpolation=None)
parser.optionxform = str
parser.read_string(content)
for section in parser.sections():
parser[section]["enabled"] = "0"
buffer = StringIO()
parser.write(buffer, space_around_delimiters=False)
return buffer.getvalue()
def _disable_existing_repos(context: Context) -> None:
for entry in context.iter_src_dir(_REPOS_DIR):
if entry.suffix != ".repo":
continue
disabled = _disable_repo_content(entry.read_text(encoding="utf-8"))
context.write_file(f"{_REPOS_DIR}/{entry.name}", disabled.encode("utf-8"))
logger.info("disabled existing yum/dnf repository file %s", entry.name)
def _write_repo_file(context: Context, repository: YumRepository) -> None:
lines = [
f"[{repository.name}]",
f"name={repository.name}",
f"baseurl={repository.index_url}",
"enabled=1",
]
if repository.gpg_url:
lines.append("gpgcheck=1")
lines.append(f"gpgkey={repository.gpg_url}")
else:
lines.append("gpgcheck=0")
context.write_file(
f"{_REPOS_DIR}/{repository.name}.repo",
("\n".join(lines) + "\n").encode("utf-8"),
)
logger.info("added yum/dnf repository %s", repository.name)
def _validate_gpg_key(name: str, content: bytes) -> None:
with tempfile.TemporaryDirectory() as gnupg_home:
path = shutil.which("gpg")
if path is not None:
result = subprocess.run(
[path, "--homedir", gnupg_home, "--batch", "--import"],
input=content,
capture_output=True,
check=False,
)
else:
raise RuntimeError(
"gpg is not available; the GNUPG package must be installed in the environment running id-configurator"
)
if result.returncode != 0:
raise RuntimeError(
f"invalid GPG key {name}: {result.stderr.decode('utf-8', 'replace')}"
)
def _import_key(context: Context, key: NamedContent) -> None:
_validate_gpg_key(key.name, key.content.encode("utf-8"))
context.write_file(
f"{_RPM_GPG_DIR}/RPM-GPG-KEY-{key.name}", key.content.encode("utf-8")
)
logger.info("imported yum/dnf GPG key %s", key.name)
@register
class YumModule(Module):
name = "yum"
def is_relevant(self, context: Context) -> bool:
yum = context.config.configurations.yum
existing_repo_files = (
entry
for entry in context.iter_src_dir(_REPOS_DIR)
if entry.suffix == ".repo"
)
return bool(yum.repositories) or bool(yum.keys) or any(existing_repo_files)
def apply(self, context: Context) -> None:
_disable_existing_repos(context)
for repository in context.config.configurations.yum.repositories:
_write_repo_file(context, repository)
for key in context.config.configurations.yum.keys:
_import_key(context, key)
View File
+48
View File
@@ -0,0 +1,48 @@
import logging
import typer
from id_tools.build import app as build_app
from id_tools.test import app as test_app
SUCCESS = 25
class ColorFormatter(logging.Formatter):
"""Logging formatter that adds ANSI color codes based on log level."""
RESET = "\033[0m"
COLORS = {
logging.DEBUG: "\033[90m", # gris
logging.INFO: "\033[34m", # bleu
SUCCESS: "\033[32m", # vert
logging.WARNING: "\033[33m", # orange
logging.ERROR: "\033[31m", # rouge
logging.CRITICAL: "\033[1;31m", # rouge gras
}
def format(self, record):
color = self.COLORS.get(record.levelno, self.RESET)
message = super().format(record)
return f"{color}{message}{self.RESET}"
def setup_logging():
handler = logging.StreamHandler()
handler.setFormatter(ColorFormatter("%(message)s"))
logging.root.addHandler(handler)
logging.root.setLevel(logging.INFO)
app = typer.Typer()
@app.callback()
def main():
setup_logging()
app.add_typer(build_app, name="build")
app.add_typer(test_app, name="test")
if __name__ == "__main__":
app()
+173
View File
@@ -0,0 +1,173 @@
import logging
import subprocess
import tempfile
import typer
from id_tools.config import SourceConfig
app = typer.Typer(help="Build utilities for JS/CSS files and emails templates")
logger = logging.getLogger(__name__)
# noinspection SpellCheckingInspection
DOCKER_FILE_BUILDER = r"""# This file is part of Interdiode
# Copyright (c) {build_year} InterDiode <contact@mail.interdiode.fr>
# All Rights Reserved
########################################################################################################################
# This file is generated by id_tools/build.py. Do not edit it manually.
########################################################################################################################
FROM dhi.io/python:{python_version}-{docker_version} AS interdiode-base
########################################################################################################################
# generate the package and download required some tools in a first image
########################################################################################################################
FROM interdiode-base AS interdiode-builder
ENV{docker_env}
USER root
RUN mkdir -p /src /usr/local/interdiode
RUN chown -R {docker_user}:{docker_group} /src /usr/local/interdiode
COPY ./pyproject.toml /src/pyproject.toml
COPY ./README.md /src/README.md
COPY ./LICENSE /src/LICENSE
COPY ./id_configurator /src/id_configurator
COPY ./tests /src/tests
RUN chown -R {docker_user}:{docker_group} /src
USER {docker_user}
RUN python3 -m pip install --user pip uv --upgrade
WORKDIR /src
RUN python3 -m uv build
"""
# noinspection SpellCheckingInspection
DOCKER_FILE_INSTALL = r"""
########################################################################################################################
# install the package without the build dependencies
########################################################################################################################
FROM interdiode-base AS interdiode-install
ENV{docker_env}
COPY --from=interdiode-builder /src/dist /tmp/dist
RUN apt-get update
RUN DEBIAN_FRONTEND=noninteractive apt-get -o Dpkg::Options::="--force-confnew" --allow-downgrades --allow-remove-essential --allow-unauthenticated -fuy dist-upgrade
RUN DEBIAN_FRONTEND=noninteractive apt-get install --yes -qq --no-install-recommends build-essential
RUN mkdir -p /src/build /chroot/usr/local/bin /usr/local/interdiode \
&& chown -R {docker_user}:{docker_group} /usr/local/interdiode /tmp/dist
USER {docker_user}
RUN /bin/sh -c "python3 -m pip install --user /tmp/dist/*.whl"
"""
# noinspection SpellCheckingInspection
DOCKER_FILE_RUNTIME = r"""########################################################################################################################
# copy the installed packages
########################################################################################################################
FROM interdiode-base AS interdiode-runtime
ENV{docker_env}
LABEL org.opencontainers.image.authors="contact@mail.interdiode.fr"
LABEL org.opencontainers.image.created="{build_date}"
LABEL org.opencontainers.image.description="Air gap made simple!"
LABEL org.opencontainers.image.documentation="{documentation_url}"
LABEL org.opencontainers.image.licenses="LicenseRef-InterDiode-Commercial-1.0"
LABEL org.opencontainers.image.revision="{vcs_revision}"
LABEL org.opencontainers.image.title="InterDiode Configurator"
LABEL org.opencontainers.image.url="{interdiode_url}"
LABEL org.opencontainers.image.vendor="InterDiode"
LABEL org.opencontainers.image.version="{id_configurator_version}"
LABEL description="Air gap made simple!"
COPY --from=interdiode-install --chown=0:0 /chroot/usr/local/bin /usr/local/bin
COPY --from=interdiode-install /usr/local/interdiode /usr/local/interdiode
RUN DEBIAN_FRONTEND=noninteractive mkdir -p /usr/local/interdiode \
&& chown -R {docker_user}:{docker_group} /usr/local/interdiode
COPY ./LICENSE /chroot/licenses/LICENSE
COPY ./LICENSE /chroot/usr/share/doc/interdiode/LICENSE
WORKDIR /
"""
def build_docker_image(
config: SourceConfig,
registry: str | None = None,
platform: str = "linux/amd64",
push: bool = False,
) -> bool:
"""Build the Docker image."""
if push and not registry:
raise ValueError("--push requires --registry to be set.")
version = config.get_project_version()
name = config.get_project_name()
values = config.get_parameters()
package_build = DOCKER_FILE_BUILDER.format(**values)
v_tag = f"{name}:{version}"
l_tag = f"{name}:latest"
image_install = DOCKER_FILE_INSTALL.format(**values)
image_build = DOCKER_FILE_RUNTIME.format(**values)
with (
tempfile.NamedTemporaryFile(
mode="w", delete=True, prefix="Dockerfile.", suffix=".package"
) as fd,
):
fd.write(package_build)
fd.write(image_install)
fd.write(image_build)
fd.flush()
# noinspection SpellCheckingInspection
cmd = [
"docker",
"buildx",
"build",
"--no-cache",
"--attest",
"type=provenance,mode=max,version=v1",
"--attest",
"type=sbom",
"--platform",
platform,
"-f",
fd.name,
]
if registry:
cmd += ["-t", f"{registry}/{v_tag}", "-t", f"{registry}/{l_tag}"]
else:
cmd += ["-t", f"{registry}/{v_tag}", "-t", f"{registry}/{l_tag}"]
if push and registry:
cmd.insert(3, "--push")
cmd.append(".")
try:
typer.echo(f"Building Docker image {v_tag} for platform {platform}...")
typer.echo(" ".join(cmd))
subprocess.check_call(cmd, cwd=config.source_dir)
except subprocess.CalledProcessError:
return False
return True
# noinspection SpellCheckingInspection
@app.command(name="docker", help="Build the Docker image.")
def docker_image(
registry: str = typer.Option(
"interdiode", help="The registry to publish the Docker image to."
),
platform: str = typer.Option(
"linux/amd64", help="Platform(s) of the Docker image."
),
docker_version: str = typer.Option(
"debian13-dev", help="Version of the base Docker image."
),
push: bool = typer.Option(
False, "--push", help="Push the built image(s) to the registry."
),
docker_login: str = typer.Option(
"", envvar="DOCKER_HUB_LOGIN", help="Nom d'utilisateur Docker"
),
docker_token: str = typer.Option(
"", envvar="DOCKER_HUB_TOKEN", help="Jeton d'accès Docker"
),
):
config = SourceConfig(docker_version)
if docker_login and docker_token:
cmd = ["docker", "login", "-u", docker_login, "--password-stdin"]
subprocess.run(
cmd, input=docker_token, text=True, check=True, cwd=config.source_dir
)
success = build_docker_image(
config, registry=registry, platform=platform, push=push
)
if not success:
raise typer.Exit(code=1)
+76
View File
@@ -0,0 +1,76 @@
import datetime
import pathlib
import re
import subprocess
import tomllib
class SourceConfig:
site_url = "https://interdiode.fr/"
documentation_url = "https://interdiode.fr/static/latest/index.html"
cosign_key_url = "https://interdiode.fr/static/cosign.interdiode.fr.pub"
docker_user = "nonroot"
docker_group = "nonroot"
# noinspection SpellCheckingInspection
docker_env = """ PATH=/usr/local/bin/:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/interdiode/bin \
PIP_DISABLE_PIP_VERSION_CHECK=on \
PIP_NO_CACHE_DIR=on \
PYTHONUSERBASE=/usr/local/interdiode \
PYTHONFAULTHANDLER=1 \
PYTHONHASHSEED=random \
PYTHONUNBUFFERED=1
"""
# noinspection SpellCheckingInspection
def __init__(self, docker_version: str = "debian13-dev"):
self.source_dir = pathlib.Path(__file__).parent.parent
with open(self.source_dir / "pyproject.toml", "rb") as fd:
self.raw_metadata = tomllib.load(fd)
self.docker_version = docker_version
def get_python_versions(self) -> list[str]:
"""Return the list of Python versions supported by the project."""
python_versions = []
for classifier in self.raw_metadata["project"]["classifiers"]:
if matcher := re.match(
r"Programming Language :: Python :: (\d\.\d+)", classifier
):
python_versions.append(matcher.group(1))
return python_versions
def get_project_name(self) -> str:
"""Get the name of InterDiode from the pyproject.toml file."""
return self.raw_metadata["project"]["name"]
def get_project_version(self) -> str:
"""Get the version of InterDiode from the pyproject.toml file."""
return self.raw_metadata["project"]["version"]
def get_git_project_revision(self) -> str:
"""Get the git revision of the project."""
out = subprocess.check_output(["git", "rev-parse", "HEAD"], cwd=self.source_dir) # nosec B607
return out.decode().strip()
def get_parameters(self, python_version: str | None = None) -> dict[str, str]:
python_version = python_version or self.get_python_versions()[-1]
return {
"build_date": datetime.datetime.now(tz=datetime.UTC).isoformat(),
"build_year": str(datetime.datetime.now(tz=datetime.UTC).year),
"docker_env": self.docker_env,
"docker_user": self.docker_user,
"docker_group": self.docker_group,
"docker_version": self.docker_version,
"documentation_url": self.documentation_url,
"interdiode_url": self.site_url,
"id_configurator_version": self.get_project_version(),
"python_version": python_version,
"vcs_revision": self.get_git_project_revision(),
}
def get_project_author_and_emails(self) -> list[dict[str, str]]:
"""Get the authors of InterDiode from the pyproject.toml file."""
return self.raw_metadata["project"]["authors"]
def get_project_authors(self) -> list[str]:
"""Get the authors of InterDiode from the pyproject.toml file."""
return [author["name"] for author in self.raw_metadata["project"]["authors"]]
+129
View File
@@ -0,0 +1,129 @@
import json
import logging
import os
import subprocess
import tempfile
import typer
from id_tools.config import SourceConfig
app = typer.Typer(
help="Publish the documentation and release announcements to external destinations."
)
logger = logging.getLogger(__name__)
@app.command(name="sign-docker", help="Signe l'image Docker sur le registre")
def sign_docker_image(
registry: str = typer.Option(
"interdiode", help="Registre cible pour l'image Docker"
),
docker_login: str = typer.Option(
"", envvar="DOCKER_HUB_LOGIN", help="Nom d'utilisateur Docker"
),
docker_token: str = typer.Option(
"", envvar="DOCKER_HUB_TOKEN", help="Jeton d'accès Docker"
),
cosign_password: str = typer.Option(
"", envvar="COSIGN_PASSWORD", help="Mot de passe de la clé privée Cosign"
),
cosign_private_key: str = typer.Option(
"", envvar="COSIGN_PRIVATE_KEY", help="Contenu de la clé privée Cosign"
),
):
config = SourceConfig()
version = config.get_project_version()
image = config.get_project_name()
v_tag = f"{image}:{version}"
l_tag = f"{image}:latest"
destinations = [f"{registry}/{v_tag}", f"{registry}/{l_tag}"]
if docker_login and docker_token:
cmd = ["docker", "login", "-u", docker_login, "--password-stdin"]
subprocess.run(
cmd, input=docker_token, text=True, check=True, cwd=config.source_dir
)
digest_references = {
get_reference_from_tag(destination) for destination in destinations
}
if cosign_password and cosign_private_key:
with (
tempfile.NamedTemporaryFile(mode="w", suffix=".key", delete=True) as key_fd,
tempfile.TemporaryDirectory() as tmp_dir,
):
key_fd.write(cosign_private_key)
key_fd.flush()
cosign_env = {**os.environ, "COSIGN_PASSWORD": cosign_password}
trivy_report = os.path.join(tmp_dir, "trivy_report.json")
for digest_reference in digest_references:
# noinspection SpellCheckingInspection
cmd = [
"trivy",
"image",
"--format",
"json",
"--output",
trivy_report,
digest_reference,
]
subprocess.run(cmd, text=True, check=True, cwd=config.source_dir)
cmd = [
"cosign",
"attest",
"--key",
key_fd.name,
"--type",
"vuln",
"--predicate",
trivy_report,
digest_reference,
]
subprocess.run(cmd, text=True, check=True, cwd=config.source_dir)
cmd = [
"cosign",
"sign",
"--yes",
"--recursive",
"--key",
key_fd.name,
digest_reference,
]
subprocess.check_call(cmd, env=cosign_env)
cmd = [
"cosign",
"verify-attestation",
"--key",
config.cosign_key_url,
"--type",
"vuln",
digest_reference,
]
subprocess.check_call(cmd, env=cosign_env)
cmd = [
"cosign",
"verify",
"--key",
config.cosign_key_url,
digest_reference,
]
subprocess.check_call(cmd, env=cosign_env)
typer.echo(f"✓ Image signée : {registry}/{v_tag} et {registry}/{l_tag}")
def get_reference_from_tag(destination: str) -> str:
# noinspection SpellCheckingInspection
cmd = [
"docker",
"buildx",
"imagetools",
"inspect",
destination,
"--format",
"{{json .Manifest}}",
]
result = subprocess.run(cmd, check=True, capture_output=True, text=True)
manifest = json.loads(result.stdout)
image_name = destination.rsplit(":", 1)[0]
reference = f"{image_name}@{manifest['digest']}"
return reference
+66
View File
@@ -0,0 +1,66 @@
"""Utilities for the Docker integration tests under tests/."""
from __future__ import annotations
import os
import pathlib
from typing import Annotated
import jinja2
import typer
app = typer.Typer(help="Utilities for the tests/ Docker integration tests")
TESTS_DIR = pathlib.Path(__file__).resolve().parent.parent / "tests"
def _env(name: str) -> str:
"""Jinja2 filter: `'NAME' | env` -> the environment variable NAME."""
return os.environ[name]
def _file(relative: str) -> str:
"""Jinja2 filter: `'relative/path' | file` -> content of tests/relative/path."""
path = (TESTS_DIR / relative).resolve()
if path != TESTS_DIR and TESTS_DIR not in path.parents:
raise ValueError(f"refusing to read a file outside of {TESTS_DIR}: {relative}")
return path.read_text(encoding="utf-8")
def render_template(text: str) -> str:
"""Render a tests/configs/*.yaml.j2 fixture as a Jinja2 template.
Custom filters: `| env` reads an environment variable, `| file` reads a
file relative to tests/. Combine with Jinja2's built-in `indent` filter
to fit multi-line content (e.g. a PEM certificate) into a YAML
block-literal field, e.g.:
content: |
{{ 'certs/test-ca.crt.pem' | file | indent(6, first=true) }}
"""
# Autoescaping is for HTML output; these templates render YAML, so escaping
# would corrupt them (e.g. turning "&" into "&amp;" inside a PEM/URL).
environment = jinja2.Environment(undefined=jinja2.StrictUndefined) # nosec B701
environment.filters["env"] = _env
environment.filters["file"] = _file
return environment.from_string(text).render()
@app.command(
name="render-config",
help="Render a tests/configs/*.yaml.j2 fixture into a ready-to-use configuration.",
)
def render_config(
template: Annotated[pathlib.Path, typer.Argument(exists=True, dir_okay=False)],
output: pathlib.Path,
) -> None:
"""Renders `template` as a Jinja2 template (see `render_template`) and
writes the result to `output`, creating its parent directories as
needed."""
content = render_template(template.read_text(encoding="utf-8"))
output.parent.mkdir(parents=True, exist_ok=True)
output.write_text(content, encoding="utf-8")
if __name__ == "__main__":
app()
+635
View File
@@ -0,0 +1,635 @@
#!/usr/bin/python
"""Ansible module wrapping id-configurator.
Applies an id-configurator configuration directly to this host's live root
filesystem, exactly like the `id_configurator` role does, but as a native
Ansible module: it imports id-configurator's own Python API in-process
(instead of shelling out to the CLI through a temporary file), which also
lets it honor Ansible's check mode - reporting which modules would be
applied without writing anything.
This module is a thin, additive wrapper: it does not change anything about
id-configurator's own behavior (the `id_configurator` Python package,
including its module registry and every individual module, is used exactly
as-is - the same `Configuration`, `Context` and module registry the CLI
itself uses).
"""
from __future__ import annotations
from pathlib import Path
from ansible.module_utils.basic import AnsibleModule
DOCUMENTATION = r"""
module: id_configurator
short_description: Apply an id-configurator configuration to this host.
description:
- Assembles the given options into an id-configurator configuration and
applies it directly to this host's live root filesystem, by importing
id-configurator's own Python API in-process (no CLI subprocess, no
temporary file).
- The option names and structure mirror id_configurator.config.Configuration
and id_configurator.config.ConfigurationsBlock one for one. The
`id_configurator` Python package must already be installed on the
target host, using the same Python interpreter Ansible uses there.
- Supports check mode - reports which modules would be applied without
writing anything.
author:
- InterDiode
options:
uuid:
type: str
required: true
description: Configuration.uuid - a UUID identifying this configuration.
organisation:
type: str
required: true
description: Configuration.organisation.
description:
type: str
default: ""
description: Configuration.description.
certificates:
type: list
elements: dict
default: []
description: ConfigurationsBlock.certificates - root certificates to install.
suboptions:
name:
type: str
required: true
description: A short, unique name for this entry.
content:
type: str
required: true
description: PEM-encoded certificate content.
apt:
type: dict
default: {}
description: ConfigurationsBlock.apt.
suboptions:
sources:
type: list
elements: dict
default: []
description: DEB822 APT sources (AptSource).
suboptions:
name:
type: str
required: true
description: A short, unique name for this source.
types:
type: list
elements: str
default: [deb]
description: DEB822 "Types" field.
uris:
type: list
elements: str
required: true
description: DEB822 "URIs" field.
suites:
type: list
elements: str
required: true
description: DEB822 "Suites" field.
components:
type: list
elements: str
default: []
description: DEB822 "Components" field.
architectures:
type: list
elements: str
default: []
description: DEB822 "Architectures" field.
languages:
type: list
elements: str
default: []
description: DEB822 "Languages" field.
targets:
type: list
elements: str
default: []
description: DEB822 "Targets" field.
enabled:
type: bool
default: true
description: DEB822 "Enabled" field.
trusted:
type: bool
description: DEB822 "Trusted" field.
signed_by:
type: str
description: DEB822 "Signed-By" field.
pdiffs:
type: bool
description: DEB822 "PDiffs" field.
by_hash:
type: bool
description: DEB822 "By-Hash" field.
allow_insecure:
type: bool
description: DEB822 "Allow-Insecure" field.
allow_weak:
type: bool
description: DEB822 "Allow-Weak" field.
allow_downgrade_to_insecure:
type: bool
description: DEB822 "Allow-Downgrade-To-Insecure" field.
check_valid_until:
type: bool
description: DEB822 "Check-Valid-Until" field.
valid_until_min:
type: str
description: DEB822 "Valid-Until-Min" field.
valid_until_max:
type: str
description: DEB822 "Valid-Until-Max" field.
check_date:
type: bool
description: DEB822 "Check-Date" field.
date_max_future:
type: str
description: DEB822 "Date-Max-Future" field.
description:
type: str
description: DEB822 "Description" field.
keys:
type: list
elements: dict
default: []
description: GPG keys trusted for the sources above (NamedContent).
suboptions:
name:
type: str
required: true
description: A short, unique name for this entry.
content:
type: str
required: true
description: PEM-encoded/ASCII-armored key content.
apk:
type: dict
default: {}
description: ConfigurationsBlock.apk.
suboptions:
repositories:
type: list
elements: dict
default: []
description: Alpine APK repositories (ApkRepository).
suboptions:
url:
type: str
required: true
description: Repository URL, e.g. ending in /main or /community.
key:
type: str
description: The signing key's name, if any (see NamedContent).
yum:
type: dict
default: {}
description: ConfigurationsBlock.yum.
suboptions:
repositories:
type: list
elements: dict
default: []
description: YUM/DNF repositories (YumRepository).
suboptions:
name:
type: str
required: true
description: A short, unique name for this repository.
index_url:
type: str
required: true
description: The repository's baseurl.
gpg_url:
type: str
description: The repository's gpgkey URL.
keys:
type: list
elements: dict
default: []
description: GPG keys imported for the repositories above (NamedContent).
suboptions:
name:
type: str
required: true
description: A short, unique name for this entry.
content:
type: str
required: true
description: PEM-encoded/ASCII-armored key content.
python:
type: dict
default: {}
description: ConfigurationsBlock.python.
suboptions:
repositories:
type: list
elements: dict
default: []
description: Python package indexes (PythonRepository).
suboptions:
name:
type: str
required: true
description: A short, unique name for this index.
index_url:
type: str
required: true
description: The package index's URL.
java:
type: dict
default: {}
description: ConfigurationsBlock.java.
suboptions:
repositories:
type: list
elements: dict
default: []
description: Maven-compatible repositories (JavaRepository).
suboptions:
name:
type: str
required: true
description: A short, unique name for this repository.
url:
type: str
required: true
description: The repository's URL.
php:
type: dict
default: {}
description: ConfigurationsBlock.php.
suboptions:
repositories:
type: list
elements: dict
default: []
description: Composer repositories (PhpRepository).
suboptions:
name:
type: str
required: true
description: A short, unique name for this repository.
url:
type: str
required: true
description: The repository's URL.
npm:
type: dict
default: {}
description: ConfigurationsBlock.npm.
suboptions:
repositories:
type: list
elements: dict
default: []
description: npm registries (NpmRepository).
suboptions:
name:
type: str
required: true
description: A short, unique name for this repository.
url:
type: str
required: true
description: The repository's URL.
rust:
type: dict
default: {}
description: ConfigurationsBlock.rust.
suboptions:
repositories:
type: list
elements: dict
default: []
description: Cargo registries (RustRepository).
suboptions:
name:
type: str
required: true
description: A short, unique name for this repository.
url:
type: str
required: true
description: The repository's URL.
go:
type: dict
default: {}
description: ConfigurationsBlock.go.
suboptions:
repositories:
type: list
elements: dict
default: []
description: Go module proxies (GoRepository).
suboptions:
name:
type: str
required: true
description: A short, unique name for this repository.
url:
type: str
required: true
description: The repository's URL.
ruby:
type: dict
default: {}
description: ConfigurationsBlock.ruby.
suboptions:
repositories:
type: list
elements: dict
default: []
description: RubyGems sources (RubyRepository).
suboptions:
name:
type: str
required: true
description: A short, unique name for this repository.
url:
type: str
required: true
description: The repository's URL.
nuget:
type: dict
default: {}
description: ConfigurationsBlock.nuget.
suboptions:
repositories:
type: list
elements: dict
default: []
description: NuGet package sources (NuGetRepository).
suboptions:
name:
type: str
required: true
description: A short, unique name for this repository.
url:
type: str
required: true
description: The repository's URL.
"""
EXAMPLES = r"""
- name: Apply an id-configurator configuration to this host
id_configurator:
uuid: 12345678-1234-5678-1234-567812345678
organisation: InterDiode
certificates:
- name: Company Root CA
content: "{{ lookup('file', 'files/company-root-ca.pem') }}"
apt:
sources:
- name: internal
uris: ["https://apt.internal.example/"]
suites: ["stable"]
"""
RETURN = r"""
applied_modules:
description: Names of the id-configurator modules that were (or, in check
mode, would be) applied, in registration order.
returned: always
type: list
elements: str
sample: [certificates, apt]
"""
# Every key here must match one of ConfigurationsBlock's own field names
# (id_configurator/config.py) - see the module docstring above.
_CONFIGURATIONS_BLOCK_FIELDS = (
"certificates",
"apt",
"apk",
"yum",
"python",
"java",
"php",
"npm",
"rust",
"go",
"ruby",
"nuget",
)
_NAMED_CONTENT_OPTIONS = {
"name": {"type": "str", "required": True},
"content": {"type": "str", "required": True},
}
_NAME_URL_OPTIONS = {
"name": {"type": "str", "required": True},
"url": {"type": "str", "required": True},
}
def _name_url_repository_config() -> dict:
"""Argument spec for a `{repositories: [{name, url}]}` config block,
shared by java/php/npm/rust/go/ruby/nuget."""
return {
"type": "dict",
"default": {},
"options": {
"repositories": {
"type": "list",
"elements": "dict",
"default": [],
"options": dict(_NAME_URL_OPTIONS),
},
},
}
ARGUMENT_SPEC = {
"uuid": {"type": "str", "required": True},
"organisation": {"type": "str", "required": True},
"description": {"type": "str", "default": ""},
"certificates": {
"type": "list",
"elements": "dict",
"default": [],
"options": dict(_NAMED_CONTENT_OPTIONS),
},
"apt": {
"type": "dict",
"default": {},
"options": {
"sources": {
"type": "list",
"elements": "dict",
"default": [],
"options": {
"name": {"type": "str", "required": True},
"types": {"type": "list", "elements": "str", "default": ["deb"]},
"uris": {"type": "list", "elements": "str", "required": True},
"suites": {"type": "list", "elements": "str", "required": True},
"components": {"type": "list", "elements": "str", "default": []},
"architectures": {
"type": "list",
"elements": "str",
"default": [],
},
"languages": {"type": "list", "elements": "str", "default": []},
"targets": {"type": "list", "elements": "str", "default": []},
"enabled": {"type": "bool", "default": True},
"trusted": {"type": "bool"},
"signed_by": {"type": "str"},
"pdiffs": {"type": "bool"},
"by_hash": {"type": "bool"},
"allow_insecure": {"type": "bool"},
"allow_weak": {"type": "bool"},
"allow_downgrade_to_insecure": {"type": "bool"},
"check_valid_until": {"type": "bool"},
"valid_until_min": {"type": "str"},
"valid_until_max": {"type": "str"},
"check_date": {"type": "bool"},
"date_max_future": {"type": "str"},
"description": {"type": "str"},
},
},
"keys": {
"type": "list",
"elements": "dict",
"default": [],
"options": dict(_NAMED_CONTENT_OPTIONS),
},
},
},
"apk": {
"type": "dict",
"default": {},
"options": {
"repositories": {
"type": "list",
"elements": "dict",
"default": [],
"options": {
"url": {"type": "str", "required": True},
"key": {"type": "str"},
},
},
},
},
"yum": {
"type": "dict",
"default": {},
"options": {
"repositories": {
"type": "list",
"elements": "dict",
"default": [],
"options": {
"name": {"type": "str", "required": True},
"index_url": {"type": "str", "required": True},
"gpg_url": {"type": "str"},
},
},
"keys": {
"type": "list",
"elements": "dict",
"default": [],
"options": dict(_NAMED_CONTENT_OPTIONS),
},
},
},
"python": {
"type": "dict",
"default": {},
"options": {
"repositories": {
"type": "list",
"elements": "dict",
"default": [],
"options": {
"name": {"type": "str", "required": True},
"index_url": {"type": "str", "required": True},
},
},
},
},
"java": _name_url_repository_config(),
"php": _name_url_repository_config(),
"npm": _name_url_repository_config(),
"rust": _name_url_repository_config(),
"go": _name_url_repository_config(),
"ruby": _name_url_repository_config(),
"nuget": _name_url_repository_config(),
}
try:
import id_configurator.modules # noqa: F401 (registers the built-in modules)
from id_configurator.config import Configuration
from id_configurator.context import Context
from id_configurator.modules.base import all_modules
HAS_ID_CONFIGURATOR = True
ID_CONFIGURATOR_IMPORT_ERROR = None
except ImportError as exc:
HAS_ID_CONFIGURATOR = False
ID_CONFIGURATOR_IMPORT_ERROR = exc
def _build_configuration_data(params: dict) -> dict:
return {
"uuid": params["uuid"],
"organisation": params["organisation"],
"description": params["description"],
"configurations": {
field: params[field] for field in _CONFIGURATIONS_BLOCK_FIELDS
},
}
def _apply_configuration(configuration, check_mode: bool) -> list[str]:
"""Apply `configuration` to this host's live root filesystem, returning
the names of every module that was (or, in check mode, would be)
applied. Uses id-configurator's own module registry and `Context`
unchanged - this is a thin caller, not a reimplementation."""
root = Path("/")
context = Context(config=configuration, dst=root, src=root)
applied_modules = []
for module_cls in all_modules():
config_module = module_cls()
if config_module.is_relevant(context):
applied_modules.append(config_module.name)
if not check_mode:
config_module.apply(context)
return applied_modules
def main() -> None:
module = AnsibleModule(argument_spec=ARGUMENT_SPEC, supports_check_mode=True)
if not HAS_ID_CONFIGURATOR:
module.fail_json(
msg=(
"id_configurator is not importable with this host's Python "
f"interpreter: {ID_CONFIGURATOR_IMPORT_ERROR}"
)
)
return
try:
configuration = Configuration.model_validate(
_build_configuration_data(module.params)
)
except Exception as exc: # pydantic.ValidationError, kept generic on purpose
module.fail_json(msg=f"invalid id-configurator configuration: {exc}")
return
applied_modules = _apply_configuration(configuration, module.check_mode)
module.exit_json(changed=bool(applied_modules), applied_modules=applied_modules)
if __name__ == "__main__":
main()
+47 -3
View File
@@ -2,6 +2,19 @@
build-backend = "uv_build"
requires = ["uv_build>=0.12.3,<0.13.0"]
[dependency-groups]
dev = [
"jinja2>=3.1.6",
"tox>=4.60.1",
"tox-docker>=5.0.0",
"tox-uv>=1.36.0",
"typer>=0.27.1"
]
test = [
"pytest>=9.1.1",
"pytest-cov>=7.1.0"
]
[project]
authors = [
{name = "InterDiode", email = "contact@mail.interdiode.fr"}
@@ -17,7 +30,13 @@ classifiers = [
'Programming Language :: Python :: 3.13',
'Programming Language :: Python :: 3.14'
]
dependencies = []
dependencies = [
"cryptography>=50.0.1",
"pydantic>=2.13.4",
"pyjks>=20.0.0",
"pyyaml>=6.0.3",
"typer>=0.27.1"
]
description = "Configure a device in an air-gapped environment."
license = {text = "Proprietary"}
maintainers = [
@@ -25,11 +44,36 @@ maintainers = [
]
name = "id-configurator"
readme = "README.md"
requires-python = ">=3.14"
version = "0.1.0"
requires-python = ">=3.13"
version = "0.2.0"
[project.scripts]
id-configurator = "id_configurator:main"
[tool.coverage.paths]
source = [
"id_configurator",
"*/site-packages/id_configurator"
]
[tool.coverage.report]
exclude_lines = [
"pragma: no cover",
"if __name__ == .__main__.:"
]
fail_under = 100
show_missing = true
[tool.coverage.run]
branch = true
source = ["id_configurator"]
[tool.pytest.ini_options]
addopts = "--cov=id_configurator --cov-report=term-missing"
testpaths = ["tests"]
[tool.uv]
default-groups = ["dev", "test"]
[tool.uv.build-backend]
module-root = ""
+21
View File
@@ -0,0 +1,21 @@
-----BEGIN CERTIFICATE-----
MIIDdTCCAl2gAwIBAgIUKq4URHQwyg7IzJ3ViAGGlM9pGCwwDQYJKoZIhvcNAQEL
BQAwQjEeMBwGA1UECgwVaWQtY29uZmlndXJhdG9yLXRlc3RzMSAwHgYDVQQDDBdp
ZC1jb25maWd1cmF0b3IgVGVzdCBDQTAeFw0yNjA5MDUyMTA4MTNaFw00NjA4MzEy
MTA4MTNaMEIxHjAcBgNVBAoMFWlkLWNvbmZpZ3VyYXRvci10ZXN0czEgMB4GA1UE
AwwXaWQtY29uZmlndXJhdG9yIFRlc3QgQ0EwggEiMA0GCSqGSIb3DQEBAQUAA4IB
DwAwggEKAoIBAQD8bMteh3MDnbW6OoFMlhA4Z2QAabVoSFlSE/Si7ut+L/3v1GY4
lixenwLQc8a2koIH9CtmOodzfO7hpro/VUjYJ9J2eTsXLLiOg7OgggG1cKV9/kt7
RF0Q4hkcanGmiffc4xbz4ZedU6lP1vzLCsgU0A8OPt6svcNjGcWbjxss0KjgxGZ7
H4UKdL7Mv4W2Y6Fw1b4eV2M6jErkQK6IPXsHGFh7JkzliNcb0sX1VfbfauS84tBh
f3EHwzLNhbjtEmQhQdqgsg4Cbk/gWur//6aEvlD/X12PAUbrp4DFD+CDuVagLX7I
7TQBXNFGKhozzQuDg7DFg1gIV8SFx2m+s60/AgMBAAGjYzBhMB0GA1UdDgQWBBSb
zJ7DsaxwCeqbfOXGJ3x+u7bjozAfBgNVHSMEGDAWgBSbzJ7DsaxwCeqbfOXGJ3x+
u7bjozAPBgNVHRMBAf8EBTADAQH/MA4GA1UdDwEB/wQEAwIBBjANBgkqhkiG9w0B
AQsFAAOCAQEAEV1+dG3agh0S/AR8jC8M33HXFyvneCCRtfb5q/loJ//VoWPbD3Gs
G+2SR0MVN+KKOMBvCBh3KrNSgFF9iFmBjNCJJZ8ycDIszUK3xLjUP2EzN1pNNl8S
D/yHzUCro3547Q+I+Nn5zRciG95EpMvtkIWem4H8luYSBt7w8y9uDUKWzaca/v3z
oGZPZP99w8WVqMKGRbsK94wsTVsWiE+oHzSufwrhvZJfQQ4jU6102IEnRS/eQFXA
+fozgEWuFatZe3Gz0HUJakrzSBwSTV6wmnEHCfu+uwdns1Ttq5FylZ4StJHsmweL
14w1xa3tjC5bQ3e1QWK3R5InHcSZ7tLpGA==
-----END CERTIFICATE-----
+28
View File
@@ -0,0 +1,28 @@
-----BEGIN PRIVATE KEY-----
MIIEvAIBADANBgkqhkiG9w0BAQEFAASCBKYwggSiAgEAAoIBAQD8bMteh3MDnbW6
OoFMlhA4Z2QAabVoSFlSE/Si7ut+L/3v1GY4lixenwLQc8a2koIH9CtmOodzfO7h
pro/VUjYJ9J2eTsXLLiOg7OgggG1cKV9/kt7RF0Q4hkcanGmiffc4xbz4ZedU6lP
1vzLCsgU0A8OPt6svcNjGcWbjxss0KjgxGZ7H4UKdL7Mv4W2Y6Fw1b4eV2M6jErk
QK6IPXsHGFh7JkzliNcb0sX1VfbfauS84tBhf3EHwzLNhbjtEmQhQdqgsg4Cbk/g
Wur//6aEvlD/X12PAUbrp4DFD+CDuVagLX7I7TQBXNFGKhozzQuDg7DFg1gIV8SF
x2m+s60/AgMBAAECggEACGXaMjqRYf+/ia2OmwIBvM6mbg5rlvI+k9N5d745HslG
n9GngPRwuh581SXfk41g6Wt/J3cvsHDp8Enqt6wFBg6fxYns+a1m38mT5/V19xet
+c4ZrBqEeh5Q4DgjsHsLQFGxyXVU5PVhPvhpD/GQ4dhg19l8RlpIg3svrtLeVZRJ
T+jy7b0LTkRwe6D7AjE+1tqZr299JBLdH9dS7DeKA3rP7O6HzSEhrtiFxJBfq0v4
MDASAhXHytiqIErylb+ZiGSeqTYDLOJBhRcyrjuR1gNlFSxOa1+4iBVo5OVGTkYj
ZuT5r18xgJ0EjnMYd5RszWqfjzWGtd0oYyfN599Z+QKBgQD+2YM1JRAHrQORkjPm
Bz3JA6R2lDVbng/q1iJsdNUkxU0iKieLJ9KOiMRF9RU9QWIbuI9bvNxShudyZIiM
jqEUiUbGnRB5u7rtUOaA8ZlWtsxn3HGJ6C1iN8+IYhPzi3BFPRUHo8gUF+PgpmI3
EYXwto6d7uKh1ExBnebt/qnZEwKBgQD9kHrmlKUbrgR7OOLnhVGjB+5AX4fWbWcP
n2JOp19O5ZsQdgtP7eYDsdV7h3UGXbH1u53paXJUKneF53k3FtaG+XR6JPnKGk6Y
18Z5TvyXtdkgDuxPITHp0DmzJ7+1SmwsY2dMj8VAf08U7aCkURli1SpXx5a9tgly
WlURS1aspQKBgFXwW4KMYP91pIJuXvnu7zLd+5PRHr0qbAuw8iLIsyeXMQ+YT4Vp
hGN0U1uT7fmeFstqTU2OPm3F+D5nEHT1k/Qs5qULSgcC+2ITqedJkX/gXMWZXOA4
XzWDxIQlLr0uBEoU8waEgG/mLn4PHYecbSJjLWyspdgUkzJ9YOZYvkXpAoGAPMhL
jcaIJbB7RoaGbfVZcnlONYQw7cff8rDEQhKBGEqV90Y0w/5FKxGQGUKJXPNM25kT
64XpcB+V8zVfq0MAPe7Cax09+6J+wMjIvP+gMUXYPIUXRIiyjFfr3PRkUjL651EU
zBeSszlRXbTw0zLVoQl0UwJ3MhbXAOCCj9i9eDkCgYBf4XDR8Mi0rKHRL7NP8mYK
pKRmH2ABtupcVJn8GgUm8N5xaVsV7z9fW+YcrNQ2jbXogyMgwoyaUNa26mikb9em
i6DZCainr36d9UtO4aCMDOfapX18d68jt5FiEeiYs7r+mGZHrGmMFwxFJKmNdqhr
affX588Yx9EvfIb9tyk9KA==
-----END PRIVATE KEY-----
+35
View File
@@ -0,0 +1,35 @@
uuid: 3d2f6a44-8b19-4c2b-9e7a-1f5c8a2d6b90
organisation: InterDiode
description: Apk test configuration
configurations:
certificates:
- name: Test CA
content: |
{{ 'certs/test-ca.crt.pem' | file | indent(6, first=true) }}
apt:
sources: []
keys: []
apk:
repositories:
- url: https://host.docker.internal:{{ 'APK_REPO_PORT' | env }}/
key: |
{{ 'dockerfiles/repos/040d0af5f97a0a44.rsa.pub' | file | indent(8, first=true) }}
yum:
repositories: []
keys: []
python:
repositories: []
java:
repositories: []
php:
repositories: []
npm:
repositories: []
rust:
repositories: []
go:
repositories: []
ruby:
repositories: []
nuget:
repositories: []
+38
View File
@@ -0,0 +1,38 @@
uuid: 0147050d-d045-4719-b084-e2b1f337290e
organisation: InterDiode
description: Apt test configuration
configurations:
certificates:
- name: Test CA
content: |
{{ 'certs/test-ca.crt.pem' | file | indent(6, first=true) }}
apt:
sources:
- name: demo
types: [deb]
uris:
- https://host.docker.internal:{{ 'APT_REPO_PORT' | env }}/
suites: ["./"]
trusted: true
keys: []
apk:
repositories: []
yum:
repositories: []
keys: []
python:
repositories: []
java:
repositories: []
php:
repositories: []
npm:
repositories: []
rust:
repositories: []
go:
repositories: []
ruby:
repositories: []
nuget:
repositories: []
+32
View File
@@ -0,0 +1,32 @@
uuid: 200f3290-3a3f-47ac-8149-dbe26c82ee34
organisation: InterDiode
description: Certificate test configuration
configurations:
certificates:
- name: Test CA
content: |
{{ 'certs/test-ca.crt.pem' | file | indent(6, first=true) }}
apt:
sources: []
keys: []
apk:
repositories: []
yum:
repositories: []
keys: []
python:
repositories: []
java:
repositories: []
php:
repositories: []
npm:
repositories: []
rust:
repositories: []
go:
repositories: []
ruby:
repositories: []
nuget:
repositories: []
+34
View File
@@ -0,0 +1,34 @@
uuid: 3f1a2b4c-5d6e-4f70-8a91-2b3c4d5e6f70
organisation: InterDiode
description: Go test configuration
configurations:
certificates:
- name: Test CA
content: |
{{ 'certs/test-ca.crt.pem' | file | indent(6, first=true) }}
apt:
sources: []
keys: []
apk:
repositories: []
yum:
repositories: []
keys: []
python:
repositories: []
java:
repositories: []
php:
repositories: []
npm:
repositories: []
rust:
repositories: []
go:
repositories:
- name: demo
url: https://host.docker.internal:{{ 'GO_REPO_PORT' | env }}/
ruby:
repositories: []
nuget:
repositories: []
+47
View File
@@ -0,0 +1,47 @@
uuid: 6a1d4e2c-9f3b-4a5d-8c7e-2b1f9a6d4c3e
organisation: InterDiode
description: Java test configuration
configurations:
certificates:
- name: Test CA
content: |
{{ 'certs/test-ca.crt.pem' | file | indent(6, first=true) }}
apt:
sources: []
keys: []
apk:
repositories: []
yum:
repositories: []
keys: []
python:
repositories: []
java:
repositories:
# First entry: named "central" so Maven's mirrorOf=* (built on
# repositories[0]) redirects everything here without invalidating the
# maven-dependency-plugin already cached under that repository id
# during the Dockerfile's warm-up step. All other tools just see it as
# an ordinary repository, and find the test artifact here.
- name: central
url: https://host.docker.internal:{{ 'JAVA_REPO_PORT' | env }}/
# Second entry: only SBT needs this. With
# -Dsbt.override.build.repos=true (the only way SBT actually consults
# this file for project dependencies, not just its own bootstrap), the
# repositories listed here become the *only* ones available - without a
# real Maven Central mirror among them, SBT can't resolve scala-library
# itself. Maven ignores it (mirrorOf=* already redirects everything to
# the first entry); Gradle/Ivy/Leiningen just see one more unused
# resolver, since the test artifact is already found via "central".
- name: maven-central-fallback
url: https://repo1.maven.org/maven2/
npm:
repositories: []
rust:
repositories: []
go:
repositories: []
ruby:
repositories: []
nuget:
repositories: []
+34
View File
@@ -0,0 +1,34 @@
uuid: f8348fe2-3f69-4eee-9c3d-d5f092c9ff80
organisation: InterDiode
description: Npm test configuration
configurations:
certificates:
- name: Test CA
content: |
{{ 'certs/test-ca.crt.pem' | file | indent(6, first=true) }}
apt:
sources: []
keys: []
apk:
repositories: []
yum:
repositories: []
keys: []
python:
repositories: []
java:
repositories: []
php:
repositories: []
npm:
repositories:
- name: demo
url: https://host.docker.internal:{{ 'NPM_REPO_PORT' | env }}/
rust:
repositories: []
go:
repositories: []
ruby:
repositories: []
nuget:
repositories: []
+34
View File
@@ -0,0 +1,34 @@
uuid: 5b3c4d6e-7f80-4192-8b13-4d5e6f708192
organisation: InterDiode
description: NuGet test configuration
configurations:
certificates:
- name: Test CA
content: |
{{ 'certs/test-ca.crt.pem' | file | indent(6, first=true) }}
apt:
sources: []
keys: []
apk:
repositories: []
yum:
repositories: []
keys: []
python:
repositories: []
java:
repositories: []
php:
repositories: []
npm:
repositories: []
rust:
repositories: []
go:
repositories: []
ruby:
repositories: []
nuget:
repositories:
- name: demo
url: https://host.docker.internal:{{ 'NUGET_REPO_PORT' | env }}/index.json
+34
View File
@@ -0,0 +1,34 @@
uuid: 7135f49c-44c4-4583-8512-2ed5c6ee726b
organisation: InterDiode
description: Php test configuration
configurations:
certificates:
- name: Test CA
content: |
{{ 'certs/test-ca.crt.pem' | file | indent(6, first=true) }}
apt:
sources: []
keys: []
apk:
repositories: []
yum:
repositories: []
keys: []
python:
repositories: []
java:
repositories: []
php:
repositories:
- name: demo
url: https://host.docker.internal:{{ 'PHP_REPO_PORT' | env }}/
npm:
repositories: []
rust:
repositories: []
go:
repositories: []
ruby:
repositories: []
nuget:
repositories: []
+34
View File
@@ -0,0 +1,34 @@
uuid: 8453d139-cbe5-43b0-ab35-4468042b8c7e
organisation: InterDiode
description: Python test configuration
configurations:
certificates:
- name: Test CA
content: |
{{ 'certs/test-ca.crt.pem' | file | indent(6, first=true) }}
apt:
sources: []
keys: []
apk:
repositories: []
yum:
repositories: []
keys: []
python:
repositories:
- name: demo
index-url: https://host.docker.internal:{{ 'PYPI_REPO_PORT' | env }}/simple/
java:
repositories: []
php:
repositories: []
npm:
repositories: []
rust:
repositories: []
go:
repositories: []
ruby:
repositories: []
nuget:
repositories: []
+34
View File
@@ -0,0 +1,34 @@
uuid: 4a2b3c5d-6e7f-4081-9a02-3c4d5e6f7081
organisation: InterDiode
description: Ruby test configuration
configurations:
certificates:
- name: Test CA
content: |
{{ 'certs/test-ca.crt.pem' | file | indent(6, first=true) }}
apt:
sources: []
keys: []
apk:
repositories: []
yum:
repositories: []
keys: []
python:
repositories: []
java:
repositories: []
php:
repositories: []
npm:
repositories: []
rust:
repositories: []
go:
repositories: []
ruby:
repositories:
- name: demo
url: https://host.docker.internal:{{ 'RUBY_REPO_PORT' | env }}/
nuget:
repositories: []
+34
View File
@@ -0,0 +1,34 @@
uuid: 63db4c0e-a4c4-402c-bc4f-4eddeef0d890
organisation: InterDiode
description: Rust test configuration
configurations:
certificates:
- name: Test CA
content: |
{{ 'certs/test-ca.crt.pem' | file | indent(6, first=true) }}
apt:
sources: []
keys: []
apk:
repositories: []
yum:
repositories: []
keys: []
python:
repositories: []
java:
repositories: []
php:
repositories: []
npm:
repositories: []
rust:
repositories:
- name: demo
url: https://host.docker.internal:{{ 'CARGO_REPO_PORT' | env }}/
go:
repositories: []
ruby:
repositories: []
nuget:
repositories: []
+34
View File
@@ -0,0 +1,34 @@
uuid: 8f6a0e2a-5b7a-4d2c-9d1a-3c2b6b9c4b7e
organisation: InterDiode
description: Yum test configuration
configurations:
certificates:
- name: Test CA
content: |
{{ 'certs/test-ca.crt.pem' | file | indent(6, first=true) }}
apt:
sources: []
keys: []
apk:
repositories: []
yum:
repositories:
- name: demo
index-url: https://host.docker.internal:{{ 'YUM_REPO_PORT' | env }}/
keys: []
python:
repositories: []
java:
repositories: []
php:
repositories: []
npm:
repositories: []
rust:
repositories: []
go:
repositories: []
ruby:
repositories: []
nuget:
repositories: []
+10
View File
@@ -0,0 +1,10 @@
FROM alpine:3.20 AS target
FROM interdiode/id-configurator:latest AS configuration
COPY --from=target / /src/
COPY --from=generated-config Apk.yaml /config.yaml
RUN id-configurator apply --config /config.yaml --src /src --dst /dst
FROM target AS runtime
COPY --from=configuration /dst/ /
RUN apk update \
&& apk add idconfiguratortestpkg \
&& apk info idconfiguratortestpkg
+10
View File
@@ -0,0 +1,10 @@
FROM debian:trixie-slim AS target
FROM interdiode/id-configurator:latest AS configuration
COPY --from=target / /src/
COPY --from=generated-config Apt.yaml /config.yaml
RUN id-configurator apply --config /config.yaml --src /src --dst /dst
FROM target AS runtime
COPY --from=configuration /dst/ /
RUN apt-get update \
&& apt-get install -y --no-install-recommends idconfiguratortestpkg \
&& dpkg -l idconfiguratortestpkg
+15
View File
@@ -0,0 +1,15 @@
FROM debian:trixie-slim AS target
FROM interdiode/id-configurator:latest AS configuration
COPY --from=target / /src/
COPY --from=generated-config Certificate.yaml /config.yaml
RUN id-configurator apply --config /config.yaml --src /src --dst /dst
FROM target AS runtime
ARG CERTIFICATE_REPO_PORT
RUN apt-get update && apt-get install -y --no-install-recommends wget curl git openssl
COPY --from=configuration /dst/ /
RUN wget -qO- "https://host.docker.internal:${CERTIFICATE_REPO_PORT}/marker.txt" | grep -qx trusted-via-root-ca
RUN curl -fsS "https://host.docker.internal:${CERTIFICATE_REPO_PORT}/marker.txt" | grep -qx trusted-via-root-ca
RUN git clone -q "https://host.docker.internal:${CERTIFICATE_REPO_PORT}/repo.git" /tmp/clone \
&& grep -qx trusted-via-root-ca /tmp/clone/marker.txt
RUN echo | openssl s_client -connect "host.docker.internal:${CERTIFICATE_REPO_PORT}" \
-servername host.docker.internal -verify_hostname host.docker.internal -verify_return_error
+19
View File
@@ -0,0 +1,19 @@
FROM debian:trixie-slim AS target
FROM interdiode/id-configurator:latest AS configuration
ARG CONFIG_REPO_PORT
ARG CONFIG_URL_CACERT_CONTENT
COPY --from=target / /src/
RUN id-configurator apply \
--config-url "https://host.docker.internal:${CONFIG_REPO_PORT}/config.yaml" \
--config-url-cacert-content "$CONFIG_URL_CACERT_CONTENT" \
--src /src --dst /dst
FROM target AS runtime
ARG CONFIG_REPO_PORT
RUN apt-get update && apt-get install -y --no-install-recommends curl
COPY --from=configuration /dst/ /
# Proves the root CA the downloaded configuration installed is actually
# trusted system-wide: no --insecure/--no-check-certificate here, unlike the
# --config-url-cacert-content fetch above (which trusts only that one CA to
# validate the download itself, before anything has been installed yet).
RUN curl -fsS "https://host.docker.internal:${CONFIG_REPO_PORT}/config.yaml" \
| grep -q 'organisation: InterDiode'
+11
View File
@@ -0,0 +1,11 @@
FROM golang:1-bookworm AS target
FROM interdiode/id-configurator:latest AS configuration
COPY --from=target / /src/
COPY --from=generated-config Go.yaml /config.yaml
RUN id-configurator apply --config /config.yaml --src /src --dst /dst
FROM target AS runtime
COPY --from=configuration /dst/ /
RUN mkdir -p /tmp/gotest && cd /tmp/gotest && go mod init gotest \
&& go get example.invalid/idconfiguratortestpkg@v0.0.1 \
&& grep -qx 'test-only module served by the id-configurator integration tests' \
"$(find "$(go env GOMODCACHE)" -name marker.txt)"
+60
View File
@@ -0,0 +1,60 @@
FROM maven:3.9-eclipse-temurin-21 AS target
FROM interdiode/id-configurator:latest AS configuration
COPY --from=target / /src/
COPY --from=generated-config Java.yaml /config.yaml
RUN id-configurator apply --config /config.yaml --src /src --dst /dst
FROM target AS runtime
# Every build tool the `java` module configures: Maven itself (already part
# of this image), Gradle, Ivy and Leiningen (all packaged for this Ubuntu
# base), and SBT (no package available, fetched as a plain release tarball).
RUN apt-get update \
&& apt-get install -y --no-install-recommends gradle ivy leiningen curl \
&& curl -sL https://github.com/sbt/sbt/releases/download/v1.10.1/sbt-1.10.1.tgz -o /tmp/sbt.tgz \
&& tar -xzf /tmp/sbt.tgz -C /opt \
&& rm /tmp/sbt.tgz
ENV PATH="/opt/sbt/bin:${PATH}"
# Warm up every tool's own bootstrap/plugin dependencies from the real,
# network-reachable Maven Central while the default resolvers are still in
# place, so the actual tests below never need anything but the fixture
# repository configured by id-configurator (plus, for SBT only, the
# maven-central-fallback repository also configured below).
RUN mvn -q org.apache.maven.plugins:maven-dependency-plugin:3.6.1:get -Dartifact=org.apache.commons:commons-lang3:3.12.0
RUN mkdir -p /opt/sbt-warmup/project \
&& echo 'scalaVersion := "2.13.14"' > /opt/sbt-warmup/build.sbt \
&& echo 'sbt.version=1.10.1' > /opt/sbt-warmup/project/build.properties \
&& cd /opt/sbt-warmup && sbt update
COPY --from=configuration /dst/ /
RUN mvn org.apache.maven.plugins:maven-dependency-plugin:3.6.1:get -Dartifact=org.interdiode:idconfiguratortestpkg:0.0.1 \
&& test -f /root/.m2/repository/org/interdiode/idconfiguratortestpkg/0.0.1/idconfiguratortestpkg-0.0.1.jar
RUN mkdir -p /tmp/gradletest && cd /tmp/gradletest \
&& cat > build.gradle <<'EOF'
configurations { deps }
dependencies { deps 'org.interdiode:idconfiguratortestpkg:0.0.1' }
task resolveDeps { doLast { configurations.deps.each { println it } } }
EOF
RUN cd /tmp/gradletest && gradle -q resolveDeps | grep -q idconfiguratortestpkg
RUN java -jar /usr/share/java/ivy.jar -settings /root/.ivy2/ivysettings.xml \
-dependency org.interdiode idconfiguratortestpkg 0.0.1 \
-retrieve "/tmp/ivyretrieve/[artifact]-[revision].[ext]" \
&& test -f /tmp/ivyretrieve/idconfiguratortestpkg-0.0.1.jar
RUN mkdir -p /tmp/leintest && cd /tmp/leintest \
&& cat > project.clj <<'EOF'
(defproject leintest "0.1.0"
:dependencies [[org.interdiode/idconfiguratortestpkg "0.0.1"]])
EOF
RUN cd /tmp/leintest && lein deps :tree | grep -q idconfiguratortestpkg
RUN mkdir -p /tmp/sbttest/project \
&& cat > /tmp/sbttest/build.sbt <<'EOF'
scalaVersion := "2.13.14"
libraryDependencies += "org.interdiode" % "idconfiguratortestpkg" % "0.0.1"
EOF
RUN echo 'sbt.version=1.10.1' > /tmp/sbttest/project/build.properties
RUN cd /tmp/sbttest \
&& sbt -Dsbt.override.build.repos=true "show Compile/dependencyClasspath" | grep -q idconfiguratortestpkg
+12
View File
@@ -0,0 +1,12 @@
FROM node:20-slim AS target
FROM interdiode/id-configurator:latest AS configuration
COPY --from=target / /src/
COPY --from=generated-config Npm.yaml /config.yaml
RUN id-configurator apply --config /config.yaml --src /src --dst /dst
FROM target AS runtime
COPY --from=configuration /dst/ /
RUN mkdir -p /tmp/npmtest && cd /tmp/npmtest \
&& npm init -y >/dev/null \
&& npm install idconfiguratortestpkg \
&& grep -qx 'test-only package served by the id-configurator integration tests' \
node_modules/idconfiguratortestpkg/marker.txt
+11
View File
@@ -0,0 +1,11 @@
FROM mcr.microsoft.com/dotnet/sdk:9.0 AS target
FROM interdiode/id-configurator:latest AS configuration
COPY --from=target / /src/
COPY --from=generated-config NuGet.yaml /config.yaml
RUN id-configurator apply --config /config.yaml --src /src --dst /dst
FROM target AS runtime
COPY --from=configuration /dst/ /
RUN mkdir -p /tmp/nugettest && cd /tmp/nugettest && dotnet new console -o . --no-restore >/dev/null \
&& dotnet add package idconfiguratortestpkg --version 0.0.1 \
&& grep -qx 'test-only package served by the id-configurator integration tests' \
"$(find /root/.nuget/packages/idconfiguratortestpkg -name marker.txt)"
+16
View File
@@ -0,0 +1,16 @@
FROM php:8.3-cli AS target
FROM interdiode/id-configurator:latest AS configuration
COPY --from=target / /src/
COPY --from=generated-config Php.yaml /config.yaml
RUN id-configurator apply --config /config.yaml --src /src --dst /dst
FROM target AS runtime
RUN apt-get update \
&& apt-get install -y --no-install-recommends ca-certificates unzip curl \
&& curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer \
&& rm -rf /var/lib/apt/lists/*
COPY --from=configuration /dst/ /
RUN mkdir -p /tmp/phptest && cd /tmp/phptest \
&& composer init -n --name test/phptest \
&& composer require interdiode/idconfiguratortestpkg \
&& grep -qx 'test-only package served by the id-configurator integration tests' \
vendor/interdiode/idconfiguratortestpkg/marker.txt
@@ -0,0 +1,24 @@
FROM python:3.13-alpine AS target
FROM interdiode/id-configurator:latest AS configuration
COPY --from=target / /src/
COPY --from=generated-config Python.yaml /config.yaml
RUN id-configurator apply --config /config.yaml --src /src --dst /dst
FROM target AS runtime
ARG PYPI_REPO_PORT
# Warm up uv and poetry themselves from the real, network-reachable PyPI
# while the default index is still in place, so the actual tests below
# never need anything but the fixture repository configured by
# id-configurator. (easy_install/distutils, the fourth tool the `python`
# module configures, is not exercised here: both were removed from Python
# itself in 3.12, and setuptools no longer ships easy_install either, so
# there is no way to actually invoke it on a current interpreter.)
RUN pip install --no-cache-dir uv poetry
COPY --from=configuration /dst/ /
RUN pip install --no-cache-dir idconfiguratortestpkg \
&& python3 -c "import idconfiguratortestpkg"
RUN uv pip install --system idconfiguratortestpkg \
&& python3 -c "import idconfiguratortestpkg"
RUN mkdir -p /tmp/poetrytest && cd /tmp/poetrytest \
&& poetry init -n --name poetrytest \
&& poetry source add --priority explicit demo "https://host.docker.internal:${PYPI_REPO_PORT}/simple/" \
&& poetry add idconfiguratortestpkg --source demo
+24
View File
@@ -0,0 +1,24 @@
FROM python:3.13-slim AS target
FROM interdiode/id-configurator:latest AS configuration
COPY --from=target / /src/
COPY --from=generated-config Python.yaml /config.yaml
RUN id-configurator apply --config /config.yaml --src /src --dst /dst
FROM target AS runtime
ARG PYPI_REPO_PORT
# Warm up uv and poetry themselves from the real, network-reachable PyPI
# while the default index is still in place, so the actual tests below
# never need anything but the fixture repository configured by
# id-configurator. (easy_install/distutils, the fourth tool the `python`
# module configures, is not exercised here: both were removed from Python
# itself in 3.12, and setuptools no longer ships easy_install either, so
# there is no way to actually invoke it on a current interpreter.)
RUN pip install --no-cache-dir uv poetry
COPY --from=configuration /dst/ /
RUN pip install --no-cache-dir idconfiguratortestpkg \
&& python3 -c "import idconfiguratortestpkg"
RUN uv pip install --system idconfiguratortestpkg \
&& python3 -c "import idconfiguratortestpkg"
RUN mkdir -p /tmp/poetrytest && cd /tmp/poetrytest \
&& poetry init -n --name poetrytest \
&& poetry source add --priority explicit demo "https://host.docker.internal:${PYPI_REPO_PORT}/simple/" \
&& poetry add idconfiguratortestpkg --source demo
+10
View File
@@ -0,0 +1,10 @@
FROM ruby:3-slim AS target
FROM interdiode/id-configurator:latest AS configuration
COPY --from=target / /src/
COPY --from=generated-config Ruby.yaml /config.yaml
RUN id-configurator apply --config /config.yaml --src /src --dst /dst
FROM target AS runtime
COPY --from=configuration /dst/ /
RUN gem install idconfiguratortestpkg --no-document \
&& grep -qx 'test-only gem served by the id-configurator integration tests' \
"$(find "$(gem environment gemdir)" -name marker.txt)"
+16
View File
@@ -0,0 +1,16 @@
FROM rust:1-slim AS target
FROM interdiode/id-configurator:latest AS configuration
COPY --from=target / /src/
COPY --from=generated-config Rust.yaml /config.yaml
RUN id-configurator apply --config /config.yaml --src /src --dst /dst
FROM target AS runtime
COPY --from=configuration /dst/ /
# Deliberately not touching CARGO_HOME (the official image sets it to
# /usr/local/cargo): id-configurator writes /.cargo/config.toml, which
# cargo's config search always consults regardless of CARGO_HOME, precisely
# so this doesn't need to line up with wherever CARGO_HOME happens to point.
RUN cargo new /tmp/rusttest -q && cd /tmp/rusttest \
&& cargo add idconfiguratortestpkg \
&& cargo fetch \
&& grep -qx '// test-only crate served by the id-configurator integration tests' \
"$(find "$CARGO_HOME/registry/src" -name lib.rs)"
+9
View File
@@ -0,0 +1,9 @@
FROM rockylinux:9 AS target
FROM interdiode/id-configurator:latest AS configuration
COPY --from=target / /src/
COPY --from=generated-config Yum.yaml /config.yaml
RUN id-configurator apply --config /config.yaml --src /src --dst /dst
FROM target AS runtime
COPY --from=configuration /dst/ /
RUN dnf install -y idconfiguratortestpkg \
&& rpm -q idconfiguratortestpkg
@@ -0,0 +1,28 @@
-----BEGIN PRIVATE KEY-----
MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQC/MKMygUuccuON
KW9N1VvuVgLnUPRMpCg03Qp3yk2HtctbdHVKKFtgjm88u4tzLMl0p+b+x65xJhPn
veEres0iz0oYgbPWMaRNmPP5ahg8NJ6ZPQaRCYrOfFGS2cAH4/NiGu31aFd3vgnj
JqSztlpZ+VXr2OQnaUHUutH1ZSmHau9R7ZbuxYeAm9VMhHFfKe5rVOwObBR4HaBy
cgF1uF3eX4jlxKOECbDntaEkAEfszSiujvuT1nFw9kP/ImtTWGI9w7nJzhY7rFRN
LnTSapjQS8Ju6HHC5EafolPVvV52NI2+J9504Es2K/GoLIX3OmxSbayf9Qp0zx15
7FQ2vDm9AgMBAAECggEAVXIdxaEeUzXPd9OoUF9F/j8Hfizieew1oeaV2pKZ0siP
MXjbQIDH0d/mW/yvZpKCGHY/mOdSEXSEmeH1I7L2whNlWI2C+3vlI97zTqk5WpLj
vLWW3TQoCn9qD5jspzobmGFTrwVqbfQLRC8NldmuCJ8T5NtioDB9AN9bnXIQPNno
h2Oo6HIeX2kL5DlqIezj42nmlFegxKhxVbDiGl3qUUbKmA3H9PodAN2ZQ9fwoFws
wswdMSord+Ajfcab/Y8wUzIELDZm0PcxvN/nSrtRQ121Jy85DE5ZmX3IonwmsiWV
i+maYv2tTrNHbeCxshSlVWFOHuaRstd6yY52wC1lSwKBgQD/H4xedtGprQs0Or2y
1qoF+A+KzMocikIcWBjTr0a0IMrufeo8IwPMeNCHj3KoJeKVsOvFA4JEbo5rFiRU
W96hsJ0W0U3wM5ImTMbmZyGayYcADf4CEOoAscKqt4Z/CphJcNzXeyZardCrZvL8
kF15TB40EoCRbCYEthZE7/bAGwKBgQC/2NeWnBo2QanMd5pZV+hVs6iVL1iZ5Mz3
whqigSejubvZLznBo17KWQwtQ5b3wkteMe19FA9ZUXBudT5Q04Qr8MQRksDy+wN2
N83Lf+3IV3oiWBE4MaBw2rdEHMUKmo6CARxr0hWALlxmmxMFlQJRwavfq1GsGriL
TWgltRB7BwKBgCtjqDHgr2PWJFOrsduamqwNNkd68NzNvtSnPwGxi6rS8PCCxbWY
lurNmVmU0gE7AEBJKLAHGr6ctugLif426bRCH1VkH4c8nhyZbjJ9U7/JioY92vdq
/QFFOkBimiM9IA1NfqG/NhSx/w1HowfFXcBla7Wbd/ZDmGuwtDThQuwFAoGBAK9F
yjl0ZtI5DYubne2QTdi2qgeaWQVIryaQNMtQ2klP8FpNNiQ+QOT3lBNqk+vJnBfz
33iNsjQ1HlYcUZGAnVmTv14ayKN1R0VjWGZae4C0wwXLlffUBzPz7smQy0iTyizm
HPQ43KHeOVcYyLkp2ROesItJtvVY+E90qfkBQoxTAoGBAMfGmprXptsq9I1J1gME
p+uVV+4LkPCURP0r+HdIFtD5RRcJlOudEsl5k3hBPYVsrWp9SpA9+05Sa8GSkqkn
qCZdP/nmlo8SruqgVRpd1zj5hWlhtLOW6+pVog2YXGDlhc6BKI8yQiIvih8ou5p2
Mf+TBaqUBGTbIqBDv0yUvcQj
-----END PRIVATE KEY-----
@@ -0,0 +1,9 @@
-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAvzCjMoFLnHLjjSlvTdVb
7lYC51D0TKQoNN0Kd8pNh7XLW3R1SihbYI5vPLuLcyzJdKfm/seucSYT573hK3rN
Is9KGIGz1jGkTZjz+WoYPDSemT0GkQmKznxRktnAB+PzYhrt9WhXd74J4yaks7Za
WflV69jkJ2lB1LrR9WUph2rvUe2W7sWHgJvVTIRxXynua1TsDmwUeB2gcnIBdbhd
3l+I5cSjhAmw57WhJABH7M0oro77k9ZxcPZD/yJrU1hiPcO5yc4WO6xUTS500mqY
0EvCbuhxwuRGn6JT1b1edjSNvifedOBLNivxqCyF9zpsUm2sn/UKdM8deexUNrw5
vQIDAQAB
-----END PUBLIC KEY-----
+13
View File
@@ -0,0 +1,13 @@
pkgname=idconfiguratortestpkg
pkgver=0.0.1
pkgrel=0
pkgdesc="Test-only package served by the id-configurator integration tests"
url="https://example.invalid"
arch="noarch"
license="MIT"
options="!check"
package() {
mkdir -p "$pkgdir"/usr/share/doc/"$pkgname"
echo test-only-package > "$pkgdir"/usr/share/doc/"$pkgname"/marker
}
@@ -0,0 +1,12 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net9.0</TargetFramework>
<PackageId>idconfiguratortestpkg</PackageId>
<Version>0.0.1</Version>
<Authors>InterDiode</Authors>
<Description>test-only package served by the id-configurator integration tests</Description>
</PropertyGroup>
<ItemGroup>
<None Include="marker.txt" Pack="true" PackagePath="content/marker.txt" />
</ItemGroup>
</Project>
@@ -0,0 +1,46 @@
# Minimal Alpine (apk) repository with a signed index, serving one hand-built
# .apk package over TLS, using the same Test CA as the pypi-repo/apt-repo/
# yum-repo fixtures, used by tox-docker to exercise the `apk` module
# integration test (tests/configs/Apk.yaml.j2,
# tests/dockerfiles/Apk.Dockerfile). The package is signed with the static
# key pair 040d0af5f97a0a44.rsa / 040d0af5f97a0a44.rsa.pub, whose public
# half is what the `apk` module configuration is expected to install into
# /etc/apk/keys.
#
# The key's filename matters, not just its content: apk embeds the public
# key's *filename* (not just a verifiable signature) into the index it
# signs, and only accepts it if a key file with that exact name is present
# in /etc/apk/keys - trying other installed keys does not help. The name
# 040d0af5f97a0a44 is the id_configurator.modules.apk._key_filename() hash
# of this specific public key's content (see tests/configs/Apk.yaml.tmpl),
# i.e. the name the `apk` module will actually use when it installs this key
# from the configuration. Regenerating the key pair requires recomputing
# that hash and renaming both files (and updating this comment) to match.
#
# Built by tox-docker, whose build context is this directory (not the repo
# root) - every COPY source below must live under tests/dockerfiles/repos/.
FROM alpine:3.20 AS packagebuild
RUN apk add --no-cache abuild build-base
RUN adduser -D builder && addgroup builder abuild
COPY 040d0af5f97a0a44.rsa 040d0af5f97a0a44.rsa.pub /home/builder/.abuild/
COPY 040d0af5f97a0a44.rsa.pub /etc/apk/keys/040d0af5f97a0a44.rsa.pub
RUN printf 'PACKAGER_PRIVKEY="/home/builder/.abuild/040d0af5f97a0a44.rsa"\n' > /home/builder/.abuild/abuild.conf \
&& chown -R builder:builder /home/builder
COPY --chown=builder:builder APKBUILD /home/builder/pkg/APKBUILD
RUN su builder -c 'cd /home/builder/pkg && abuild checksum && abuild -r'
# apk always appends the *client's* architecture to the repository URL, so
# the built index/package land in a directory named after whatever
# architecture this image was actually built for (native to the Docker host
# unless emulated). A flat copy of the index at the repo root lets the
# healthcheck probe it without knowing that architecture name in advance.
RUN arch_dir=$(ls /home/builder/packages/builder) \
&& mkdir -p /srv \
&& cp -r "/home/builder/packages/builder/$arch_dir" "/srv/$arch_dir" \
&& cp "/home/builder/packages/builder/$arch_dir/APKINDEX.tar.gz" /srv/APKINDEX.tar.gz
FROM nginx:alpine
COPY --from=packagebuild /srv /srv
COPY repo-server.crt.pem /etc/nginx/certs/repo-server.crt.pem
COPY repo-server.key.pem /etc/nginx/certs/repo-server.key.pem
COPY apk-repo.nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 8443
@@ -0,0 +1,10 @@
server {
listen 8443 ssl default_server;
server_name _;
ssl_certificate /etc/nginx/certs/repo-server.crt.pem;
ssl_certificate_key /etc/nginx/certs/repo-server.key.pem;
root /srv;
autoindex off;
}
@@ -0,0 +1,27 @@
# Minimal flat-format APT repository serving one hand-built .deb package over
# TLS, used by tox-docker to exercise the `apt` module integration test
# (tests/configs/Apt.yaml.tmpl, tests/dockerfiles/Apt.Dockerfile).
#
# Built by tox-docker, whose build context is this directory (not the repo
# root) - every COPY source below must live under tests/dockerfiles/repos/.
FROM debian:trixie-slim AS packagebuild
RUN apt-get update \
&& apt-get install -y --no-install-recommends dpkg-dev \
&& rm -rf /var/lib/apt/lists/*
RUN set -eux; \
pkg=idconfiguratortestpkg; \
root=/build/$pkg; \
mkdir -p "$root/DEBIAN" "$root/usr/share/doc/$pkg"; \
printf 'test-only package served by the id-configurator integration tests\n' > "$root/usr/share/doc/$pkg/marker"; \
printf 'Package: %s\nVersion: 0.0.1\nArchitecture: all\nMaintainer: InterDiode <contact@mail.interdiode.fr>\nDescription: Test-only package served by the id-configurator integration tests\n' "$pkg" > "$root/DEBIAN/control"; \
dpkg-deb --root-owner-group --build "$root" "/srv/${pkg}_0.0.1_all.deb"
WORKDIR /srv
RUN dpkg-scanpackages --multiversion . > Packages \
&& gzip -9c Packages > Packages.gz
FROM nginx:alpine
COPY --from=packagebuild /srv /srv
COPY repo-server.crt.pem /etc/nginx/certs/repo-server.crt.pem
COPY repo-server.key.pem /etc/nginx/certs/repo-server.key.pem
COPY apt-repo.nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 8443
@@ -0,0 +1,10 @@
server {
listen 8443 ssl default_server;
server_name _;
ssl_certificate /etc/nginx/certs/repo-server.crt.pem;
ssl_certificate_key /etc/nginx/certs/repo-server.key.pem;
root /srv;
autoindex off;
}
@@ -0,0 +1,94 @@
"""Build a minimal, offline, hand-rolled crate and its sparse registry index
entry, for integration-testing the `rust` module of id-configurator.
No network access and no build backend (cargo) is used: the crate is a
plain gzipped tar assembled with the standard library only, so this script
can run in a minimal Python image with nothing installed.
See https://doc.rust-lang.org/cargo/reference/registries.html#index-format
for the sparse registry index/config.json format this reproduces.
"""
from __future__ import annotations
import hashlib
import io
import json
import pathlib
import tarfile
CRATE = "idconfiguratortestpkg"
VERSION = "0.0.1"
CRATE_FILE = f"{CRATE}-{VERSION}.crate"
OUT_DIR = pathlib.Path("/srv")
CARGO_TOML = f"""[package]
name = "{CRATE}"
version = "{VERSION}"
edition = "2021"
"""
LIB_RS = "// test-only crate served by the id-configurator integration tests\n"
def _add_file(archive: tarfile.TarFile, name: str, content: bytes) -> None:
info = tarfile.TarInfo(name=f"{CRATE}-{VERSION}/{name}")
info.size = len(content)
archive.addfile(info, io.BytesIO(content))
def build_crate() -> bytes:
buffer = io.BytesIO()
with tarfile.open(fileobj=buffer, mode="w:gz") as archive:
_add_file(archive, "Cargo.toml", CARGO_TOML.encode("utf-8"))
_add_file(archive, "src/lib.rs", LIB_RS.encode("utf-8"))
return buffer.getvalue()
def index_path(name: str) -> pathlib.Path:
"""Mirror cargo's sparse index path convention for a crate name."""
if len(name) <= 2:
return pathlib.Path(str(len(name))) / name
if len(name) == 3:
return pathlib.Path("3") / name[0] / name
return pathlib.Path(name[0:2]) / name[2:4] / name
def write_index(out_dir: pathlib.Path, crate: bytes) -> None:
entry = {
"name": CRATE,
"vers": VERSION,
"deps": [],
"cksum": hashlib.sha256(crate).hexdigest(),
"features": {},
"yanked": False,
}
path = out_dir / index_path(CRATE)
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps(entry) + "\n", encoding="utf-8")
def write_config_json(out_dir: pathlib.Path) -> None:
# Cargo requires a fully-qualified "dl" download template, so this bakes
# in a placeholder that nginx's sub_filter rewrites to the real
# https://<host used by the client>/ at request time (see
# cargo-repo.nginx.conf) - the host and port this fixture is actually
# reached on aren't known at image-build time.
config = {
"dl": "__BASE_URL__/api/v1/crates/{crate}/{version}/download",
"api": "__BASE_URL__",
}
(out_dir / "config.json").write_text(json.dumps(config), encoding="utf-8")
def main() -> None:
OUT_DIR.mkdir(parents=True, exist_ok=True)
crate = build_crate()
download_path = OUT_DIR / "api/v1/crates" / CRATE / VERSION / "download"
download_path.parent.mkdir(parents=True, exist_ok=True)
download_path.write_bytes(crate)
write_index(OUT_DIR, crate)
write_config_json(OUT_DIR)
if __name__ == "__main__":
main()
@@ -0,0 +1,76 @@
"""Build a minimal, offline, hand-rolled Composer package and its
`packages.json` index, for integration-testing the `php` module of
id-configurator.
No network access and no build backend (composer) is used: the package is a
plain zip file assembled with the standard library only, so this script can
run in a minimal Python image with nothing installed.
"""
from __future__ import annotations
import hashlib
import io
import json
import pathlib
import zipfile
VENDOR = "interdiode"
PACKAGE = "idconfiguratortestpkg"
NAME = f"{VENDOR}/{PACKAGE}"
VERSION = "0.0.1"
ZIP_NAME = f"{PACKAGE}-{VERSION}.zip"
OUT_DIR = pathlib.Path("/srv")
COMPOSER_JSON = json.dumps({"name": NAME, "version": VERSION, "type": "library"})
def build_zip() -> bytes:
buffer = io.BytesIO()
with zipfile.ZipFile(buffer, "w", zipfile.ZIP_DEFLATED) as archive:
archive.writestr("composer.json", COMPOSER_JSON)
archive.writestr(
"marker.txt",
"test-only package served by the id-configurator integration tests\n",
)
return buffer.getvalue()
def write_packages_json(out_dir: pathlib.Path, zip_content: bytes) -> None:
# Composer requires a fully-qualified dist URL: unlike PyPI's simple
# index, it does not resolve relative/root-relative ones against the
# repository's own URL for this (non-v2) repository format. The host
# and port this fixture will actually be reached on aren't known at
# image-build time (tox-docker only assigns them once the container
# starts), so this bakes in a placeholder that nginx's sub_filter
# rewrites to the real https://<host used by the client>/ at request
# time (see php-repo.nginx.conf).
packages = {
"packages": {
NAME: {
VERSION: {
"name": NAME,
"version": VERSION,
"dist": {
"type": "zip",
"url": f"__BASE_URL__/{ZIP_NAME}",
"shasum": hashlib.sha1(zip_content).hexdigest(), # nosec B324
},
}
}
}
}
(out_dir / "packages.json").write_text(
json.dumps(packages, indent=2), encoding="utf-8"
)
def main() -> None:
OUT_DIR.mkdir(parents=True, exist_ok=True)
zip_content = build_zip()
(OUT_DIR / ZIP_NAME).write_bytes(zip_content)
write_packages_json(OUT_DIR, zip_content)
if __name__ == "__main__":
main()
+32
View File
@@ -0,0 +1,32 @@
#!/bin/sh
# Build a minimal, hand-rolled RubyGems package and a full gem repository
# index for it (legacy + compact index formats), for integration-testing
# the `ruby` module of id-configurator. Uses RubyGems' own `gem build` and
# `gem generate_index` (the same tools a real gem server host would use)
# rather than hand-rolling the compact index's binary/marshal formats.
set -eu
PACKAGE=idconfiguratortestpkg
VERSION=0.0.1
MARKER="test-only gem served by the id-configurator integration tests"
gem install rubygems-generate_index --no-document
mkdir -p "/build/${PACKAGE}-${VERSION}"
cd "/build/${PACKAGE}-${VERSION}"
cat > "${PACKAGE}.gemspec" <<EOF
Gem::Specification.new do |s|
s.name = "${PACKAGE}"
s.version = "${VERSION}"
s.summary = "test-only gem"
s.authors = ["InterDiode"]
s.license = "MIT"
s.files = ["marker.txt"]
end
EOF
echo "${MARKER}" > marker.txt
gem build "${PACKAGE}.gemspec"
mkdir -p /srv/gems
cp ./*.gem /srv/gems/
gem generate_index -d /srv
@@ -0,0 +1,47 @@
"""Build a minimal, offline, hand-rolled Go module proxy tree, for
integration-testing the `go` module of id-configurator.
No network access and no Go toolchain is used: the module proxy protocol
(https://go.dev/ref/mod#module-proxy) only requires a handful of static
files, none of which embed the server's own base URL (unlike Composer/
npm/cargo), so this can be served directly by nginx with no rewriting.
"""
from __future__ import annotations
import io
import json
import pathlib
import zipfile
MODULE = "example.invalid/idconfiguratortestpkg"
VERSION = "v0.0.1"
OUT_DIR = pathlib.Path("/srv")
MARKER = "test-only module served by the id-configurator integration tests\n"
GO_MOD = f"module {MODULE}\n\ngo 1.21\n"
def build_zip() -> bytes:
buffer = io.BytesIO()
prefix = f"{MODULE}@{VERSION}"
with zipfile.ZipFile(buffer, "w") as archive:
archive.writestr(f"{prefix}/go.mod", GO_MOD)
archive.writestr(f"{prefix}/marker.txt", MARKER)
return buffer.getvalue()
def main() -> None:
module_dir = OUT_DIR / MODULE / "@v"
module_dir.mkdir(parents=True, exist_ok=True)
(module_dir / "list").write_text(f"{VERSION}\n", encoding="utf-8")
(module_dir / f"{VERSION}.info").write_text(
json.dumps({"Version": VERSION, "Time": "2024-01-01T00:00:00Z"}),
encoding="utf-8",
)
(module_dir / f"{VERSION}.mod").write_text(GO_MOD, encoding="utf-8")
(module_dir / f"{VERSION}.zip").write_bytes(build_zip())
if __name__ == "__main__":
main()
+66
View File
@@ -0,0 +1,66 @@
"""Build a minimal, offline, hand-rolled jar and its pom.xml, laid out as a
Maven-compatible repository, for integration-testing the `java` module of
id-configurator.
No network access and no build backend (maven/gradle) are used: the jar is
a plain zip file assembled with the standard library only, so this script
can run in a minimal Python image with nothing installed.
"""
from __future__ import annotations
import hashlib
import io
import pathlib
import zipfile
GROUP = "org.interdiode"
GROUP_PATH = GROUP.replace(".", "/")
ARTIFACT = "idconfiguratortestpkg"
VERSION = "0.0.1"
OUT_DIR = pathlib.Path("/srv") / GROUP_PATH / ARTIFACT / VERSION
MANIFEST = "Manifest-Version: 1.0\n"
POM = f"""<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0">
<modelVersion>4.0.0</modelVersion>
<groupId>{GROUP}</groupId>
<artifactId>{ARTIFACT}</artifactId>
<version>{VERSION}</version>
<packaging>jar</packaging>
<description>Test-only artifact served by the id-configurator integration tests</description>
</project>
"""
def build_jar() -> bytes:
buffer = io.BytesIO()
with zipfile.ZipFile(buffer, "w", zipfile.ZIP_DEFLATED) as archive:
archive.writestr("META-INF/MANIFEST.MF", MANIFEST)
archive.writestr(
"marker.txt",
"test-only artifact served by the id-configurator integration tests\n",
)
return buffer.getvalue()
def _write_with_checksums(path: pathlib.Path, content: bytes) -> None:
"""Write `content` to `path`, plus the .sha1/.md5 sidecar files most
build tools (Leiningen's aether resolver in particular) require to be
present, unlike Maven which merely warns when they are missing."""
path.write_bytes(content)
sha1_digest = hashlib.sha1(content).hexdigest() # nosec B324
(path.parent / f"{path.name}.sha1").write_text(sha1_digest, encoding="ascii")
md5_digest = hashlib.md5(content).hexdigest() # nosec B324
(path.parent / f"{path.name}.md5").write_text(md5_digest, encoding="ascii")
def main() -> None:
OUT_DIR.mkdir(parents=True, exist_ok=True)
_write_with_checksums(OUT_DIR / f"{ARTIFACT}-{VERSION}.jar", build_jar())
_write_with_checksums(OUT_DIR / f"{ARTIFACT}-{VERSION}.pom", POM.encode("utf-8"))
if __name__ == "__main__":
main()
@@ -0,0 +1,76 @@
"""Build a minimal, offline, hand-rolled npm package and its registry
metadata document, for integration-testing the `npm` module of
id-configurator.
No network access and no build backend (npm/node) is used: the tarball is a
plain gzipped tar assembled with the standard library only, so this script
can run in a minimal Python image with nothing installed.
"""
from __future__ import annotations
import base64
import hashlib
import io
import json
import pathlib
import tarfile
PACKAGE = "idconfiguratortestpkg"
VERSION = "0.0.1"
TARBALL_NAME = f"{PACKAGE}-{VERSION}.tgz"
OUT_DIR = pathlib.Path("/srv")
PACKAGE_JSON = json.dumps({"name": PACKAGE, "version": VERSION})
MARKER = "test-only package served by the id-configurator integration tests\n"
def _add_file(archive: tarfile.TarFile, name: str, content: bytes) -> None:
info = tarfile.TarInfo(name=f"package/{name}")
info.size = len(content)
archive.addfile(info, io.BytesIO(content))
def build_tarball() -> bytes:
buffer = io.BytesIO()
with tarfile.open(fileobj=buffer, mode="w:gz") as archive:
_add_file(archive, "package.json", PACKAGE_JSON.encode("utf-8"))
_add_file(archive, "marker.txt", MARKER.encode("utf-8"))
return buffer.getvalue()
def write_registry_metadata(out_dir: pathlib.Path, tarball: bytes) -> None:
# npm requires a fully-qualified tarball URL, so this bakes in a
# placeholder that nginx's sub_filter rewrites to the real
# https://<host used by the client>/ at request time (see
# npm-repo.nginx.conf) - the host and port this fixture is actually
# reached on aren't known at image-build time.
sha1 = hashlib.sha1(tarball).hexdigest() # nosec B324
sha512_b64 = base64.b64encode(hashlib.sha512(tarball).digest()).decode("ascii")
metadata = {
"name": PACKAGE,
"dist-tags": {"latest": VERSION},
"versions": {
VERSION: {
"name": PACKAGE,
"version": VERSION,
"dist": {
"shasum": sha1,
"integrity": f"sha512-{sha512_b64}",
"tarball": f"__BASE_URL__/{TARBALL_NAME}",
},
}
},
}
(out_dir / PACKAGE).write_text(json.dumps(metadata, indent=2), encoding="utf-8")
def main() -> None:
OUT_DIR.mkdir(parents=True, exist_ok=True)
tarball = build_tarball()
(OUT_DIR / TARBALL_NAME).write_bytes(tarball)
write_registry_metadata(OUT_DIR, tarball)
if __name__ == "__main__":
main()
@@ -0,0 +1,61 @@
"""Build a minimal NuGet v3 static feed around one real, `dotnet pack`-built
.nupkg, for integration-testing the `nuget` module of id-configurator.
Only two resources are populated: the flat container (`PackageBaseAddress`,
used to enumerate versions and download the .nupkg directly) and the
top-level service index pointing at it. `dotnet add package`/`dotnet
restore` with an explicit, dependency-free package version only ever calls
these two - never the (considerably more complex) registration resource -
so it is intentionally not implemented here.
The service index bakes in a __BASE_URL__ placeholder for its "@id" fields,
since the host:port this fixture is actually reached on is only known once
tox-docker starts the container, not at image-build time (see
nuget-repo.nginx.conf). The flat container's own files never embed a URL,
so they need no such rewriting.
"""
from __future__ import annotations
import json
import pathlib
import shutil
PACKAGE_ID = "idconfiguratortestpkg"
VERSION = "0.0.1"
NUPKG_NAME = f"{PACKAGE_ID}.{VERSION}.nupkg"
NUPKG_SRC = pathlib.Path("/nupkg") / NUPKG_NAME
OUT_DIR = pathlib.Path("/srv")
def write_flat_container() -> None:
package_dir = OUT_DIR / "flatcontainer" / PACKAGE_ID
version_dir = package_dir / VERSION
version_dir.mkdir(parents=True, exist_ok=True)
(package_dir / "index.json").write_text(
json.dumps({"versions": [VERSION]}), encoding="utf-8"
)
shutil.copy(NUPKG_SRC, version_dir / NUPKG_NAME)
def write_service_index() -> None:
index = {
"version": "3.0.0",
"resources": [
{
"@id": "__BASE_URL__/flatcontainer/",
"@type": "PackageBaseAddress/3.0.0",
}
],
}
(OUT_DIR / "index.json").write_text(json.dumps(index), encoding="utf-8")
def main() -> None:
OUT_DIR.mkdir(parents=True, exist_ok=True)
write_flat_container()
write_service_index()
if __name__ == "__main__":
main()
@@ -0,0 +1,80 @@
"""Build a minimal, offline, hand-rolled wheel and PEP 503 "simple" index for
integration-testing the `python` module of id-configurator.
No network access and no build backend (setuptools/build/pip) are used: the
wheel is a plain zip file assembled with the standard library only, so this
script can run in a minimal Python image with nothing installed.
"""
from __future__ import annotations
import base64
import hashlib
import pathlib
import zipfile
PACKAGE = "idconfiguratortestpkg"
VERSION = "0.0.1"
WHEEL_NAME = f"{PACKAGE}-{VERSION}-py3-none-any.whl"
OUT_DIR = pathlib.Path("/srv")
METADATA = f"""Metadata-Version: 2.1
Name: {PACKAGE}
Version: {VERSION}
Summary: Test-only package served by the id-configurator integration tests
"""
WHEEL_FILE = """Wheel-Version: 1.0
Generator: id-configurator-tests
Root-Is-Purelib: true
Tag: py3-none-any
"""
INIT_PY = '"""Test-only package used to verify the python module mirror."""\n'
def _record_line(path: str, content: bytes) -> str:
digest = base64.urlsafe_b64encode(hashlib.sha256(content).digest()).rstrip(b"=")
return f"{path},sha256={digest.decode('ascii')},{len(content)}"
def build_wheel(packages_dir: pathlib.Path) -> bytes:
dist_info = f"{PACKAGE}-{VERSION}.dist-info"
files = {
f"{PACKAGE}/__init__.py": INIT_PY.encode("utf-8"),
f"{dist_info}/METADATA": METADATA.encode("utf-8"),
f"{dist_info}/WHEEL": WHEEL_FILE.encode("utf-8"),
}
record_lines = [_record_line(name, content) for name, content in files.items()]
record_lines.append(f"{dist_info}/RECORD,,")
files[f"{dist_info}/RECORD"] = ("\n".join(record_lines) + "\n").encode("utf-8")
packages_dir.mkdir(parents=True, exist_ok=True)
wheel_path = packages_dir / WHEEL_NAME
with zipfile.ZipFile(wheel_path, "w", zipfile.ZIP_DEFLATED) as archive:
for name, content in files.items():
archive.writestr(name, content)
return wheel_path.read_bytes()
def write_simple_index(simple_dir: pathlib.Path, wheel_content: bytes) -> None:
package_dir = simple_dir / PACKAGE
package_dir.mkdir(parents=True, exist_ok=True)
digest = hashlib.sha256(wheel_content).hexdigest()
html = f"""<!DOCTYPE html>
<html>
<body>
<a href="../../packages/{WHEEL_NAME}#sha256={digest}">{WHEEL_NAME}</a>
</body>
</html>
"""
(package_dir / "index.html").write_text(html, encoding="utf-8")
def main() -> None:
wheel_content = build_wheel(OUT_DIR / "packages")
write_simple_index(OUT_DIR / "simple", wheel_content)
if __name__ == "__main__":
main()
@@ -0,0 +1,17 @@
# Minimal cargo sparse registry (index entry, config.json and one hand-built
# crate) served over TLS, using the same Test CA as the other *-repo
# fixtures, used by tox-docker to exercise the `rust` module integration
# test (tests/configs/Rust.yaml.j2, tests/dockerfiles/Rust.Dockerfile).
#
# Built by tox-docker, whose build context is this directory (not the repo
# root) - every COPY source below must live under tests/dockerfiles/repos/.
FROM python:3.13-slim AS packagebuild
COPY build_test_cargo_package.py /build_test_cargo_package.py
RUN python3 /build_test_cargo_package.py
FROM nginx:alpine
COPY --from=packagebuild /srv /srv
COPY repo-server.crt.pem /etc/nginx/certs/repo-server.crt.pem
COPY repo-server.key.pem /etc/nginx/certs/repo-server.key.pem
COPY cargo-repo.nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 8443
@@ -0,0 +1,22 @@
server {
listen 8443 ssl default_server;
server_name _;
ssl_certificate /etc/nginx/certs/repo-server.crt.pem;
ssl_certificate_key /etc/nginx/certs/repo-server.key.pem;
root /srv;
autoindex off;
# config.json bakes in a __BASE_URL__ placeholder for its "dl"/"api"
# templates, since the host:port this fixture is actually reached on is
# only known once tox-docker starts the container, not at image-build
# time. $http_host reflects exactly what the client (cargo) used to
# connect, e.g. host.docker.internal:55123.
location = /config.json {
default_type application/json;
sub_filter '__BASE_URL__' 'https://$http_host';
sub_filter_once off;
sub_filter_types application/json;
}
}
@@ -0,0 +1,32 @@
# Minimal HTTPS server, using the same Test CA as the other *-repo fixtures,
# used by tox-docker to exercise the `certificates` module integration test
# (tests/configs/Certificate.yaml.j2, tests/dockerfiles/Certificate.Dockerfile).
# Unlike the package-manager fixtures, this one has nothing to do with a
# specific package format: it serves a static marker file (for wget, curl and
# openssl s_client) and a bare git repository over the "dumb" HTTP protocol
# (for `git clone`).
#
# Built by tox-docker, whose build context is this directory (not the repo
# root) - every COPY source below must live under tests/dockerfiles/repos/.
FROM alpine:3.20 AS gitbuild
RUN apk add --no-cache git
RUN git config --global init.defaultBranch main \
&& git config --global user.email test@example.invalid \
&& git config --global user.name "id-configurator tests"
RUN mkdir -p /work \
&& cd /work \
&& git init -q \
&& echo "trusted-via-root-ca" > marker.txt \
&& git add marker.txt \
&& git commit -q -m "initial commit"
RUN git clone --bare -q /work /srv/repo.git \
&& cd /srv/repo.git \
&& git update-server-info
FROM nginx:alpine
RUN mkdir -p /srv && echo "trusted-via-root-ca" > /srv/marker.txt
COPY --from=gitbuild /srv/repo.git /srv/repo.git
COPY repo-server.crt.pem /etc/nginx/certs/repo-server.crt.pem
COPY repo-server.key.pem /etc/nginx/certs/repo-server.key.pem
COPY certificate-repo.nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 8443
@@ -0,0 +1,10 @@
server {
listen 8443 ssl default_server;
server_name _;
ssl_certificate /etc/nginx/certs/repo-server.crt.pem;
ssl_certificate_key /etc/nginx/certs/repo-server.key.pem;
root /srv;
autoindex off;
}
@@ -0,0 +1,40 @@
# Static id-configurator configuration served by the config-url-repo fixture,
# to be downloaded (not copied in locally) via `id-configurator apply
# --config-url ... --config-url-cacert-content ...`
# (tests/dockerfiles/ConfigUrl.Dockerfile). No per-run templating is needed:
# unlike the module-* fixtures, nothing in this document needs to reference
# its own server's dynamic host:port.
#
# The certificate below is tests/certs/test-ca.crt.pem - the same Test CA
# that signs repo-server.crt.pem in this directory. It's duplicated here
# (rather than read at build time) because a Docker build's context is
# this directory alone; keep both copies in sync if the Test CA is ever
# regenerated.
uuid: 8f3e4b2c-9a41-4d3e-8f1a-2b6c7d8e9f01
organisation: InterDiode
description: Configuration downloaded via --config-url
configurations:
certificates:
- name: Test CA
content: |
-----BEGIN CERTIFICATE-----
MIIDdTCCAl2gAwIBAgIUKq4URHQwyg7IzJ3ViAGGlM9pGCwwDQYJKoZIhvcNAQEL
BQAwQjEeMBwGA1UECgwVaWQtY29uZmlndXJhdG9yLXRlc3RzMSAwHgYDVQQDDBdp
ZC1jb25maWd1cmF0b3IgVGVzdCBDQTAeFw0yNjA5MDUyMTA4MTNaFw00NjA4MzEy
MTA4MTNaMEIxHjAcBgNVBAoMFWlkLWNvbmZpZ3VyYXRvci10ZXN0czEgMB4GA1UE
AwwXaWQtY29uZmlndXJhdG9yIFRlc3QgQ0EwggEiMA0GCSqGSIb3DQEBAQUAA4IB
DwAwggEKAoIBAQD8bMteh3MDnbW6OoFMlhA4Z2QAabVoSFlSE/Si7ut+L/3v1GY4
lixenwLQc8a2koIH9CtmOodzfO7hpro/VUjYJ9J2eTsXLLiOg7OgggG1cKV9/kt7
RF0Q4hkcanGmiffc4xbz4ZedU6lP1vzLCsgU0A8OPt6svcNjGcWbjxss0KjgxGZ7
H4UKdL7Mv4W2Y6Fw1b4eV2M6jErkQK6IPXsHGFh7JkzliNcb0sX1VfbfauS84tBh
f3EHwzLNhbjtEmQhQdqgsg4Cbk/gWur//6aEvlD/X12PAUbrp4DFD+CDuVagLX7I
7TQBXNFGKhozzQuDg7DFg1gIV8SFx2m+s60/AgMBAAGjYzBhMB0GA1UdDgQWBBSb
zJ7DsaxwCeqbfOXGJ3x+u7bjozAfBgNVHSMEGDAWgBSbzJ7DsaxwCeqbfOXGJ3x+
u7bjozAPBgNVHRMBAf8EBTADAQH/MA4GA1UdDwEB/wQEAwIBBjANBgkqhkiG9w0B
AQsFAAOCAQEAEV1+dG3agh0S/AR8jC8M33HXFyvneCCRtfb5q/loJ//VoWPbD3Gs
G+2SR0MVN+KKOMBvCBh3KrNSgFF9iFmBjNCJJZ8ycDIszUK3xLjUP2EzN1pNNl8S
D/yHzUCro3547Q+I+Nn5zRciG95EpMvtkIWem4H8luYSBt7w8y9uDUKWzaca/v3z
oGZPZP99w8WVqMKGRbsK94wsTVsWiE+oHzSufwrhvZJfQQ4jU6102IEnRS/eQFXA
+fozgEWuFatZe3Gz0HUJakrzSBwSTV6wmnEHCfu+uwdns1Ttq5FylZ4StJHsmweL
14w1xa3tjC5bQ3e1QWK3R5InHcSZ7tLpGA==
-----END CERTIFICATE-----
@@ -0,0 +1,13 @@
# Serves a static id-configurator configuration document over TLS, using the
# same Test CA as the other *-repo fixtures, used by tox-docker to exercise
# `id-configurator apply --config-url ... --config-url-cacert-content ...`
# (tests/dockerfiles/ConfigUrl.Dockerfile).
#
# Built by tox-docker, whose build context is this directory (not the repo
# root) - every COPY source below must live under tests/dockerfiles/repos/.
FROM nginx:alpine
COPY config-url-config.yaml /srv/config.yaml
COPY repo-server.crt.pem /etc/nginx/certs/repo-server.crt.pem
COPY repo-server.key.pem /etc/nginx/certs/repo-server.key.pem
COPY config-url-repo.nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 8443
@@ -0,0 +1,10 @@
server {
listen 8443 ssl default_server;
server_name _;
ssl_certificate /etc/nginx/certs/repo-server.crt.pem;
ssl_certificate_key /etc/nginx/certs/repo-server.key.pem;
root /srv;
autoindex off;
}
@@ -0,0 +1,17 @@
# Minimal Go module proxy serving one hand-built module over TLS, using the
# same Test CA as the other *-repo fixtures, used by tox-docker to exercise
# the `go` module integration test (tests/configs/Go.yaml.j2,
# tests/dockerfiles/Go.Dockerfile).
#
# Built by tox-docker, whose build context is this directory (not the repo
# root) - every COPY source below must live under tests/dockerfiles/repos/.
FROM python:3.13-slim AS packagebuild
COPY build_test_go_module.py /build_test_go_module.py
RUN python3 /build_test_go_module.py
FROM nginx:alpine
COPY --from=packagebuild /srv /srv
COPY repo-server.crt.pem /etc/nginx/certs/repo-server.crt.pem
COPY repo-server.key.pem /etc/nginx/certs/repo-server.key.pem
COPY go-repo.nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 8443
@@ -0,0 +1,10 @@
server {
listen 8443 ssl default_server;
server_name _;
ssl_certificate /etc/nginx/certs/repo-server.crt.pem;
ssl_certificate_key /etc/nginx/certs/repo-server.key.pem;
root /srv;
autoindex off;
}
@@ -0,0 +1,16 @@
Name: idconfiguratortestpkg
Version: 0.0.1
Release: 1
Summary: Test-only package served by the id-configurator integration tests
License: MIT
BuildArch: noarch
%description
Test-only package served by the id-configurator integration tests.
%install
mkdir -p %{buildroot}/usr/share/doc/%{name}
echo 'test-only package served by the id-configurator integration tests' > %{buildroot}/usr/share/doc/%{name}/marker
%files
/usr/share/doc/%{name}/marker
@@ -0,0 +1,17 @@
# Minimal Maven-compatible repository serving one hand-built jar over TLS,
# using the same Test CA as the pypi-repo/apt-repo/yum-repo/apk-repo
# fixtures, used by tox-docker to exercise the `java` module integration
# test (tests/configs/Java.yaml.j2, tests/dockerfiles/Java.Dockerfile).
#
# Built by tox-docker, whose build context is this directory (not the repo
# root) - every COPY source below must live under tests/dockerfiles/repos/.
FROM python:3.13-slim AS jarbuild
COPY build_test_jar.py /build_test_jar.py
RUN python3 /build_test_jar.py
FROM nginx:alpine
COPY --from=jarbuild /srv /srv
COPY repo-server.crt.pem /etc/nginx/certs/repo-server.crt.pem
COPY repo-server.key.pem /etc/nginx/certs/repo-server.key.pem
COPY java-repo.nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 8443
@@ -0,0 +1,10 @@
server {
listen 8443 ssl default_server;
server_name _;
ssl_certificate /etc/nginx/certs/repo-server.crt.pem;
ssl_certificate_key /etc/nginx/certs/repo-server.key.pem;
root /srv;
autoindex off;
}
@@ -0,0 +1,17 @@
# Minimal npm registry (one package's metadata document plus its tarball)
# served over TLS, using the same Test CA as the other *-repo fixtures, used
# by tox-docker to exercise the `npm` module integration test
# (tests/configs/Npm.yaml.j2, tests/dockerfiles/Npm.Dockerfile).
#
# Built by tox-docker, whose build context is this directory (not the repo
# root) - every COPY source below must live under tests/dockerfiles/repos/.
FROM python:3.13-slim AS packagebuild
COPY build_test_npm_package.py /build_test_npm_package.py
RUN python3 /build_test_npm_package.py
FROM nginx:alpine
COPY --from=packagebuild /srv /srv
COPY repo-server.crt.pem /etc/nginx/certs/repo-server.crt.pem
COPY repo-server.key.pem /etc/nginx/certs/repo-server.key.pem
COPY npm-repo.nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 8443
@@ -0,0 +1,22 @@
server {
listen 8443 ssl default_server;
server_name _;
ssl_certificate /etc/nginx/certs/repo-server.crt.pem;
ssl_certificate_key /etc/nginx/certs/repo-server.key.pem;
root /srv;
autoindex off;
# The package metadata document bakes in a __BASE_URL__ placeholder for
# its tarball URL, since the host:port this fixture is actually reached
# on is only known once tox-docker starts the container, not at
# image-build time. $http_host reflects exactly what the client (npm)
# used to connect, e.g. host.docker.internal:55123.
location = /idconfiguratortestpkg {
default_type application/json;
sub_filter '__BASE_URL__' 'https://$http_host';
sub_filter_once off;
sub_filter_types application/json;
}
}
@@ -0,0 +1,23 @@
# Minimal NuGet v3 static feed serving one real, dotnet-packed .nupkg over
# TLS, using the same Test CA as the other *-repo fixtures, used by
# tox-docker to exercise the `nuget` module integration test
# (tests/configs/NuGet.yaml.j2, tests/dockerfiles/NuGet.Dockerfile).
#
# Built by tox-docker, whose build context is this directory (not the repo
# root) - every COPY source below must live under tests/dockerfiles/repos/.
FROM mcr.microsoft.com/dotnet/sdk:9.0 AS packagebuild
COPY IdConfiguratorTestPkg.csproj /pkgsrc/IdConfiguratorTestPkg.csproj
RUN echo 'test-only package served by the id-configurator integration tests' > /pkgsrc/marker.txt \
&& dotnet pack /pkgsrc -c Release -o /nupkg
FROM python:3.13-slim AS feedbuild
COPY --from=packagebuild /nupkg /nupkg
COPY build_test_nuget_feed.py /build_test_nuget_feed.py
RUN python3 /build_test_nuget_feed.py
FROM nginx:alpine
COPY --from=feedbuild /srv /srv
COPY repo-server.crt.pem /etc/nginx/certs/repo-server.crt.pem
COPY repo-server.key.pem /etc/nginx/certs/repo-server.key.pem
COPY nuget-repo.nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 8443
@@ -0,0 +1,22 @@
server {
listen 8443 ssl default_server;
server_name _;
ssl_certificate /etc/nginx/certs/repo-server.crt.pem;
ssl_certificate_key /etc/nginx/certs/repo-server.key.pem;
root /srv;
autoindex off;
# The service index bakes in a __BASE_URL__ placeholder for its "@id"
# fields, since the host:port this fixture is actually reached on is
# only known once tox-docker starts the container, not at image-build
# time. $http_host reflects exactly what the client (nuget) used to
# connect, e.g. host.docker.internal:55123.
location = /index.json {
default_type application/json;
sub_filter '__BASE_URL__' 'https://$http_host';
sub_filter_once off;
sub_filter_types application/json;
}
}
@@ -0,0 +1,17 @@
# Minimal Composer repository ("composer" type: a static packages.json plus
# one hand-built zip package) served over TLS, using the same Test CA as the
# other *-repo fixtures, used by tox-docker to exercise the `php` module
# integration test (tests/configs/Php.yaml.j2, tests/dockerfiles/Php.Dockerfile).
#
# Built by tox-docker, whose build context is this directory (not the repo
# root) - every COPY source below must live under tests/dockerfiles/repos/.
FROM python:3.13-slim AS packagebuild
COPY build_test_composer_package.py /build_test_composer_package.py
RUN python3 /build_test_composer_package.py
FROM nginx:alpine
COPY --from=packagebuild /srv /srv
COPY repo-server.crt.pem /etc/nginx/certs/repo-server.crt.pem
COPY repo-server.key.pem /etc/nginx/certs/repo-server.key.pem
COPY php-repo.nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 8443
@@ -0,0 +1,22 @@
server {
listen 8443 ssl default_server;
server_name _;
ssl_certificate /etc/nginx/certs/repo-server.crt.pem;
ssl_certificate_key /etc/nginx/certs/repo-server.key.pem;
root /srv;
autoindex off;
# packages.json bakes in a __BASE_URL__ placeholder for its dist URL,
# since the host:port this fixture is actually reached on is only known
# once tox-docker starts the container, not at image-build time.
# $http_host reflects exactly what the client (Composer) used to
# connect, e.g. host.docker.internal:55123.
location = /packages.json {
default_type application/json;
sub_filter '__BASE_URL__' 'https://$http_host';
sub_filter_once off;
sub_filter_types application/json;
}
}
@@ -0,0 +1,16 @@
# Minimal PEP 503 "simple" package index, serving one hand-built test wheel
# over TLS, used by tox-docker to exercise the `python` module integration
# test (tests/configs/Python.yaml.tmpl, tests/dockerfiles/Python.Dockerfile).
#
# Built by tox-docker, whose build context is this directory (not the repo
# root) - every COPY source below must live under tests/dockerfiles/repos/.
FROM python:3.13-slim AS wheelbuild
COPY build_test_wheel.py /build_test_wheel.py
RUN python3 /build_test_wheel.py
FROM nginx:alpine
COPY --from=wheelbuild /srv /srv
COPY repo-server.crt.pem /etc/nginx/certs/repo-server.crt.pem
COPY repo-server.key.pem /etc/nginx/certs/repo-server.key.pem
COPY pypi-repo.nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 8443
@@ -0,0 +1,10 @@
server {
listen 8443 ssl default_server;
server_name _;
ssl_certificate /etc/nginx/certs/repo-server.crt.pem;
ssl_certificate_key /etc/nginx/certs/repo-server.key.pem;
root /srv;
autoindex off;
}
@@ -0,0 +1,22 @@
-----BEGIN CERTIFICATE-----
MIIDqTCCApGgAwIBAgIUUuJBYcNFFtJcKH2PDPTA+QZMP3QwDQYJKoZIhvcNAQEL
BQAwQjEeMBwGA1UECgwVaWQtY29uZmlndXJhdG9yLXRlc3RzMSAwHgYDVQQDDBdp
ZC1jb25maWd1cmF0b3IgVGVzdCBDQTAeFw0yNjA5MDMyMTM5NDNaFw00NjA4Mjky
MTM5NDNaMDYxHjAcBgNVBAoMFWlkLWNvbmZpZ3VyYXRvci10ZXN0czEUMBIGA1UE
AwwLcmVwby1zZXJ2ZXIwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCu
xCtsiQxqntOBY+LlKxgXs5N0WSKJY5jeof0SzIJWNr7W8GBMbh9fqnXms4Vnimhs
AYa6RsTmfGpSzLO73+NlzpuApV9kTBXZmeHVqrJ6S/mFKBZDN74uFiDWl93K1Z/G
VccntAdbCsNCMRt71CZ7DD0vFfQSN4qQ3fPDT9EpV8Xv6Pj/MFJvEMp+pu4f7mwh
xJQBtlC72okHvcMp6MsbfDFYRTUWFtsDpCct7pps1sRVyalLvxIaFugzpuiTU2+Q
0FdlBi5IRhCZjrOlW4kGAyjiP+q/SwZ8j1YJnjifoxLrcn+26ONCNfv/JKsnl8Yr
hKGwbrlzEgeoekE8TH9xAgMBAAGjgaIwgZ8wCQYDVR0TBAIwADALBgNVHQ8EBAMC
BaAwEwYDVR0lBAwwCgYIKwYBBQUHAwEwMAYDVR0RBCkwJ4IUaG9zdC5kb2NrZXIu
aW50ZXJuYWyCCWxvY2FsaG9zdIcEfwAAATAdBgNVHQ4EFgQUmPGECu+/yUR3pzKk
RSDMtRQvFEAwHwYDVR0jBBgwFoAUm8yew7GscAnqm3zlxid8fru246MwDQYJKoZI
hvcNAQELBQADggEBAPZ3xMe6vECPfrWGEru2eSL75JCUl+zLZm40EkCLidiBTXch
dvuJYHEjp6oSnTdgPOPlyjgFY/jMkHuRvl46Zwr74eFBZyYqaYxm50Drc4/09/VF
bo59/fRbO6UCk/yvTw83q5URL2d9zw79dnWJaOfCHlm9laY2DvD5cPZbYNZ1dlEg
nwNiRmw2/twIjuvMBGG+Ooy1V2rOOzR+ytcyf7Ekpotc4DpS/NLC0cPL/gjDoJMc
Ot/YZo55Myrrk8IN681ou5Kv+vAUURxCcMaJaFMNfM4HBPW2dwrOp5tplGw0C/pX
Kqub6yjjUmzPCIRiWUsW+4mg3UxaL41Q9013Mlw=
-----END CERTIFICATE-----
@@ -0,0 +1,28 @@
-----BEGIN PRIVATE KEY-----
MIIEvAIBADANBgkqhkiG9w0BAQEFAASCBKYwggSiAgEAAoIBAQCuxCtsiQxqntOB
Y+LlKxgXs5N0WSKJY5jeof0SzIJWNr7W8GBMbh9fqnXms4VnimhsAYa6RsTmfGpS
zLO73+NlzpuApV9kTBXZmeHVqrJ6S/mFKBZDN74uFiDWl93K1Z/GVccntAdbCsNC
MRt71CZ7DD0vFfQSN4qQ3fPDT9EpV8Xv6Pj/MFJvEMp+pu4f7mwhxJQBtlC72okH
vcMp6MsbfDFYRTUWFtsDpCct7pps1sRVyalLvxIaFugzpuiTU2+Q0FdlBi5IRhCZ
jrOlW4kGAyjiP+q/SwZ8j1YJnjifoxLrcn+26ONCNfv/JKsnl8YrhKGwbrlzEgeo
ekE8TH9xAgMBAAECggEADfew5MUoEEloHzo4qX1bn2FQLDqHfEvqegVssLKJl1+O
ukAoSIawmTiKOola5LiQhrlK53S9sRIWfIGbVJOcz/rGpAetHqeTqeCwYB96980+
QJPzzr5HKjwrtoFMTgHrk0vKiMDM3OWr6PL8Me+PSvXic4xUnzQZlGE4zAGDQ10V
xCs0ZrCYGVbMxzOWSBZivd5Tbj7OeCuLEPRqUX5D824bloaudLtO+0pEe8oaFEdt
cgwMPUGdz9+XPQlaYPOtcTbk0fTcrUYzyjcYR1CwI48xj6jXnLvFAdTjZnjSgmSr
lcxOJNxpRgj8MYvHjHQekyxxBUF3HsaWfbTscWzghQKBgQDyVMgirF7bykCe+E/q
HZSio3xdX8TTVyJvxBZeG+sAHFvc0Z0ihA1CrIZIZOYK6c+mF1CHVQ5jPEc0GKwC
mzXjqO3kLEUumbrBGXtmPQd7wJ/jVTv9u4JsfqCzMVj3xNTEbpmp5FvZ2JofEb5f
7mFWKl1L1JZ58M5B1XUgYSqZ/wKBgQC4n8NZgq1k1jsxGzTggJzDvi230lkIJG1+
Tf3wm8AoqCoGtG4O53a6fpsNKlNgNgivk5cUUACj2HRXC44tz5/MydRLRzePLG8M
MCjrmPEHCD9zKKn9z0DMjHVL2TAy3aRD1Flo8HcERMD8xyklX2kPtJqkiKUJm2Gm
5qrRlRuGjwKBgHYCeZABFxS7sqp3zBr7msV1Dfw+xhART9DT0fy0ZcWhCc9IONh1
LCv67k3U7P4FjLyaf/nXnvEOirJBDYsbYKvkOS0zreEatgqRSTmpoLpQPMytVlCi
/smtBL8e2e+W5/Jc1fIZJe25gJx+vJHk+XvHl7V06m6pqRgARlClYOSrAoGAZ4uS
8FzOCsPsJrpviYoYYeADnBkpPdvQXQo45f00ZJkMGP6Qy2Mdv7LKGzQl1sv3SyGn
lgjYkwwPE9KzRxYvNZfsrNugWXg+P8Oy74+zfpIto9SmombenNoZ3xRMZ/rO6gvq
qWRUG9zDsAGzHfbI4boVETph9WVj+0tbenf6HBkCgYBWM8J+M+UoHmKZyarZFKum
uuVHFM4WzdBVM22NyFFrkYmfCD6tnNb2ZfcwabX2NY+d2AJLA9aNdVlgQHb/SJaC
orwNrOuW0wLjGEivXZieGTRHwrcS7IMtG78nm/fMuEMjEwTQkW61bGmSmWxCCnGe
2uuQwlL68XytRGkpgPKK6g==
-----END PRIVATE KEY-----
@@ -0,0 +1,17 @@
# Minimal RubyGems repository serving one hand-built gem over TLS, using the
# same Test CA as the other *-repo fixtures, used by tox-docker to exercise
# the `ruby` module integration test (tests/configs/Ruby.yaml.j2,
# tests/dockerfiles/Ruby.Dockerfile).
#
# Built by tox-docker, whose build context is this directory (not the repo
# root) - every COPY source below must live under tests/dockerfiles/repos/.
FROM ruby:3-slim AS packagebuild
COPY build_test_gem.sh /build_test_gem.sh
RUN sh /build_test_gem.sh
FROM nginx:alpine
COPY --from=packagebuild /srv /srv
COPY repo-server.crt.pem /etc/nginx/certs/repo-server.crt.pem
COPY repo-server.key.pem /etc/nginx/certs/repo-server.key.pem
COPY ruby-repo.nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 8443
@@ -0,0 +1,10 @@
server {
listen 8443 ssl default_server;
server_name _;
ssl_certificate /etc/nginx/certs/repo-server.crt.pem;
ssl_certificate_key /etc/nginx/certs/repo-server.key.pem;
root /srv;
autoindex off;
}

Some files were not shown because too many files have changed in this diff Show More