projectsyn/commodore

= Concepts

== Configuration Hierarchy

Commodore uses the https://kapitan.dev/inventory/[Kapitan inventory], which
uses https://reclass.pantsfullofunix.net/[reclass], to provide a hierarchical
configuration lookup.

Commodore doesn't assume how your configuration hierarchy will look like.
Instead, the hierarchy can be configured within the hierarchy itself.
Commodore only prepares the entrypoint of the hierarchy to build upon.

See xref:commodore:ROOT:reference/hierarchy.adoc[inventory hierarchy] for further details.

== Inventory

The Commodore inventory is really a https://kapitan.dev/inventory/[Kapitan inventory].

Commodore collects the contents of the inventory from different Git repositories.
Those Git repositories are either global (to the Syn instance) or in the scope
of a customer.
A customer may have one or more configuration repositories for their different
Project Syn-managed platforms.
The inventory Git repositories aren't versioned, as they always reflect the
desired state.

Inventory repositories are cloned directly into the Kapitan `inventory` directory to make their contents available as inventory classes in Kapitan.

Commodore also makes use of the inventory to manage xref:commodore:ROOT:reference/architecture.adoc#_component_discovery_and_versions[component repository locations and versions].

== Configuration Packages

Commodore provides support to fetch additional inventory classes based on configurations present in the global and tenant repositories
These inventory classes are stored in Git repositories.
Commodore calls these bundles of inventory classes _configuration packages_.
Commodore allows configuration packages to be stored in an arbitrary directory in the repository.

Configuration packages can include components and provide arbitrary configuration parameters.
Commodore clones the Git repositories containing configuration packages into `dependencies/pkg.<package-name>` and symlinks them to `inventory/classes/<package-name>` to make the contents available as Kapitan inventory classes.
To ensure components included through packages are processed correctly, Commodore ensures that configuration packages are present before discovering components.

See https://syn.tools/syn/SDDs/0028-reusable-config-packages.html[SDD 0028 - Reusable Commodore Component Configuration Packages] for more details on the design of configuration packages.

== Components

Each Commodore component is managed in an individual Git repository and provides all the code which necessary to install some software or tool.
The repository name must be named the same as the component.
Commodore allows components to be stored in an arbitrary directory in the repository.

Each component includes the
https://kapitan.dev/compile/#supported-input-types[Kapitan templates] (for
example Jsonnet), and
https://kapitan.dev/inventory/#inventory-classes[classes] which define how to
install the software. The classes must be included somewhere in the Kapitan
inventory to install the software.

Components can optionally provide Jsonnet libraries which can be used by other components.
To allow Commodore to make component libraries available to other components, they must be placed in directory `lib/`.
Components must ensure that all Jsonnet libraries which they provide are prefixed with the component name.
For example, component `foo-bar` can only contain libraries whose file name starts with `foo-bar`, such as `foo-bar.libsonnet`, or `foo-bar-baz.libsonnet`.
Commodore allows components to advertise aliases for their component libraries.
Component library aliases are described in more detail in the xref:commodore:ROOT:reference/architecture.adoc#_component_template_libraries[architecture documentation].

Components can specify Jsonnet dependencies in a `jsonnetfile.json` in the component's root directory.
Commodore uses https://github.com/jsonnet-bundler/jsonnet-bundler[jsonnet-bundler] to fetch all components' dependencies.
Optionally, components can provide a `jsonnetfile.jsonnet` instead of `jsonnetfile.json`.
If a `jsonnetfile.jsonnet` is present it will be rendered to `jsonnetfile.json` before Commodore fetches component dependencies.
The `jsonnetfile.jsonnet` approach is described in more details in the xref:commodore:ROOT:reference/architecture.adoc#_component_dependencies[architecture documentation].

Components can define and use postprocessing filters which are applied on the result of the component's compiled Kapitan catalog.
Components define postprocessing filters in the <<_inventory,inventory>> in key `parameters.commodore.postprocess.filters`.
Details of the filter definition format and the postprocessing process can be found in the xref:commodore:ROOT:reference/architecture.adoc#_postprocessing_filters[architecture documentation].

Component parameters expose a limited set of configuration options for the
software managed by the component.
The component is expected to provide sensible default values for most, if not
all, component parameters.
One common component parameter, which appears in most components, allows
configuring which version of the software will be installed.

A component must define two Kapitan classes, which must be available as
`class/<component-name>.yml` and `class/defaults.yml` in the component
repository.

The class `class/<component-name>.yml` defines how the component is compiled,
and is commonly referred to as the _component class_.
The class `class/defaults.yml` defines defaults for the component parameters,
and is commonly referred to as the _component defaults_.

Each component repository is cloned into `dependencies/`, its component
class is symlinked to `inventory/classes/components/<component-name>.yml` and
its component defaults is symlinked to `inventory/classes/defaults/<component-name>.yml`.

Commodore includes all component defaults in the
<<_configuration_hierarchy,configuration hierarchy>> before the global
defaults (`global.common`).

Splitting the component defaults into a separate class which gets included
early in the <<_configuration_hierarchy,configuration hierarchy>> allows
components to provide defaults which are guaranteed to have the lowest
priority when resolving the hierarchy.

Component versions and remote repository locations can be specified in the
<<_inventory,inventory repositories>>, and versions can be expressed as
arbitrary
https://git-scm.com/docs/gitglossary#Documentation/gitglossary.txt-aiddeftree-ishatree-ishalsotreeish[Git
tree-ishs].

Commodore provides a reclass variable `${_base_directory}` which always refers to the component's base directory.
Users are encouraged to use `${_base_directory}` when specifying Kapitan dependencies and compilation inputs in the component class.
The component template uses `${_base_directory}` for the generated Kapitan compilation inputs.

== Cluster Catalog

The cluster catalog contains the rendered manifests generated by Commodore and
Kapitan and the Kapitan <<_secret_references,secret references>>.
The cluster catalog is stored as a Git repository and checked out as
`catalog/` when Commodore operates on it.
Commodore puts the rendered manifests (the Kapitan output) in directory
`manifests/` in the catalog repository.
Secret references are stored in directory `refs/` in the catalog repository.

== Secret References

Commodore makes use of Kapitan's https://kapitan.dev/secrets/[secret
references] mechanism to ensure no plain text secrets are present in the
<<_cluster_catalog,cluster catalog>>.

At this time, Commodore only supports the Kapitan "Vaultkv" secret type.