design/m1.1_search_and_replace_design.md from cyberark/secrets-provider-for-k8s

design/m1.1_search_and_replace_design.md
Summary

Maintainability

Test Coverage

Issues
# DRAFT: Solution Design - Kubernetes Developer Experience M1.1 - Config File Search-and-Replace

## Table of Contents

- [Useful Links](#useful-links)
- [Background](#background)
  + [Objective](#objective)
  + [The Challenges](#the-challenges)
  + [Competitive Solutions](#competitive-solutions)
  + [Alternatives to Consider](#alternatives-to-consider)
- [Solution](#solution)
  + [User Experience](#user-experience)
  + [Performance Metrics](#performance-metrics)
  + [Upgrading Existing Secrets Provider Deployments](#upgrading-existing-secrets-provider-deployments)
  + [Project Scope and Limitations](#project-scope-and-limitations)
- [Design](#design)
  + [Flow Diagram](#flow-diagram)
  + [Workflow](#workflow)
- [Performance](#performance)
- [Backwards Compatibility](#backwards-compatibility)
- [Affected Components](#affected-components)
- [Test Plan](#test-plan)
  + [Test environments](#test-environments)
  + [Test assumptions](#test-assumptions)
  + [Out of scope](#out-of-scope)
  + [Test prerequisites](#test-prerequisites)
  + [Test Cases](#test-cases)
- [Logs](#logs)
- [Documentation](#documentation)
- [Version update](#version-update)
- [Security](#security)
  + [Preventing Leaking of Sensitive Information in Error Logs](#preventing-leaking-of-sensitive-information-in-error-logs)
- [Audit](#audit)
- [Development Tasks](#development-tasks)
- [Definition of Done](#definition-of-done)
  + [Future Work](#future-work)
- [Solution Review](#solution-review)
- [Appendix](#appendix)

## Useful Links

<table>
<thead>
<tr class="header">
<th>Name</th>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td>M1 PRD: Conjur Developer Experience for K8s</td>
<td><p>Sharepoint (private)</p></td>
</tr>
<tr class="even">
<td>Aha Card</td>
<td><p>SCR-E-76 (private)</p></td>
</tr>
<tr class="odd">
<td>M1 Feature Doc</td>
<td>Sharepoint (private)</td>
</tr>
<tr class="even">
<td>M1 Solution Design Doc</td>
<td><a href="https://github.com/cyberark/secrets-provider-for-k8s/blob/main/design/m1_push_to_file_design.md">link</a></td>
</tr>
<tr class="odd">
<td>Kubernetes Documentation:Annotations</td>
<td><a href="https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/">link</a></td>
</tr>
<tr class="even">
<td>Exposing Pod Annotations to Containers</td>
<td><a href="https://kubernetes.io/docs/tasks/inject-data-application/downward-api-volume-expose-pod-information/">link</a></td>
</tr>
<tr class="odd">
<td>Secrets Provider Documentation</td>
<td><a href="https://docs.cyberark.com/Product-Doc/OnlineHelp/AAM-DAP/11.2/en/Content/Integrations/Kubernetes_deployApplicationsConjur-k8s-Secrets.htm?tocpath=Integrations%7COpenShift%252C%20Kubernetes%252C%20and%20GKE%7CDeploy%20Applications%7C_____3">link</a></td>
</tr>
</tbody>
</table>

## Background

### Objective

The [Milestone 1 (M1) Push-to-File](https://github.com/cyberark/secrets-provider-for-k8s/blob/main/design/m1_push_to_file_design.md)
initiative enhanced the Secrets Provider init container integration by adding
the capability of **creating and writing** "secrets" files directly into an
application container's file system using shared volumes.

As a followup to the initial M1 milestone, the M1.1 milestone proposes
to provide the Secrets Provider init container with the capability of
**modifying** existing application configuration files using a
search-and-replace mechanism to insert secrets that have been retrieved
from Conjur into the configuration file.

The intent is to provide a more declarative method for personas such as
developers or DevOps to specify how to apply Conjur secrets to configuration
files. This should help simplify the process of applying or upgrading environment-specific
secrets or configuration (e.g. secrets to be used for development vs. secrets
for production) to those configuration files.

### The Challenges

#### Kubernetes Creates Empty Directories for Shared Volumes

The primary challenge to this design is to determine a convenient way for
making the original (or baseline) configuration file available to the
Secrets Provider init container so that the Secrets Provider can make
modifications.

Unfortunately, it's not possible for an application container to allow access
to the original configuration file to the Secrets Provider by simply using a
[Kubernetes shared volume](https://kubernetes.io/docs/concepts/storage/volumes/#emptydir),
even if Kubernetes [subPaths](https://kubernetes.io/docs/concepts/storage/volumes/#using-subpath)
are used to minimize the granularity of sharing down to the per-file level.
The problem is that Kubernetes uses `emptyDir` volumes to provide file sharing
between containers of a Pod, and as the name implies, Kubernetes creates
an empty directory at the `mountPath` used for the shared volume by each
container. Once the `emptyDir` volume has been mounted, any pre-existing files
that had been in a container's file system at that `mountPath` are no longer
visible.

A good blog that describes this challenge can be found
[here](https://www.fairwinds.com/blog/kubernetes-emptydir-not-the-same-as-dockers-volumes-from).

#### Kubernetes ConfigMap and Downward API Volumes are Read-Only

In alternative solutions that are considered below, it's important to keep
in mind that Kubernetes volumes that are used to mount ConfigMap data or
Downward API metadata are created with read-only access. This is intentional,
and is a security measure (e.g. so that processes running in the container
can't change the intended configuration to elevate privilege).

This means that if we use ConfigMaps or Annotations to make the original
configuration file available to the Secrets Provider, we can't simply modify
the configuration file and write the modified file back to the same volume.

### Competitive Solutions

#### Hashicorp: Create/Modify Config File Using Templates

The [Hashicorp Vault Agent Container](https://learn.hashicorp.com/tutorials/vault/kubernetes-sidecar?in=vault/kubernetes)
also allows configuration files to be created and modified using templates,
as described
[here](https://learn.hashicorp.com/tutorials/vault/kubernetes-sidecar?in=vault/kubernetes#apply-a-template-to-the-injected-secrets).

Again, this is not a search-and-replace option, since the structure of the
configuration file is defined via a template.

#### Hashicorp: Mount Secrets through Container Storage Interface (CSI)

Hashicorp supports a
[Kubernetes Container Storage Interace (CSI) driver](https://learn.hashicorp.com/tutorials/vault/kubernetes-secret-store-driver?in=vault/kubernetes)
for integrating applications with their Vault.

Strictly speaking, this is also not a search-and-replace option, since
secrets/files would be injected into application file systems in their
entirety (without modification).

It may be possible, although probably overly complicated,  to use a CSI
driver to inject an original configuration file into a Secrets Provider
init container, which can then be modified and written to a volume shared
by an application container.

### Alternatives to Consider

#### How to Make the Original Configuration File Available to Secrets Provider?

As described in the
[Using a ConfigMap to Share the Original Config File](#using-a-configmap-to-share-the-original-config-file)
section below, this design recommends using a ConfigMap to expose the
the original config file to the Secrets Provider, at least for the initial
release of the M1.1 Search-and-Replace feature.

Here are some of the options that were considered for making the original
config file available to the Secrets Provider init container:

| Method | Description |
|--------|-------------|
| Share via ConfigMap | A developer/DevOps would create a custom ConfigMap that contains the original, baseline config file. The Secrets Provider would read the original config file, make modifications, and write the modified file to a volume shared with the application. |
| Share via Pod Annotations | A developer/DevOps would apply a special Pod Annotation that contains the original config file as the Annotation value. The Secrets Provider would read the original config file via Downward API volume, make modifications, and write the modified file to a volume shared with the application. |
| Store original file as secret in Conjur | A copy of the original, unmodified configuration file could be stored as a secret in Conjur. The Secrets Provider would retrieve the original config file from Conjur, make modifications, and write the modified file to a volume shared with the application. |
| Store copy of original file online | A copy of the original, unmodified configuration file could be stored in online storage (e.g. AWS S3, Google Cloud Storage, etc.) The Secrets Provider would retrieve the online, original config file, make modifications, and write the modified file to a volume shared with the application. |
| Share via custom init container | A developer/DevOps would need to create a custom init container that contains the original config file. At run time, this init container would copy the config file to a volume shared with the Secrets Provider. The Secrets Provider would make modifications and write the modified file to another volume shared with the application. |
| Modify the application to copy the config file to shared volume | A developer/DevOps would need to modify the application image or deployment manifest so that the application container copies the original config file from its file system to a shared volume at startup. This would require that the Secrets Provider be run as a sidecar container rather than an init container. |
| Use a `kubectl` sidecar container to copy file from/to application container | An additional sidecar container could be developed that is capable of using `kubectl cp ...` (or the Kubernetes API) to read the original config file from the application container. This needs to be run as a sidecar since it requires the application container to be running when the sidecar reads the original config file. This would also require that the Secrets Provider be run as a sidecar container, and that the operations of the `kubectl` sidecar, the Secrets Provider sidecar, and the application container be synchronized to run in sequence. |

Of these options, the approach of using a ConfigMap for sharing the original
config file was chosen because it is probably the simplest to use from a
customer's perspective. Compared to the approach of using Annotations, this
approach has the advantage that it avoids cluttering the resulting Deployment
manifest with config file content. The (relatively small) downside is that
developers/DevOps personas are required to define one extra ConfigMap `volume`
and `volumeMount` in their application manifests for exposing the original
config file.

#### What Syntax to Use for Search-and-Replace Specifications?

In addition to exposing the original config file content via a ConfigMap,
this design recommends using Pod Annotations for configuring the
Search-and-Replace feature (see the [Pod Annotations](#pod-annotations)
section below).

Rather than introducing a new syntax (or language) to be used for specifying
search-and-replace operations to be performed on a config file, it will be
much easier to use this feature if the search-and-replace operations can
be specified using a well-known, industry standard syntax.

For example, let's say we have an original config file that is a basic,
`.ini` file:

    ```
    [database]
    server=192.0.2.62
    port=143
    cert="<Database certificate goes here>"
    ```

And let's say that we want the following search-and-replace operations
to be performed to customize this config file:

- Replace the database server IP address with an IP address retrieved as a Conjur secret.
- Replace the database server port number with a port number retrieved as a Conjur secret.
- Replace the certificate value with a certificate has been retrieved as a
  Conjur secret that needs to be base64 decoded before being inserted into
  the config file.

Here are several options that were considered for providing users with a
familiar or well-known syntax for specifying the search-and-replace operations
to perform on the configuration file:

| Option | Example |
|--------|---------|
| Use `sed` substitution syntax | <pre><code>conjur.org/config-file-modifications.payroll: &#124;</code><br /><code>   - s/server=.\*/server={{.db-server}}/</code><br /><code>   - s/port=.\*/port={{.db-port}}/</code><br />   - s/cert=.\*/cert="{{.db-cert &#124; b64dec}}"/</code></pre> |
| Use `awk` substitution syntax | <pre><code>conjur.org/config-file-modifications.payroll: &#124;</code><br /><code>   - sub(/server=.\*/,server={{.db-server}})</code><br /><code>   - sub(/port=.\*/,port={{.db-port}})</code><br /><code>   - sub(/cert=.\*/,cert="{{.db-cert &#124; b64dec}})"</code></pre> |
| Use match/replace `regex` expression pairs | <pre><code>conjur.org/config-file-modifications.payroll: &#124;</code><br /><code>   - match: "server=.\*"</code><br /><code>     replace: "server={{.db-server}}"</code><br /><code>   - match: "port=.\*"</code><br /><code>     replace: "port={{.db-port}}"</code><br /><code>   - match: "cert=.\*"</code><br /><code>     replace: "cert="{{.db-cert &#124; b64dec}}"</code></pre> |

Some notes on these options:

- Each of the examples uses "replacement templates" that are represented
  as [Go templates](https://pkg.go.dev/text/template). For example:
  - `{{.db-server}}`: Replace this field with the value retrieved from Conjur
    for the secret with the alias `db-server`.
  - `{{.db-port}}`: Replace this field with the value retrieved from Conjur
    for the secret with the alias `db-port`.
  - `{{.db-cert | b64dec}}`: Replace this field with the base64-decoded
    value retrieved from Conjur for the secret with the alias `db-cert`.
- The syntax for both `sed` and `awk` conveniently use a single string
  to specify each search-and-replace operation.
- Both `sed` and `awk` syntax can incorporate `regex` matching.
- Each of these syntax options can be extended to include support for
  regex capture groups. (See
  [this blog](https://www.ryanchapin.com/using-sed-with-regex-capture-groups/)
  for an example of how regex capture groups can be used with `sed`.)

## Solution

### User Experience

#### Using a ConfigMap to Share the Original Config File

The initial design for M1.1 Search-and-Replace will support using a ConfigMap
to provide the Secrets Provider with a copy of the original configuration file
(for reasons described in the section
[How to Make the Original Configuration File Available to Secrets Provider?](#how-to-make-the-original-configuration-file-available-to-secrets-provider)
above).

##### Example ConfigMap for a Simple, Original Config File

Here is an example ConfigMap that provides a simple config file:

```
apiVersion: v1
kind: ConfigMap
metadata:
    name: payroll-ini-configmap
data:
    # Sample application .ini file
    payroll.ini: |
      ; last modified 1 April 2019
      [owner]
      name=John Doe
      organization=Acme Widgets Inc.
      [database]
      server=192.0.2.62
      port=143
      file=payroll.dat
      cert="<Database certificate goes here>"
```

##### Example Volume Mounts for the Secrets Provider

For the M1.1 Search-and-Replace feature, the Secrets Provider init container
will require two `volumeMount` definitions in its corresponding Deployment
manifest:

- One `volumeMount` for reading the original config file (via a ConfigMap `volume`)
- One `volumeMount` for writing the modified config file (via an emptyDir `volume`)

Here are example `volumeMount` definitions for the Secrets Provider init
container.

_**NOTE: The paths used in the `mountPath` definitions for Search-and-Replace
 must be written exactly as shown. The Secrets Provider is hard-coded to
 look for the original config file and will write the modified config file
 to these respective directories. The names used for the `volumeMounts` are
 arbitrary. **_

```
        volumeMounts:
        - name: payroll-config-orig
          mountPath: /conjur/config-file-src
        - name: payroll-config
          mountPath: /conjur/config-file-dest
```

##### Example Volume Mount for the Application Container

For the M1.1 Search-and-Replace feature, the application will require a
`volumeMount` definition for specifying a path from which the application will
be expected to read the modified config file. The path used for the
`mountPath` definition is application-specific.

Here is an example:

```
        volumeMounts:
        - name: payroll-config
          mountPath: /etc/payroll
```

##### Example Volume Definitions

For the M1.1 Search-and-Replace feature, the application will require two
`volume` definitions in its corresponding Deployment manifest:

- One `configMap` `volume` for providing the original config file
- One `emptyDir` `volume` for sharing the modified config file

For example:

```
      volumes:
      - name: payroll-config-orig
        configMap:
          name: payroll-ini-configmap
      - name: payroll-config
        emptyDir:
          medium: Memory
```

#### Using `sed` Substitution Syntax for Search-and-Replace Specifications

The M1.1 Search-and-Replace feature will support the `sed` substitution
command syntax for specifying search-and-replace operations to be performed
on a config file. This will allow each search-and-replace specification to be
conveniently defined with single substitute command string. This syntax also
lends itself to support of regex capture groups. Details are provided below.

#### Pod Annotations

The M1.1 Search-and-Replace feature is configured via Pod Annotations that are
applied to application Deployments. Pod Annotations are specified on a
per-secret-group basis as described in the
[M1 Solution Design Document](https://github.com/cyberark/secrets-provider-for-k8s/blob/main/design/m1_push_to_file_design.md).

The Pod Annotations that support configuration file search-and-replace 
functionality are as follows:

| Annotation                                     | Description | Details |
|------------------------------------------------|-------------|---------|
| conjur.org/conjur-secrets.{secret-group}       | List of secrets to be retrieved from Conjur. This Annotation is not new for this design, and is described in the M1 Solution Design Document. | See [M1 Supported Annotations](https://github.com/cyberark/secrets-provider-for-k8s/blob/main/design/m1_push_to_file_design.md#supported-annotations) |
| conjur.org/config-file-src-path.{secret-group} | Relative filepath for Secrets Provider to read the original configuration file. This filepath should be relative to the `volumeMount` point for the `configMap` volume used to expose the original config file. (See [Example Volume Mounts for the Secrets Provider](#example-volume-mounts-for-the-secrets-provider)) | See [Annotation for Configuring the Source Filepath](#annotation-for-configuring-the-source-filepath) |
| conjur.org/config-file-dest-path.{secret-group} | Relative filepath for the application container to read the modified configuration file. This filepath is relative to the application container's `volumeMount` point for the `emptyDir` volume used to share the modified config file. (See [Example Volume Mount for the Application Container](#example-volume-mount-for-the-application-container)). | See [Annotation for Configuring the Destination Filepath](#annotation-for-configuring-the-destination-filepath) |
| conjur.org/config-file-modifications.{secret-group} | List of search-and-replace operations to be performed on the config file. | See [Annotation for Configuring File Modifications](#annotation-for-configuring-file-modifications) |

#### Example Search-and-Replace Pod Annotations

For illustration purposes, here is an example of how the M1.1
Search-and-Replace  feature can be configured via Pod Annotations.

Let's suppose the configuration file to be modified is a simple `.ini`
file:

```
; last modified 1 April 2019
[owner]
name=John Doe
organization=Acme Widgets Inc.

[database]
server=192.0.2.62
port=143
file=payroll.dat
cert="<Database certificate goes here>"
```

Furthermore, let's suppose that we want the following search-and-replace
operations to be performed to customize this config file:

- Replace the last modified date with the current date.
- Replace the following fields with secrets retrieved from Conjur:
  - Owner name
  - Owner organization
  - Database server IP address
  - Database server port
  - (Future support) Database certificate (stored as a base64-encoded string
    on Conjur)

Here is an example of a Kubernetes Deployment patch YAML file that can
be used to configure the search-and-replace operations described above:

```
spec:
  template:
    metadata:
      annotations:
        conjur.org/conjur-secrets.payroll: |
          - owner-name: prod/payroll/owner/name
          - owner-org:  prod/payroll/owner/org
          - db-server:  prod/payroll/db/server
          - db-port:    prod/payroll/db/port
        conjur.org/config-file-src-path.payroll: payroll.ini
        conjur.org/config-file-dest-path.payroll: payroll.ini
        conjur.org/config-file-modifications.payroll: |
          - s/name=.*/name={{.owner-name}}/
          - s/organization=.*/organization={{.owner-org}}/
          - s/server=.*/server={{.db-server}}/
          - s/port=.*/name={{.db-port}}/
          - s/cert=.*/cert="{{.db-cert | b64dec}}"/
```

#### Annotation for Configuring the Source Filepath

The relative filepath from which the Secrets Provider should read the original
config file is configured using the following Pod Annotation (see example
above). This filepath should be relative to the `volumeMount` point for the
`configMap` volume used to expose the original config file. (See
[Example Volume Mounts for the Secrets Provider](#example-volume-mounts-for-the-secrets-provider)):

```
conjur.org/config-file-src-filepath.<secret-group>: |
  <config-file-content>
```

#### Annotation for Configuring the Destination Filepath

The relative filepath from which the application container should read the
modified config file is configured using the following Pod Annotation (see
example above). This filepath is relative to the application container's
`volumeMount` point for the `emptyDir` volume used to share the modified
config file. (See
[Example Volume Mount for the Application Container](#example-volume-mount-for-the-application-container)):

```
conjur.org/config-file-dest-filepath.<secret-group>: <dest-full-filepath>
```

#### Annotation for Configuring File Modifications

The search-and-replace operations to be performed on the baseline config
file are configured using the following Pod Annotation (see example above):

```
conjur.org/config-file-modifications.<secret-group>: |
  <list-of-sed-substitute-commands>
```

Each `sed` substitute command in the list is of the form:

```
s/<match-pattern>/<replacement-pattern>/
```

Some notes:

- The Secrets Provider is expected to iterate through this list,
  performing the search-and-replace operation specified on each entry in the
  list in sequence.
- Unless the `sed` global `g` flag is used, substitute commands will only
  modify the **first** occurrence of the matched pattern. If the global
  `g` flag is used, then all occurrences of the matched pattern will
  be modified.
- Logically speaking, the Secrets Provider executes each substitute command
  equivalent to running the following `sed` command: 

  ```
  sed -i <sed-substitute-command> <configFile>
  ```

  (Note that the implementation of this execution may or may not use execution
  of the actual `sed` command via `os.Exec()`. That is, the design may
  involve a pure Golang implementation for `sed`-like functionality.)
- As displayed in the examples above, the "replacement pattern" defined for
  each `sed` substitute command can make use of one or more "replacement
  templates" to indicate which and how Conjur secrets values should be
  retrieved and inserted. Replacement templates are described in more in the
  following section.
- At some point in the future, the `sed` substitute commands will support
  the use of regex capture groups. (See the
  [(FUTURE SUPPORT) Regex Capture Groups](#future-support-regex-capture-groups)
  section below.)

##### Replacement Templates

As described in the previous section, the replacement pattern defined for
each `sed` substitute command can make use of one or more "replacement
templates" to indicate which and how Conjur secrets values should be
retrieved and inserted. Replacement templates are represented as
Go templates.

Using Go template syntax for replacement templates keeps this syntax
consistent with the syntax used for the
[M1 custom template support](https://github.com/cyberark/secrets-provider-for-k8s/blob/main/design/m1_push_to_file_design.md#example-custom-templates),
and it allows the M1.1 Search-and-Replace implementation to reuse the
at least a portion of the Go template processing that was developed
for M1.

Examples of replacement templates include:

| Example | Description |
|---------|-------------|
| <code>{{date}}</code> | (FUTURE SUPPORT) Insert the current date |
| <code>{{.db-url}}</code> | Insert the value retrieved from Conjur for the secret with alias `db-url` |
| <code>{{.db-cert &#124; b64dec}}</code> | Insert the base64-decoded value retrieved from Conjur for the secret with alias `db-cert` |

Some notes:
- Tokens that begin with `.` represent the secret aliases for values retrieved
  from Conjur.
- In the future, replacement templates may also include support for
  [Go template functions](https://pkg.go.dev/text/template#hdr-Functions).
  The supported Go template functions may include:
  - Standard (built-in) Go template functions
  - A select subset of functions from the
    [Sprig template function library](http://masterminds.github.io/sprig/).
  - Custom functions created specifically for M1/M1.1 features

  Some examples of template functions that are under consideration:

  | Template Function | Description |
  |-------------------|-------------|
  | date | (FUTURE SUPPORT) Insert the current date |
  | b64enc | (FUTURE SUPPORT) Base64 encode the value before insertion |
  | b64dec | (FUTURE SUPPORT) Base64 decode the value before insertion |
  | quote | (FUTURE SUPPORT) Surround the value in quotes before insertion |
  | indent <N> | (FUTURE SUPPORT) Indent the line by N spaces before insertion |

##### Escaping/Un-Escaping any Pre-Existing Curly Braces in Baseline Config File

Since the search-and-replace syntax makes use of `{{` and `}}` double-curly
braces to signify a replacement template, care must be taken to preserve any
pre-existing curly braces in the original (baseline) config file. This can
be achieved by having the Secrets Provider perform the following
as part of its search-and-replace workflow:

- Escaping curly braces in the baseline file **BEFORE** executing all
  search-and-replace operations on the config file. This can be done by
  inserting a `\` backslash in front of each `{` or `}` curly brace.
- Un-escaping any escaped curly braces in the modified config file **AFTER**
  all replacement templates have been parsed and executed. This can be done by
  removing any `\` backslash characters that precede any `{` or
  `}` curly braces.

##### (FUTURE SUPPORT) Regex Capture Groups

In the future, the M1 Search-and-Replace feature may support the use of
regex capture groups to specify how specific fields that are matched
in the `sed` substitute command can be inserted into the config file.
See [this blog](https://www.ryanchapin.com/using-sed-with-regex-capture-groups/)
for an example of how regex capture groups can be used with `sed`.

#### Avoiding Leaking of Sensitive Secrets Data in Container Logs

When config files are rendered using secrets values that have been
retrieved from Conjur, we need to be careful that those sensitive secrets
values are not exposed in container logs as part of error messages
when rendering errors occur. This may require a test run of Go replacement
template execution using "dummy" Conjur secret values in order
to provide detailed error messages to the user while avoiding an leaking
of sensitive secret values.

### Performance Metrics

<TBD>

### Upgrading Existing Secrets Provider Deployments

<TBD>

### Project Scope and Limitations

<TBD>

#### Out of Scope

<TBD>

##  Design

### Flow Diagram

The flow diagram for the M1.1 Search-and-Replace feature is the same as it
was for the M1 Push-to-File design. For the M1.1 Search-and-Replace feature,
the "Pod Annotations from Manifest" source in this diagram may also include
the new Annotations described in the [Pod Annotations](#pod-annotations)
section above, including the Annotation that provides the Secrets Provider
with the baseline config file.

![Secrets Provider Push to File Flow Diagram](./m1_push_to_file_flow.png)

### Workflow

The workflow that the Secrets Provider will execute for the M1
Search-and-Replace feature is as follows:

- At startup, parse all Pod Annotations (same as in M1 design) 
- After all Annotations are parse, retrieve secrets from Conjur for
  each secrets group (same as in M1 design) 
- Iterate through the configuration for each secret group (same as in M1 design):
  - If the Annotations for this secret group include the source config file and
    the destination filepath for config file search-and-replace, then:
    - If the Annotations for this secret group include config file
      modifications, then:
      - Escape all pre-existing curly braces in the baseline config file by
        inserting a backslash in front of each curly brace.
      - Iterate through all entries in the modification list:
        - Perform the `sed` substitution.
      - After `sed` substitutions have been performed for all entries in
        the modification list, iterate through all replacement templates
        in the resulting modified config file:
        - Execute a "test run" of the Go replacement template execution
          using "dummy" secrets values, logging any template execution errors
          and exiting Secrets Provider execution if errors are encountered.
          (This "test run" allows for detailed errors to be displayed
          without risk of leaking sensitive secrets values.)
        - Execute the Go replacement template using actual secret values
          retrieved from Conjur, while suppressing raw template execution
          errors. If errors do occur, display a sanitized error message
          and exit Secrets Provider execution.
      - Un-escape any escaped curly braces (i.e. those with a preceding
        backslash) in the modified config file.
    - Write the modified (or unmodified, if modifications were not configured)
      config file to the destination filepath.

## Performance

<TBD>

## Backwards Compatibility

## Affected Components

<TBD>

## Test Plan

<TBD>

### Test environments

### Test assumptions

### Out of scope

### Test prerequisites

### Test Cases

#### Unit Tests

#### Error Handling / Recovery / Supportability Tests

#### E2E tests

#### Security testing

####  Performance testing

## Logs

<TBD>

## Documentation

<TBD>

## Version update

<TBD>

## Security

### Preventing Leaking of Sensitive Secrets Values in Error Logs

<TBD>

### Code Injection Threat Analysis

<TBD - Explain why this isn't possible, and how we'll test for this in UT>

## Audit

No changes to Conjur audit behavior are required for this feature.

##  Development Tasks

<TBD>

## Definition of Done

<TBD>

### Future Work

<TBD>

## Solution Review

<table>
<thead>
<tr class="header">
<th><strong>Persona</strong></th>
<th><strong>Name</strong></th>
<th><strong>Design Approval</strong></th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td>Team Leader</td>
<td>Dane LeBlanc</td>
<td><ul>
<p> </p>
</ul></td>
</tr>
<tr class="even">
<td>Product Owner</td>
<td>Alex Kalish</td>
<td><ul>
<p> </p>
</ul></td>
</tr>
<tr class="odd">
<td>System Architect</td>
<td>Rafi Schwarz</td>
<td><ul>
<p> </p>
</ul></td>
</tr>
<tr class="even">
<td>Security Architect</td>
<td>Andy Tinkham</td>
<td><ul>
<p> </p>
</ul></td>
</tr>
<tr class="odd">
<td>QA Architect</td>
<td>Andy Tinkham</td>
<td><ul>
<p> </p>
</ul></td>
</tr>
</tbody>
</table>

## Appendix