deeplearning4j/deeplearning4j

# Configuration Objects

## Status

ACCEPTED

Discussed by Alex Black, raver119 and Paul Dubs on 27. November 2019.

## Context
Some Ops (esp. convolution) have many parameters. Many of them can have reasonable defaults, but even then creating
signatures for evey reasonable configuration may be impossible, as those signatures would require different naming in
order to be actually distinguishable from each other.

In other cases, an op may have a lot of same typed parameters that are required (e.g. GRU, LSTM, SRU) but it is very
easy to mix them up. 

For both of those cases (many optional parameters, easily mixed up required parameters) it is reasonable to use a
config holder with builder pattern in languages that do not support named or default parameters. 

In our current codebase those configurations are often used across several related ops. 


## Decision
We add a `Config("name"){ ... }` section to the namespace context. It supports `Input` and `Arg` definitions in the same
way that `Op` does.

Ops that want to use that config can use `useConfig(conf)`. As configs are often reused across related objects, this
will have the effect of a mixin: All inputs and args defined in that config will also be automatically defined on that
Op. If there is a naming conflict, an exception will be thrown at construction time.

For default signatures, configs will be passed at the end, in the order that they were added to the Op.

If other signatures are desired, configs, like regular inputs and args, can be passed to `Signature`.

In languages that do not support default or named parameters, a config holder will be created, that will take the
parameters of the config using a builder pattern. For languages with default and named parameters, no additional config
holder will be created, and the parameters of the config will be treated as if they were directly configured on the Op.

### Example
This example shows a very simple case in order to highlight how this feature would be used. 
```kotlin
fun RNN() = Namespace("RNN"){
    val sruWeights = Config("SRUWeights"){
        Input(FLOATING_POINT, "weights"){ description = "Weights, with shape [inSize, 3*inSize]" }
        Input(FLOATING_POINT, "bias"){ description = "Biases, with shape [2*inSize]" }
    }

    Op("SRU"){
        Input(FLOATING_POINT, "x"){ description = "..." }
        Input(FLOATING_POINT, "initialC"){ description = "..." }
        Input(FLOATING_POINT, "mask"){ description = "..." }
        
        useConfig(sruWeights)
    
        Output(FLOATING_POINT, "out"){ description = "..." }
    }
    
    Op("SRUCell"){
        val x = Input(FLOATING_POINT, "x"){ description = "..." }
        val cLast = Input(FLOATING_POINT, "cLast"){ description = "..." }
        
        val conf = useConfig(sruWeights)
    
        Output(FLOATING_POINT, "out"){ description = "..." }
    
        // Just for demonstration purposes
        Signature(x, cLast, conf)
    }
}
```
 
## Consequences

### Advantages
* Ops that share parameters can make that sharing explicit
* Easier definition of related ops with common parameters
* Simplifies usage of complex ops in languages without named parameters

### Disadvantages
* Not all parameters are defined directly within an op anymore
* Configs are not shareable across namespaces