Simple config properties API for Kotlin

Nat Pryce

Last update: Dec 30, 2022

Related tags

Utility kotlin jvm configuration environment-variables properties command-line-parser

Overview

Konfig - A Type Safe Configuration API for Kotlin

Konfig provides an extensible, type-safe API for configuration properties gathered from multiple sources — built in resources, system properties, property files, environment variables, command-line arguments, etc.

A secondary goal of Konfig is to make configuration "self explanatory”.

Misconfiguration errors are reported with the location and “true name” of the badly configured property. E.g. a program may look up a key defined as Key("http.port", intType). At runtime, it will be parsed from an environment variable named HTTP_PORT. So the error message reports the name of the environment variable, so that the user can easily find and fix the error.

Configuration can be inspected and listed. For example, it can be exposed by HTTP to a network management system to help site reliability engineers understand the current configuration of a running application.

Getting Started

To get started, add com.natpryce:konfig:<version> as a dependency, import com.natpryce.konfig.* and then:

Define typed property keys

val server_port = Key("server.port", intType)
val server_host = Key("server.host", stringType)

Build a Configuration object that loads properties:

val config = systemProperties() overriding
             EnvironmentVariables() overriding
             ConfigurationProperties.fromFile(File("/etc/myservice.properties")) overriding
             ConfigurationProperties.fromResource("defaults.properties")

Define some properties. For example, in defaults.properties:
```
server.port=8080
server.host=0.0.0.0
```
Look up properties by key. They are returned as typed values, not strings, and so can be used directly:
```
val server = Server(config[server_port], config[server_host])
server.start()
```

Konfig can load properties from:

Java property files and resources
Java system properties
Environment variables
Hard-coded maps (with convenient syntax)
Command-line parameters (with long and short option syntax)

Konfig can easily be extended with new property types and sources of configuration data.

Konfig can report where configuration properties are searched for and where they were found.

Naming of Properties

Konfig's Configuration objects expect property names to follow Java property name conventions: dots to represent hierarchy, lower-case identifiers within the hierarchy, hyphens to separate words in those identifiers.

For example: servers.file-storage.s3-api-key, servers.file-storage.s3-bucket.

Each Configuration implementation maps from that naming convention to the convention used by the underlying configuration store. E.g. the EnvironmentVariables implementation maps Java property name convention to the upper-case-and-underscores convention used for Unix environment variables.

Configuration is an interface and Key is a data class. This makes it straight forward to write an implementation of Configuration that translates the names of keys to different naming conventions, if your configuration follows an unusual convention.

Reflectomagic key definition

Konfig has a few ways to reduce boilerplate code when defining configuration keys.

You can use Kotlin's delgated property protocol to name keys after the constants that hold them:

val host by stringType // defines a key named "host"
val port by intType    // defines a key named "port"

...

val client = TcpClient(configuration[host], configuration[port])

You can declare objects that extend PropertyGroup to define hierarchies of property keys that follow the Konfig naming conventions described above:

object server : PropertyGroup() {
    val base_uri by uriType   // defines a key named "server.base-uri"
    val api_key by stringType // defines a key named "server.api-key"
}

...

val client = HttpClient(configuration[server.base_uri], configuration[server.api_key])

Comments

Add option to exclude underscore replacement

Is it possible to add functionality to make this magic.kt test pass? I have some properties with key name '_' underscore.

From:

object a_group : PropertyGroup() { val a_property by stringType }

@Test fun underscore_not_replaced_with_hyphen() { assertThat(a_group.a_property.name, equalTo("a_group.a_property")) }

magic.kt:23

opened by dzastinas 3
Support different file formats for config

Thank you for your hard work!

I have a question about supported config file format. Will it be possible to use format other than java properties format (yml etc)?

opened by ayrat555 3
How do I read config props from resource/file only if it exists and otherwise ignore it?
val config = systemProperties() overriding EnvironmentVariables() overriding ConfigurationProperties.fromResource("local.properties")

Currently this fails when local.properties doesn't exist. Is there a way to let it ignore that file, if it's not there?
opened by deiga 3
Update gradle to a newer version (4.3) with kotlin DSL for gradle

Gradle has evolved quite a bit since this project got started. We now have kotlin as a first class language to write the configuration (yay strong types).

Also bump the version of kotlin, dokka and other dependencies.

opened by ColinHebert 2

[Suggestion] support of dynamic configuration

Currently Konfig only support static configuration. For example myproperty.value can be directly named and referenced. But say I want to generate some more dynamic elements such as

server.port=8080
server.host=0.0.0.0
server.modules=a, b
server.modules.a.initMessage=A Init
server.modules.a.stopMessage=A Stop
server.modules.b.initMessage=B Init
server.modules.b.stopMessage=B Stop

To begin with, let's assume we have a data class to support a "Module":

data class Module(val initMessage: String, val stopMessage: String)

We also have a property group:

object server : PropertyGroup() {
    val modules by listType(stringType)
}

val moduleIds : List<String> = config[server.modules]

Right now we can extract a list of strings, containing (a, b), it would be interesting to actually obtain the list of modules we're after or something we can work with.

Because the modules a and b are defined dynamically, we can't pre-bake them in our PropertyGroup

Adding a special helper, we can do a bit more:

typealias Extractor<V> = (String, String, Configuration) -> V
fun <V> Configuration.get(keys: Key<List<String>>, extractor: Extractor<V>): Map<String, V> {
    return this[keys]
            .map { it to extractor(keys.name, it, this) }
            .toMap()
}

This allows us to change our code a bit:

val moduleIds : List<String> = config[server.modules]
val mapModuleIdToInitMessage : Map<String, String> = config.get(server.modules){ prefix, id, config ->
  config[Key("$prefix.$id.initMessage", stringType)]
}

We now have a very basic map ("a" to "A Init", "b" to "B Init")

We could do the same thing to obtain the actual module object:

val moduleIds : List<String> = config[server.modules]
val mapModules : Map<String, Module> = config.get(server.modules){ prefix, id, config ->
  Module(config[Key("$prefix.$id.initMessage", stringType)], config[Key("$prefix.$id.stopMessage", stringType)])
}

we now have a map ("a" to ("A Init", "A Stop"), "b" to ("B Init", "B Stop")). We can extract the values if we so desire mapModules.values, etc.

While this works, I'm not very fond of having the "stopMessage" and "initMessage" extraction being done without involving the PropertyGroup, to improve that we could add a few helpers in the PropertyGroup:

object server : PropertyGroup() {
    val modules by listType(stringType)

    fun modulesInitMessage(id: String) = key("${server::modules.name}.$id.initMessage", stringType)
    fun modulesStopMessage(id: String) = key("${server::modules.name}.$id.stopMessage", stringType)
}

Now we can use moduleInitMessage("a") to obtain dynamically a key for the initMessage of module a.

Which means that we can slightly alter our call:

val mapModules : Map<String, Module> = config.get(server.modules){ _, id, config ->
  Module(config[server.modulesInitMessage(id)], config[server.modulesStopMessage(id)])
}

This feels much cleaner already.

We could argue that because the new get method is on config, we could forgo the entire config passing to the extractor. And potentially enforce a usage pattern by not providing the prefix either.

It could get as simple as:

data class Module(val initMessage: String, val stopMessage: String)

object server : PropertyGroup() {
    val modules by listType(stringType)

    fun modulesInitMessage(id: String) = key("${server::modules.name}.$id.initMessage", stringType)
    fun modulesStopMessage(id: String) = key("${server::modules.name}.$id.stopMessage", stringType)
}

val mapModules : Map<String, Module> = config.get(server.modules){
    Module(config[server.modulesInitMessage(it)], config[server.modulesStopMessage(it)])
}

What do you think? Is it worth supporting this?

opened by ColinHebert 2

Property group with underscore in name doesn't work

import com.natpryce.konfig.*
import org.junit.Test

class KonfigUnderscoreTest {

    object db_server : PropertyGroup() {
        val port by intType
        val host by stringType
    }

    val config = ConfigurationProperties.fromResource("db_server.properties")

    val port = config[db_server.port]
    val host = config[db_server.host]

    @Test
    fun konfig_test(){
        println("$host:$port")
    }

}

com.natpryce.konfig.Misconfiguration: db-server.port property not found; searched:

db-server.port in resource db_server.properties

It seems to silently convert underscores to dashes somewhere.

wontfix

opened by darkpaw 2

java.lang.IllegalArgumentException when accessing configuration properties
Hi,

the stacktrace is:

Exception in thread "main" java.lang.IllegalArgumentException: Parameter specified as non-null is null: method kotlin.jvm.JvmClassMappingKt.getKotlinClass, parameter $receiver at kotlin.jvm.JvmClassMappingKt.getKotlinClass(JvmClassMapping.kt) at com.natpryce.konfig.PropertyGroup.outer(magic.kt:16) at com.natpryce.konfig.PropertyGroup.namePrefix(magic.kt:18) at com.natpryce.konfig.PropertyGroup.name(magic.kt:17) at com.natpryce.konfig.PropertyGroup.key(magic.kt:23) at com.natpryce.konfig.MagicKt.getValue(magic.kt:28)

I took a look at the code and there is a silent requirement for the configuration class to be inside another class:

open class PropertyGroup(private val outer: PropertyGroup? = null) : PropertyKeys() { private fun outer() = outer ?: javaClass.enclosingClass.kotlin.objectInstance as? PropertyGroup

If the configuration object is a top level class in a separate file, javaClass.enclosingClass is null. This is the case in all the tests, btw, that the config object is inside an other class (eg.: https://github.com/npryce/konfig/blob/master/src/test/kotlin/com/natpryce/konfig/magic_tests.kt#L19) However it's not obvious from the README.md that it is a requirement.

I'm wondering how to fix this - I guess it should not be necessary to require a top level class, however I might be missing something.

Either the documentation should be updated or the code :)
opened by Kornel 2

Packages and file facades are not yet supported in Kotlin reflection

Using:

fun main(args: Array<String>) {
    val p = object : PropertyGroup(){
        val aa by intType
    }

    val config = ConfigurationProperties.fromResource("test.props")
    println(config[p.aa])
}

And test.props: aa=3

I get:

Exception in thread "main" java.lang.UnsupportedOperationException: Packages and file facades are not yet supported in Kotlin reflection. Meanwhile please use Java reflection to inspect this class: class ConfigTestKt
    at kotlin.reflect.jvm.internal.KClassImpl.reportUnresolvedClass(KClassImpl.kt:170)
    at kotlin.reflect.jvm.internal.KClassImpl.access$reportUnresolvedClass(KClassImpl.kt:38)
    at kotlin.reflect.jvm.internal.KClassImpl$descriptor_$1.invoke(KClassImpl.kt:46)
    at kotlin.reflect.jvm.internal.KClassImpl$descriptor_$1.invoke(KClassImpl.kt:38)
    at kotlin.reflect.jvm.internal.ReflectProperties$LazySoftVal.invoke(ReflectProperties.java:93)
    at kotlin.reflect.jvm.internal.KClassImpl.getDescriptor(KClassImpl.kt:50)
    at kotlin.reflect.jvm.internal.KClassImpl$objectInstance_$1.invoke(KClassImpl.kt:136)
    at kotlin.reflect.jvm.internal.ReflectProperties$LazyVal.invoke(ReflectProperties.java:63)
    at kotlin.reflect.jvm.internal.KClassImpl.getObjectInstance(KClassImpl.kt:149)
    at com.natpryce.konfig.PropertyGroup.outer(magic.kt:16)
    at com.natpryce.konfig.PropertyGroup.namePrefix(magic.kt:18)
    at com.natpryce.konfig.PropertyGroup.name(magic.kt:17)
    at com.natpryce.konfig.PropertyGroup.key(magic.kt:23)
    at com.natpryce.konfig.MagicKt.getValue(magic.kt:28)
    at ConfigTestKt$main$p$1.getAa(ConfigTest.kt)
    at ConfigTestKt.main(ConfigTest.kt:12)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)

This is with Kotlin 1.0.2-release-IJ143-96.

opened by jonyt 2

add `default` field into `CommandLineOption`

Please add default field into CommandLineOption:

` val cmdName = CommandLineOption(Key("name", stringType), "name", "name", "DefaultName", "Description of NAME")

`

And if use -h command it will print:

Description of NAME. Default value: DefaultName

And if not setted in configuration - use Deafult Value

opened by icreator 1
get list of all available keys

Is it possible to be able to get a list of every possible key in a configuration? At the moment it's all gets/contains etc. For background, what we want to do is basically run a test that ensures the keys are consistent/available across all envs, as otherwise we don't know till we start the app if the config files have all the params (we had the issue recently).

Thanks :)

opened by godspeed20 1
seems to have issue with _
For some reason when I specify a name with _ in it, it converts it to -

object controlScript : PropertyGroup() { val APPDYNAMICS_AGENT_UNIQUE_HOST_ID by stringType }

println(config[controlScript.APPDYNAMICS_AGENT_UNIQUE_HOST_ID] )

Exception in thread "main" com.natpryce.konfig.Misconfiguration: controlScript.APPDYNAMICS-AGENT-UNIQUE-HOST-ID property not found; searched:

controlScript.APPDYNAMICS-AGENT-UNIQUE-HOST-ID in /Users/laipt/Desktop/petels/POC.cicd/cicd/src/main/resources/application-dev.properties
opened by peterlai-roboops 1

add `default` field into `Key`

Please add default field into Key:

    val nameKey = Key("name", stringType, "DefaultName")
    val name = config[nameKey] /// if not contains in CONFIG -> ="DefaultName"

If use -h command it will print:

Description of NAME. Default value: DefaultName

And if not setted in configuration - use Deafult Value

instead I do now:

    val connectiontimeoutKey = Key("connection.timeout", intType)
    fun getConnectionTimeout(): Int {
        return if (connectiontimeoutKey in vals) vals[connectiontimeoutKey]
        else DEFAULT_CONNECTION_TIMEOUT
    }

but I need:

    val connectiontimeoutKey = Key("connection.timeout", intType, DEFAULT_CONNECTION_TIMEOUT)
    fun getConnectionTimeout(): Int {
        return vals[connectiontimeoutKey]
    }

opened by icreator 1

Use Thread.currentThread().contextClassLoader instead of ClassLoader.…

…getSystemClassLoader()

More context: https://bitbucket.org/asomov/snakeyaml/issues/318 I have yet another use case but it is not easy to explain it since it is no open source.

The new way should improve the way to find *.properties in the classpath. It is fully backwards compatible.

opened by asomov 0
Consider use of `typeOf`

Kotlin 1.3.40 added typeOf to the standard library, which may simplify code. However, it is marked as "experimental". Reading the tea leaves, Kotlin plans to make this feature standard (non-experimental) at some point, so I presume there are some multi-platform issues for the present. My focus is JVM, so I'm satisfied with how typeOf works.

https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.reflect/type-of.html

konfig might be simplified/improved with use of typeOf.

opened by binkley 0
Possible type mismatch in property_types.kt
I cloned your repo and opened it in my IDE. An error popped out on line 106 about a type mismatch

Type mismatch. Required: ParseResult<T> Found: ParseResult<out T>

I don't know if this is important because I've just started to learn Kotlin. I am running IntelliJ IDEA 2019.2.3 (Community Edition) Java 1.8.0_121
opened by hanzo2001 0

Owner

Nat Pryce

GitHub

Expirable Disk Lru Cache is a secure(with encryption) wrapper for [DiskLruCache](https://github.com/JakeWharton/DiskLruCache) that allows expiring of key/value pairs by specifying evictionTimeSpan. It has very simple API.

ExpirableDiskLruCache ExpirableDiskLruCache is a wrapper for DiskLruCache that allows expiring of key/value pairs by specifying evictionTimeSpan. It h