Konf

A type-safe cascading configuration library for Kotlin/Java/Android, supporting most configuration formats.

Features

• Type-safe. Get/set value in config with type-safe APIs.
• Thread-safe. All APIs for config is thread-safe.
• Batteries included. Support sources from JSON, XML, YAML, HOCON, TOML, properties, map, command line and system environment out of box.
• Cascading. Config can fork from another config by adding a new layer on it. Each layer of config can be updated independently. This feature is powerful enough to support complicated situation such as configs with different values share common fallback config, which is automatically updated when configuration file changes.
• Self-documenting. Document config item with type, default value and description when declaring.
• Extensible. Easy to customize new sources for config or expose items in config.

Contents

• Konf
  • Features
  • Contents
  • Prerequisites
  • Use in your projects
    • Maven
    • Gradle
    • Gradle Kotlin DSL
    • Maven (master snapshot)
    • Gradle (master snapshot)
    • Gradle Kotlin DSL (master snapshot)
  • Quick start
  • Define items
  • Use config
    • Create config
    • Add config spec
    • Retrieve value from config
    • Cast config to value
    • Check whether an item exists in config or not
    • Modify value in config
    • Subscribe the update event
    • Export value in config as property
    • Fork from another config
  • Load values from source
    • Subscribe the update event for load operation
    • Strict parsing when loading
    • Path substitution
  • Prefix/Merge operations for source/config/config spec
  • Export/Reload values in config
  • Supported item types
  • Optional features
  • Build from source
  • Breaking Changes
    • v0.19.0
    • v0.17.0
    • v0.15
    • v0.10
• License

Prerequisites

• JDK 8 or higher
• tested on Android SDK 23 or higher

Use in your projects

This library has been published to Maven Central and JitPack.

Konf is modular, you can use different modules for different sources:

• konf-core: for built-in sources (JSON, properties, map, command line and system environment)
• konf-hocon: for built-in + HOCON sources
• konf-toml: for built-in + TOML sources
• konf-xml: for built-in + XML sources
• konf-yaml: for built-in + YAML sources
• konf-git: for built-in + Git sources
• konf: for all sources mentioned above
• konf-js: for built-in + JavaScript (use GraalVM JavaScript) sources

Maven

<dependency>
  <groupId>com.uchuhimo</groupId>
  <artifactId>konf</artifactId>
  <version>1.1.2</version>
</dependency>

Gradle

compile 'com.uchuhimo:konf:1.1.2'

Gradle Kotlin DSL

compile(group = "com.uchuhimo", name = "konf", version = "1.1.2")

Maven (master snapshot)

Add JitPack repository to <repositories> section:

<repository>
    <id>jitpack.io</id>
    <url>https://jitpack.io</url>
</repository>

Add dependencies:

<dependency>
    <groupId>com.github.uchuhimo</groupId>
    <artifactId>konf</artifactId>
    <version>master-SNAPSHOT</version>
</dependency>

Gradle (master snapshot)

Add JitPack repository:

repositories {
    maven { url 'https://jitpack.io' }
}

Add dependencies:

compile 'com.github.uchuhimo.konf:konf:master-SNAPSHOT'

Gradle Kotlin DSL (master snapshot)

Add JitPack repository:

repositories {
    maven(url = "https://jitpack.io")
}

Add dependencies:

compile(group = "com.github.uchuhimo.konf", name = "konf", version = "master-SNAPSHOT")

Quick start

1. Define items in config spec:

   object ServerSpec : ConfigSpec() {
       val host by optional("0.0.0.0")
       val tcpPort by required<Int>()
   }

2. Construct config with items in config spec and values from multiple sources:

   val config = Config { addSpec(ServerSpec) }
           .from.yaml.file("server.yml")
           .from.json.resource("server.json")
           .from.env()
           .from.systemProperties()

   or:

   val config = Config { addSpec(ServerSpec) }.withSource(
       Source.from.yaml.file("server.yml") +
       Source.from.json.resource("server.json") +
       Source.from.env() +
       Source.from.systemProperties()
   )

   This config contains all items defined in ServerSpec, and load values from 4 different sources. Values in resource file server.json will override those in file server.yml, values from system environment will override those in server.json, and values from system properties will override those from system environment.

   If you want to watch file server.yml and reload values when file content is changed, you can use watchFile instead of file:

   val config = Config { addSpec(ServerSpec) }
           .from.yaml.watchFile("server.yml")
           .from.json.resource("server.json")
           .from.env()
           .from.systemProperties()

3. Define values in source. You can define in any of these sources:

   • in server.yml:

     server:
         host: 0.0.0.0
         tcp_port: 8080

   • in server.json:

     {
         "server": {
             "host": "0.0.0.0",
             "tcp_port": 8080
         }
     }

   • in system environment:

     SERVER_HOST=0.0.0.0
     SERVER_TCPPORT=8080

   • in command line for system properties:

     -Dserver.host=0.0.0.0 -Dserver.tcp_port=8080

4. Retrieve values from config with type-safe APIs:

   data class Server(val host: String, val tcpPort: Int) {
       fun start() {}
   }
   
   val server = Server(config[ServerSpec.host], config[ServerSpec.tcpPort])
   server.start()

5. Retrieve values from multiple sources without using config spec:

   val server = Config()
           .from.yaml.file("server.yml")
           .from.json.resource("server.json")
           .from.env()
           .from.systemProperties()
           .at("server")
           .toValue<Server>()
   server.start()

Define items

Config items is declared in config spec, added to config by Config#addSpec. All items in same config spec have same prefix. Define a config spec with prefix local.server:

object ServerSpec : ConfigSpec("server") {
}

If the config spec is binding with single class, you can declare config spec as companion object of the class:

class Server {
    companion object : ConfigSpec("server") {
        val host by optional("0.0.0.0")
        val tcpPort by required<Int>()
    }
}

The config spec prefix can be automatically inferred from the class name, leading to further simplification like:

object ServerSpec : ConfigSpec() {
}

or

class Server {
    companion object : ConfigSpec() {
    }
}

Here are some examples showing the inference convention: Uppercase to uppercase, lowercase to lowercase, SuffixSpec to suffix, TCPService to tcpService.

The config spec can also be nested. For example, the path of Service.Backend.Login.user in the following example will be inferred as "service.backend.login.user":

object Service : ConfigSpec() {
    object Backend : ConfigSpec() {
        object Login : ConfigSpec() {
            val user by optional("admin")
        }
    }
}

There are three kinds of item:

• Required item. Required item doesn't have default value, thus must be set with value before retrieved in config. Define a required item with description:

  val tcpPort by required<Int>(description = "port of server")

  Or omit the description:

  val tcpPort by required<Int>()

• Optional item. Optional item has default value, thus can be safely retrieved before setting. Define an optional item:

  val host by optional("0.0.0.0", description = "host IP of server")

  Description can be omitted.
• Lazy item. Lazy item also has default value, however, the default value is not a constant, it is evaluated from thunk every time when retrieved. Define a lazy item:

  val nextPort by lazy { config -> config[tcpPort] + 1 }

You can also define config spec in Java, with a more verbose API (compared to Kotlin version in "quick start"):

public class ServerSpec {
  public static final ConfigSpec spec = new ConfigSpec("server");

  public static final OptionalItem<String> host =
      new OptionalItem<String>(spec, "host", "0.0.0.0") {};

  public static final RequiredItem<Integer> tcpPort = new RequiredItem<Integer>(spec, "tcpPort") {};
}

Notice that the {} part in item declaration is necessary to avoid type erasure of item's type information.

Use config

Create config

Create an empty new config:

val config = Config()

Or an new config with some initial actions:

val config = Config { addSpec(Server) }

Add config spec

Add multiple config specs into config:

config.addSpec(Server)
config.addSpec(Client)

Retrieve value from config

Retrieve associated value with item (type-safe API):

val host = config[Server.host]

Retrieve associated value with item name (unsafe API):

val host = config.get<String>("server.host")

or:

val host = config<String>("server.host")

Cast config to value

Cast config to a value with the target type:

val server = config.toValue<Server>()

Check whether an item exists in config or not

Check whether an item exists in config or not:

config.contains(Server.host)
// or
Server.host in config

Check whether an item name exists in config or not:

config.contains("server.host")
// or
"server.host" in config

Check whether all values of required items exist in config or not:

config.containsRequired()

Throw exception if some required items in config don't have values:

config.validateRequired()

Modify value in config

Associate item with value (type-safe API):

config[Server.tcpPort] = 80

Find item with specified name, and associate it with value (unsafe API):

config["server.tcpPort"] = 80

Discard associated value of item:

config.unset(Server.tcpPort)

Discard associated value of item with specified name:

config.unset("server.tcpPort")

Associate item with lazy thunk (type-safe API):

config.lazySet(Server.tcpPort) { it[basePort] + 1 }

Find item with specified name, and associate it with lazy thunk (unsafe API):

config.lazySet("server.tcpPort") { it[basePort] + 1 }

Subscribe the update event

Subscribe the update event of an item:

val handler = Server.host.onSet { value -> println("the host has changed to $value") }

Subscribe the update event before every set operation:

val handler = Server.host.beforeSet { config, value -> println("the host will change to $value") }

or

val handler = config.beforeSet { item, value -> println("${item.name} will change to $value") }

Subscribe the update event after every set operation:

val handler = Server.host.afterSet { config, value -> println("the host has changed to $value") }

or

val handler = config.afterSet { item, value -> println("${item.name} has changed to $value") }

Cancel the subscription:

handler.cancel()

Export value in config as property

Export a read-write property from value in config:

var port by config.property(Server.tcpPort)
port = 9090
check(port == 9090)

Export a read-only property from value in config:

val port by config.property(Server.tcpPort)
check(port == 9090)

Fork from another config

val config = Config { addSpec(Server) }
config[Server.tcpPort] = 1000
// fork from parent config
val childConfig = config.withLayer("child")
// child config inherit values from parent config
check(childConfig[Server.tcpPort] == 1000)
// modifications in parent config affect values in child config
config[Server.tcpPort] = 2000
check(config[Server.tcpPort] == 2000)
check(childConfig[Server.tcpPort] == 2000)
// modifications in child config don't affect values in parent config
childConfig[Server.tcpPort] = 3000
check(config[Server.tcpPort] == 2000)
check(childConfig[Server.tcpPort] == 3000)

Load values from source

Use from to load values from source doesn't affect values in config, it will return a new child config by loading all values into new layer in child config:

val config = Config { addSpec(Server) }
// values in source is loaded into new layer in child config
val childConfig = config.from.env()
check(childConfig.parent === config)

All out-of-box supported sources are declared in DefaultLoaders, shown below (the corresponding config spec for these samples is ConfigForLoad):

These sources can also be manually created using their provider, and then loaded into config by config.withSource(source).

All from APIs also have their standalone version that return sources without loading them into the config, shown below:

Format of system properties source is same with that of properties source. System environment source follows the same mapping convention with properties source, but with the following name convention:

• All letters in name are in uppercase
• . in name is replaced with _

HOCON/JSON/properties/TOML/XML/YAML/JavaScript source can be loaded from a variety of input format. Use properties source as example:

• From file: config.from.properties.file("/path/to/file")
• From watched file: config.from.properties.watchFile("/path/to/file", 100, TimeUnit.MILLISECONDS)
  • You can re-trigger the setup process every time the updated file is loaded by watchFile("/path/to/file") { config, source -> setup(config) }
• From string: config.from.properties.string("server.port = 8080")
• From URL: config.from.properties.url("http://localhost:8080/source.properties")
• From watched URL: config.from.properties.watchUrl("http://localhost:8080/source.properties", 1, TimeUnit.MINUTES)
  • You can re-trigger the setup process every time the URL is loaded by watchUrl("http://localhost:8080/source.properties") { config, source -> setup(config) }
• From Git repository: config.from.properties.git("https://github.com/uchuhimo/konf.git", "/path/to/source.properties", branch = "dev")
• From watched Git repository: config.from.properties.watchGit("https://github.com/uchuhimo/konf.git", "/path/to/source.properties", period = 1, unit = TimeUnit.MINUTES)
  • You can re-trigger the setup process every time the Git file is loaded by watchGit("https://github.com/uchuhimo/konf.git", "/path/to/source.properties") { config, source -> setup(config) }
• From resource: config.from.properties.resource("source.properties")
• From reader: config.from.properties.reader(reader)
• From input stream: config.from.properties.inputStream(inputStream)
• From byte array: config.from.properties.bytes(bytes)
• From portion of byte array: config.from.properties.bytes(bytes, 1, 12)

If source is from file, file extension can be auto detected. Thus, you can use config.from.file("/path/to/source.json") instead of config.from.json.file("/path/to/source.json"), or use config.from.watchFile("/path/to/source.json") instead of config.from.json.watchFile("/path/to/source.json"). Source from URL also support auto-detected extension (use config.from.url or config.from.watchUrl). The following file extensions can be supported:

You can also implement Source to customize your new source, which can be loaded into config by config.withSource(source).

Subscribe the update event for load operation

Subscribe the update event before every load operation:

val handler = config.beforeLoad { source -> println("$source will be loaded") }

You can re-trigger the setup process by subscribing the update event after every load operation:

val handler = config.afterLoad { source -> setup(config) }

Cancel the subscription:

handler.cancel()

Strict parsing when loading

By default, Konf extracts desired paths from sources and ignores other unknown paths in sources. If you want Konf to throws exception when unknown paths are found, you can enable FAIL_ON_UNKNOWN_PATH feature:

config.enable(Feature.FAIL_ON_UNKNOWN_PATH)
    .from.properties.file("server.properties")
    .from.json.resource("server.json")

Then config will validate paths from both the properties file and the JSON resource. Furthermore, If you want to validate the properties file only, you can use:

config.from.enable(Feature.FAIL_ON_UNKNOWN_PATH).properties.file("/path/to/file")
    .from.json.resource("server.json")

Path substitution

Path substitution is a feature that path references in source will be substituted by their values.

Path substitution rules are shown below:

• Only quoted string value will be substituted. It means that Konf's path substitutions will not conflict with HOCON's substitutions.
• The definition of a path variable is ${path}, e.g., ${java.version}.
• The path variable is resolved in the context of the current source.
• If the string value only contains the path variable, it will be replaced by the whole sub-tree in the path; otherwise, it will be replaced by the string value in the path.
• Use ${path:-default} to provide a default value when the path is unresolved, e.g., ${java.version:-8}.
• Use $${path} to escape the path variable, e.g., $${java.version} will be resolved to ${java.version} instead of the value in java.version.
• Path substitution works in a recursive way, so nested path variables like ${jre-${java.specification.version}} are allowed.
• Konf also supports all key prefix of StringSubstitutor's default interpolator.

By default, Konf will perform path substitution for every source (except system environment source) when loading them into the config. You can disable this behaviour by using config.disable(Feature.SUBSTITUTE_SOURCE_BEFORE_LOADED) for the config or source.disabled(Feature.SUBSTITUTE_SOURCE_BEFORE_LOADED) for a single source.

By default, Konf will throw exception when some path variables are unresolved. You can use source.substituted(false) manually to ignore these unresolved variables.

To resolve path variables refer to other sources, you can merge these sources before loading them into the config. For example, if we have two sources source1.json and source2.properties, source1.json is:

{ 
  "base" : { "user" : "konf" , "password" : "passwd" }
}

source2.properties is:

connection.jdbc=mysql://${base.user}:${base.password}@server:port

use:

config.withSource(
    Source.from.file("source1.json") +
    Source.from.file("source2.properties")
)

We can resolve mysql://${base.user}:${base.password}@server:port as mysql://konf:passwd@server:port.

Prefix/Merge operations for source/config/config spec

All of source/config/config spec support add prefix operation, remove prefix operation and merge operation as shown below:

Export/Reload values in config

Export all values in config as a tree:

val tree = config.toTree()

Export all values in config to map in key-value format:

val map = config.toMap()

Export all values in config to hierarchical map:

val map = config.toHierarchicalMap()

Export all values in config to map in flat format:

val map = config.toFlatMap()

Export all values in config to JSON:

val file = createTempFile(suffix = ".json")
config.toJson.toFile(file)

Reload values from JSON:

val newConfig = Config {
    addSpec(Server)
}.from.json.file(file)
check(config == newConfig)

Config can be saved to a variety of output format in HOCON/JSON/properties/TOML/XML/YAML/JavaScript. Use JSON as example:

• To file: config.toJson.toFile("/path/to/file")
• To string: config.toJson.toText()
• To writer: config.toJson.toWriter(writer)
• To output stream: config.toJson.toOutputStream(outputStream)
• To byte array: config.toJson.toBytes()

You can also implement Writer to customize your new writer (see JsonWriter for how to integrate your writer with config).

Supported item types

Supported item types include:

• All primitive types
• All primitive array types
• BigInteger
• BigDecimal
• String
• Date and Time
  • java.util.Date
  • OffsetTime
  • OffsetDateTime
  • ZonedDateTime
  • LocalDate
  • LocalTime
  • LocalDateTime
  • Year
  • YearMonth
  • Instant
  • Duration
• SizeInBytes
• Enum
• Array
• Collection
  • List
  • Set
  • SortedSet
  • Map
  • SortedMap
• Kotlin Built-in classes
  • Pair
  • Triple
  • IntRange
  • CharRange
  • LongRange
• Data classes
• POJOs supported by Jackson core modules

Konf supports size in bytes format described in HOCON document with class SizeInBytes.

Konf supports both ISO-8601 duration format and HOCON duration format for Duration.

Konf uses Jackson to support Kotlin Built-in classes, Data classes and POJOs. You can use config.mapper to access ObjectMapper instance used by config, and configure it to support more types from third-party Jackson modules. Default modules registered by Konf include:

• Jackson core modules
• JavaTimeModule in jackson-modules-java8
• jackson-module-kotlin

Optional features

There are some optional features that you can enable/disable in the config scope or the source scope by Config#enable(Feature)/Config#disable(Feature) or Source#enabled(Feature)/Source#disable(Feature). You can use Config#isEnabled() or Source#isEnabled() to check whether a feature is enabled.

These features include:

• FAIL_ON_UNKNOWN_PATH: feature that determines what happens when unknown paths appear in the source. If enabled, an exception is thrown when loading from the source to indicate it contains unknown paths. This feature is disabled by default.
• LOAD_KEYS_CASE_INSENSITIVELY: feature that determines whether loading keys from sources case-insensitively. This feature is disabled by default except for system environment.
• LOAD_KEYS_AS_LITTLE_CAMEL_CASE: feature that determines whether loading keys from sources as little camel case. This feature is enabled by default.
• OPTIONAL_SOURCE_BY_DEFAULT: feature that determines whether sources are optional by default. This feature is disabled by default.
• SUBSTITUTE_SOURCE_BEFORE_LOADED: feature that determines whether sources should be substituted before loaded into config. This feature is enabled by default.

Build from source

Build library with Gradle using the following command:

./gradlew clean assemble

Test library with Gradle using the following command:

./gradlew clean test

Since Gradle has excellent incremental build support, you can usually omit executing the clean task.

Install library in a local Maven repository for consumption in other projects via the following command:

./gradlew clean install

Breaking Changes

v0.19.0

Since all sources are substituted before loaded into config by default, all path variables will be substituted now. You can use config.disable(Feature.SUBSTITUTE_SOURCE_BEFORE_LOADED) to disable this change.

v0.17.0

After migrated to tree-based source APIs, many deprecated APIs are removed, including:

• Source: all isXXX and toXXX APIs
• Config: layer, addSource and withSourceFrom

v0.15

After modularized Konf, hocon/toml/xml/yaml/git/watchGit in DefaultLoaders become extension properties/functions and should be imported explicitly. For example, you should import com.uchuhimo.konf.source.hocon before using config.from.hocon; in Java, config.from().hocon is unavailable, please use config.from().source(HoconProvider.INSTANCE) instead.

If you use JitPack, you should use com.github.uchuhimo.konf:konf:<version> instead of com.github.uchuhimo:konf:<version> now.

v0.10

APIs in ConfigSpec have been updated to support item name's auto-detection, please migrate to new APIs. Here are some examples:

• val host = optional("host", "0.0.0.0") to val host by optional("0.0.0.0")
• val port = required<Int>("port") to val port by required<Int>()
• val nextPort = lazy("nextPort") { config -> config[port] + 1 } to val nextPort by lazy { config -> config[port] + 1 }

License

© uchuhimo, 2017-2019. Licensed under an Apache 2.0 license.