Problem

Currently the only way to get IDE and build tools support is via JUnitKSpecRunner. This approach is very limited as you cant run a specific spec.

Possible solutions

1. JUnit 5 looks promising, especially the test-engine-api and the launcher. If all goes well with it, we don't even need to write any plugins (someone may already did).
2. Write a custom test engine for KSpec. Currently exploring it, you can checkout this branch.

Introduce ContextVisitResult that control the tree traversal. There are still some checks missing like when preVisitExampleGroup returns SKIP and handling for the return value of postVisitExampleGroup.

Resolves #32.

Broken down kspec-core into several modules.

• kspec-core - core classes.
• kspec-engine - defines the runtime behaviour of KSpec.
• kspec-launcher - provides infrastructure in writing IDE and build tool plugins.
• kspec-console-runner - console runner for KSpec (uses kspec-launcher).

This resolves #19, resolves #21 and bring a lot of changes internally. Example groups can now be tagged and its children will automatically inherit those tags. before, after and around hooks now gets called for every example group as well.

Resolves #14

Added filtering via tags when running examples.

config.filter.include(tag1, tag2) // run only examples with tag1 or tag2
config.filter.exclude(tag1, tag2) // exclude examples with either tag1 or tag2

config.filter.matching(tag1, ...tagn) // same as filter.include but will only apply if
                                      // any example matches the given tags

focus() tag is provided and is the default value for config.filter.matching. To focus an example tag it with focus() or use fit(...) instead of it(...).

Current runner extends ParentRunner, which treats the class containing the @RunWith annotation to be the root test suite. As an improvement we can make every Given clause as the root test suite.

e.g.

class TestSpec: Speck({
    Given("this foo") {
        When("this bar") {
            Then("foobar") { 
                ...
            }
        }
    }
})

Current runner will have this hierarchy:

TestSpec
  --> Given this foo
      --> When this bar
         --> Then foobar

What we really want is:

Given this foo
--> When this bar
   --> Then foobar

Specs can be configured by overriding configure:

class SomeSpec: KSpec() {
    override fun configure(config: KSpecConfig) {
        ...
    }
    ...
}

Configuration can be shared via inheritance however it's not that flexible. Allow specifying configurations via an annotation (@Configurations) which can contains an array of Configuration classes, e.g:

Configuration

class SharedConfiguration: Configuration {
    override fun apply(config: KSpecConfig) {
        ...
    }
}

Spec

@Configurations(SharedConfiguration::class)
class SomeSpec: KSpec() {
    ...
}

Configurations are applied in the order they are listed.

This resolves #19, resolves #21 and bring a lot of changes internally. Example groups can now be tagged and its children will automatically inherit those tags. before, after and around hooks now gets called for every example group as well.

#17 introduces the ability to filter examples before a run, we can rework pending examples to take advantage of that. Implementation should be the same as how focused examples are implemented.

Support fdescribe and fcontext, semantics is the same with fit.

With #24 we can do a lot more processing, move tags property up the hierarchy to Context.

This can be done per spec basis.

class SomeSpec: KSpec() {
    override fun configure(configuration: KSpecConfiguration) {
    }

    ...
}

Provide the following methods:

• [x] before(example) - invoked before an example is executed.
• [x] after(example) - invoked after an example is executed.
• [x] around(example) - provide a way to wrap an execution of an example (e.g. run in a different thread - timeout?)

Does KSpec support any form of property based testing. There are no examples I have found so far.

Property-based testing is testing à la QuickCheck, Hypothesis, ScalaCheck, KotlinTest, Specks, Spock/Spock Generator, etc.

It would be good nice if individual KSpec blocks could be run but the intelliJ test runner, without needing to use fit, fdescribe etc.

Is anyone working on this now?

So Jetbrains just announced Kotlin's Post 1.0 Roadmap and one of the planned features is local delegated properties. This can be very useful with subjects!

describe(Foo::class) {
    val foo by subject() // using no-arg constructor
}

// or

describe(Foo::class) {
    val foo by subject {
        return@subject Foo(1, 2)
    }
}

This is very similar to RSpec's memoized helpers, heck we can even support let.

describe("foo") {
    var count = 0
    val something by let {
        return@let ++count
    }

    it("it should be 1") {
        assert(something == 1)
    }

    it("it should be 2") {
        assert(something == 2)
    }
}

Here is a small sample that introduces an intermediate class that shortens specification classes. We lose flexibility a bit (no way to change the runner via @RunWith) and it also clashes the name with the Spec interface. This was inspired by https://github.com/JetBrains/spek/pull/71.

import com.natpryce.hamkrest.assertion.assertThat
import com.natpryce.hamkrest.equalTo
import io.polymorphicpanda.kspec.KSpec
import io.polymorphicpanda.kspec.describe
import io.polymorphicpanda.kspec.it
import io.polymorphicpanda.kspec.junit.JUnitKSpecRunner
import org.junit.runner.RunWith

@RunWith(JUnitKSpecRunner::class)
open class Spec(private val definition: Spec.() -> Unit) : KSpec() {
    override fun spec() {
        definition()
    }
}

class ExampleSpec : Spec({
    describe("a group") {
        it("example") {
            assertThat(2 + 3, equalTo(5))
        }
    }
})