kog

A simple, experimental Kotlin web framework inspired by Clojure's Ring.

A kog application is a function that takes a Request and returns a Response.

Built on top of Jetty.

import com.danneu.kog.Server
import com.danneu.kog.Response

Server({ Response().text("hello world") }).listen(3000)

Goals

Simplicity
Middleware
Functional composition

Install
Quick Start
- Hello World
- Type-Safe Routing
Concepts
JSON
- JSON Encoding
- JSON Decoding
Routing
- Router mounting
Cookies
- Request Cookies
- Response Cookies
Included Middleware
HTML Templating
WebSockets
- Idle Timeout
Caching
- In-Memory Cache
Environment Variables
Heroku Deploy
Example: Tiny Pastebin Server
Content Negotiation
- Most acceptable language
License

Install

repositories {
    maven { url "https://jitpack.io" }
}

dependencies {
    compile "com.danneu:kog:x.y.z"
    // Or always get latest
    compile "com.danneu:kog:master-SNAPSHOT"
}

Quick Start

Hello World

 Hello world") } // or use the Handler typealias: val handler: Handler = { req -> Response().html("
 Hello world") } fun main(args: Array
  ) { Server(handler).listen(3000) }"> 
  import com.danneu.kog.Response
import com.danneu.kog.Request
import com.danneu.kog.Handler
import com.danneu.kog.Server

fun handler(req: Request): Response {
  return Response().html("Hello world")
}

// or use the Handler typealias:

val handler: Handler = { req ->
  Response().html("Hello world") 
}

fun main(args: Array<String>) {
  Server(handler).listen(3000)
} 
 

Type-Safe Routing

Response().text("list users") }) get("/users/ ", fun(id: Int): Handler = { req -> Response().text("show user $id") }) get("/users/ /edit", fun(id: Int): Handler = { req -> Response().text("edit user $id") }) // Wrap routes in a group to dry up middleware application group("/stories/ ", listOf(middleware)) { get("/comments", listOf(middleware), fun(id: java.util.UUID): Handler = { Response().text("listing comments for story $id") }) } delete("/admin/users/ ", listOf(ensureAdmin()), fun(id: Int): Handler = { req -> Response().text("admin panel, delete user $id") }) get("/ // ", fun(a: Int, b: Int, c: Int): Handler = { req -> Response().json(JE.obj("sum" to JE.num(a + b + c))) }) } } val handler = middleware1(middleware2(middleware3(router.handler()))) fun main(args: Array ) { Server(handler).listen(3000) }">

import com.danneu.json.Encoder as JE
import com.danneu.kog.Router
import com.danneu.kog.Response
import com.danneu.kog.Request
import com.danneu.kog.Handler
import com.danneu.kog.Server

val router = Router {
    get("/users", fun(): Handler = { req ->
        Response().text("list users")
    })
    
    get("/users/
        
         "
        , fun(id: Int): Handler = { req ->
        Response().text("show user $id")
    })
    
    get("/users/
        
         /edit
         "
        , fun(id: Int): Handler = { req ->
        Response().text("edit user $id")
    })
    
    // Wrap routes in a group to dry up middleware application
    group("/stories/
        
         "
        , listOf(middleware)) {
        get("/comments", listOf(middleware), fun(id: java.util.UUID): Handler = { 
            Response().text("listing comments for story $id")
        })
    }
    
    delete("/admin/users/
        
         "
        , listOf(ensureAdmin()), fun(id: Int): Handler = { req ->
        Response().text("admin panel, delete user $id")
    })
    
    get("///
        
         ", 
         fun(
         a
         : 
         Int, 
         b
         : 
         Int, 
         c
         : 
         Int): 
         Handler 
         = { req 
         ->
        
         Response().json(
         JE.obj(
         "sum" to 
         JE.num(a 
         + b 
         + c)))
    })
  }
}


         val handler 
         = middleware1(middleware2(middleware3(router.handler())))


         fun 
         main(
         args
         : 
         Array<
         String>) {
  
         Server(handler).listen(
         3000)
}

Concepts

A kog application is simply a function that takes a Request and returns a Response.

Request & Response

The Request and Response have an API that makes it easy to chain transformations together.

Example junk-drawer:

Hello") // text/html Response().json(JE.obj("number" to JE.num(42))) // application/json {"number": 42} Response().json(JE.array(JE.num(1), JE.num(2), JE.num(3))) // application/json [1, 2, 3] Response().file(File("video.mp4")) // video/mp4 (determines response headers from File metadata) Response().stream(File("video.mp4")) // video/mp4 Response().setHeader(Header.AccessControlAllowOrigin, "*") Response().type = ContentType(Mime.Html, mapOf("charset", "utf-8")) Response().appendHeader(Header.Custom("X-Fruit"), "orange") Response().redirect("/") // 302 redirect Response().redirect("/", permanent = true) // 301 redirect Response().redirectBack(request, "/") // 302 redirect ">
import com.danneu.kog.Status import com.danneu.kog.json.Encoder as JE import java.util.File Response() // skeleton 200 response Response(Status.NotFound) // 404 response Response.notFound() <-- Sugar // 404 response Response().text("Hello") // text/plain Response().html("
Hello
") // text/html Response().json(JE.obj("number" to JE.num(42))) // application/json {"number": 42} Response().json(JE.array(JE.num(1), JE.num(2), JE.num(3))) // application/json [1, 2, 3] Response().file(File("video.mp4")) // video/mp4 (determines response headers from File metadata) Response().stream(File("video.mp4")) // video/mp4 Response().setHeader(Header.AccessControlAllowOrigin, "*") Response().type = ContentType(Mime.Html, mapOf("charset", "utf-8")) Response().appendHeader(Header.Custom("X-Fruit"), "orange") Response().redirect("/") // 302 redirect Response().redirect("/", permanent = true) // 301 redirect Response().redirectBack(request, "/") // 302 redirect

request.type // ContentType(mime=Mime.Html, params=mapOf("charset" to "utf-8")) request.href // http://example.com/users?sort=created request.path // "/users" request.method // Method.get request.params // Map request.json(decoder) // com.danneu.result.Result request.utf8() // "{\"foo\": \"bar\"}" request.headers // [(Header.Host, "example.com"), ...] request.getHeader(Header.Host) // "example.com"? request.getHeader(Header.Custom("xxx")) // null request.setHeader(Header.UserAgent, "MyCrawler/0.0.1") // Request }">
import com.danneu.kog.json.Decoder as JD import com.danneu.kog.Header // GET http://example.com/users?sort=created, json body is {"foo": "bar"} var handler: Handler = { request -> request.type // ContentType(mime=Mime.Html, params=mapOf("charset" to "utf-8")) request.href // http://example.com/users?sort=created request.path // "/users" request.method // Method.get request.params // Map request.json(decoder) // com.danneu.result.Result request.utf8() // "{\"foo\": \"bar\"}" request.headers // [(Header.Host, "example.com"), ...] request.getHeader(Header.Host) // "example.com"? request.getHeader(Header.Custom("xxx")) // null request.setHeader(Header.UserAgent, "MyCrawler/0.0.1") // Request }

Handler

typealias Handler = (Request) -> Response

Your application is a function that takes a Request and returns a Response.

) { Server(handler).listen(3000) }">
val handler: Handler = { request in Response().text("Hello world") } fun main(args: Array<String>) { Server(handler).listen(3000) }

Middleware

typealias Middleware = (Handler) -> Handler

Middleware functions let you run logic when the request is going downstream and/or when the response is coming upstream.

) { Server(logger(handler)).listen() }">
val logger: Middleware = { handler -> { request -> println("Request coming in") val response = handler(request) println("Response going out") response }} val handler: Handler = { Response().text("Hello world") } fun main(args: Array<String>) { Server(logger(handler)).listen() }

Since middleware are just functions, it's trivial to compose them:

import com.danneu.kog.middleware.compose // `logger` will touch the request first and the response last val middleware = compose(logger, cookieParser, loadCurrentUser) Server(middleware(handler)).listen(3000)

Tip: Short-Circuiting Lambdas

You often want to bail early when writing middleware and handlers, like short-circuiting your handler with a 400 Bad Request when the client gives you invalid data.

The compiler will complain if you return inside a lambda expression, but you can fix this by using a label@:

val middleware: Middleware = { handler -> handler@ { req -> val data = req.query.get("data") ?: return@handler Response.badRequest() Response().text("You sent: $data") }}

JSON

kog wraps the small, fast, and simple ralfstx/minimal-json library with combinators for working with JSON.

Note: json combinators and the result monad have been extracted from kog:

danneu/kotlin-json-combinator

danneu/kotlin-result

JSON Encoding

kog's built-in JSON encoder has these methods: .obj(), .array(), .num(), .str(), .null(), .bool().

They all return com.danneu.json.JsonValue objects that you pass to Response#json.

import com.danneu.json.Encoder as JE val handler: Handler = { req -> Response().json(JE.obj("hello" to JE.str("world"))) }

import com.danneu.json.Encoder as JE val handler: Handler = { req -> Response().json(JE.array(JE.str("a"), JE.str("b"), JE.str("c"))) // Or Response().json(JE.array(listOf(JE.str("a"), JE.str("b"), JE.str("c")))) }

import com.danneu.json.Encoder as JE val handler: Handler = { req -> Response().json(JE.obj( "ok" to JE.bool(true), "user" to JE.obj( "id" to JE.num(user.id), "username" to JE.str(user.uname), "luckyNumbers" to JE.array(JE.num(3), JE.num(9), JE.num(27)) ) )) }

It might seem redundant/tedious to call JE.str("foo") and JE.num(42), but it's type-safe so that you can only pass things into the encoder that's json-serializable. I'm not sure if kotlin supports anything simpler at the moment.

JSON Decoding

kog comes with a declarative JSON parser combinator inspired by Elm's.

Decoder is a decoder that will return Result when invoked on a JSON string.

// failure Response.badRequest() }) }">
import com.danneu.json.Decoder as JD import com.danneu.json.Encoder as JE // example request payload: [1, 2, 3] val handler = { request -> request.json(JD.array(JD.int)).fold({ nums -> // success Response().json(JE.obj("sum" to JE.num(nums.sum()))) }, { parseException -> // failure Response.badRequest() }) }

We can use Result#getOrElse() to rewrite the previous example so that invalid user-input will defaults to an empty list of numbers.

import com.danneu.json.Decoder as JD import com.danneu.json.Encoder as JE // example request payload: [1, 2, 3] val handler = { req -> val sum = req.json(JD.array(JD.int)).getOrElse(emptyList()).sum() Response().json(JE.obj("sum" to JE.num(sum))) }

This authentication handler parses the username/password combo from the request's JSON body:

val decoder = JD.pairOf( JD.get(listOf("user", "uname"), JD.string), JD.get("password", JD.string) ) val (uname, password) = request.json(decoder) // ... authenticate user ... Response().json(JE.obj("success" to JE.obj("uname" to JE.str(uname)))) }">
import com.danneu.json.Decoder as JD import com.danneu.json.Encoder as JE // example request payload: {"user": {"uname": "chuck"}, "password": "secret"} val handler = { request -> val decoder = JD.pairOf( JD.get(listOf("user", "uname"), JD.string), JD.get("password", JD.string) ) val (uname, password) = request.json(decoder) // ... authenticate user ... Response().json(JE.obj("success" to JE.obj("uname" to JE.str(uname)))) }

Check out danneu/kotlin-json-combinator and danneu/kotlin-result for more examples.

Routing

kog's router is type-safe because routes only match if the URL params can be parsed into the arguments that your function expects.

Available coercions:

kotlin.Int

kotlin.Long

kotlin.Float

kotlin.Double

java.util.UUID

For example:

", fun(uuid: java.util.UUID): Handler = { req -> Response().text("you provided a uuid of version ${uuid.version} with a timestamp of ${uuid.timestamp}") }) }">
Router { // GET /uuid/aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa -> 200 Ok // GET /uuid/AAAAAAAA-AAAA-AAAA-AAAA-AAAAAAAAAAAA -> 200 Ok // GET /uuid/42 -> 404 Not Found // GET /uuid/foo -> 404 Not Found get("/uuid/ " , fun(uuid: java.util.UUID): Handler = { req -> Response().text("you provided a uuid of version ${uuid.version} with a timestamp of ${uuid.timestamp}") }) }

Here's a more meandering example:

Response().text("homepage") }) get("/users/ ", fun(id: Int): Handler = { req -> Response().text("show user $id") }) get("/users/ /edit", fun(id: Int): Handler = { req -> Response().text("edit user $id") }) // Wrap routes in a group to dry up middleware application group("/stories/ ", listOf(middleware)) { get("/comments", listOf(middleware), fun(id: java.util.UUID): Handler = { Response().text("listing comments for story $id") }) } delete("/admin/users/ ", listOf(ensureAdmin()), fun(id: Int): Handler = { req -> Response().text("admin panel, delete user $id") }) get("/ // ", fun(a: Int, b: Int, c: Int): Handler = { req -> Response().json(JE.obj("sum" to JE.num(a + b + c))) }) get("/hello/world", fun(a: Int, b: String): Handler = { Response().text("this route can never match the function (Int, Int) -> ...") }) get("/hello/world", fun(): Handler = { Response().text("this route *will* match") }) } } fun main(args: Array ) { Server(handler).listen(3000) }">
import com.danneu.kog.json.Encoder as JE import com.danneu.kog.Router import com.danneu.kog.Response import com.danneu.kog.Request import com.danneu.kog.Handler import com.danneu.kog.Server val router = Router(middleware1(), middleware2()) { get("/", fun(): Handler = { req -> Response().text("homepage") }) get("/users/ " , fun(id: Int): Handler = { req -> Response().text("show user $id") }) get("/users/ /edit " , fun(id: Int): Handler = { req -> Response().text("edit user $id") }) // Wrap routes in a group to dry up middleware application group("/stories/ " , listOf(middleware)) { get("/comments", listOf(middleware), fun(id: java.util.UUID): Handler = { Response().text("listing comments for story $id") }) } delete("/admin/users/ " , listOf(ensureAdmin()), fun(id: Int): Handler = { req -> Response().text("admin panel, delete user $id") }) get("/// ", fun( a : Int, b : Int, c : Int): Handler = { req -> Response().json( JE.obj( "sum" to JE.num(a + b + c))) }) get( "/hello/world", fun( a : Int, b : String): Handler = { Response().text( "this route can never match the function (Int, Int) -> ...") }) get( "/hello/world", fun(): Handler = { Response().text( "this route *will* match") }) } } fun main( args : Array< String>) { Server(handler).listen( 3000) }

Router mounting

Router#mount(subrouter) will merge a child router into the current router.

Useful for breaking your application into individual routers that you then mount into a top-level router.

val subrouter = Router { get("/foo", fun(): Handler = { Response() }) } val router = Router { mount(subrouter) }

curl http://localhost:3000/foo # 200 Ok

Or mount routers at a prefix:

val subrouter = Router { get("/foo", fun(): Handler = { Response() }) } val router = Router { mount("/subrouter", subrouter) }

curl http://localhost:3000/foo # 404 Not Found curl http://localhost:3000/subrouter/foo # 200 Ok

Or mount routers in a group:

val subrouter = Router { get("/foo", fun(): Handler = { Response() }) } val router = Router { group("/group") { mount("/subrouter", subrouter) } }

Note: The mount prefix must be static. It does not support dynamic patterns like "/users/".

Cookies

Request Cookies

Request#cookies is a MutableMap which maps cookie names to cookie values received in the request.

Response Cookies

Response#cookies is a MutableMap which maps cookie names to cookie objects that will get sent to the client.

Here's a handler that increments a counter cookie on every request that will expire in three days:

val count = request.parseCounter() + 1 Response().text("count: $count").setCounter(count) } fun main(args: Array ) { Server(handler).listen(9000) }">
import com.danneu.kog.Response import com.danneu.kog.Handler import com.danneu.kog.Server import com.danneu.kog.cookies.Cookie import java.time.OffsetDateTime fun Request.parseCounter(): Int = try { cookies.getOrDefault("counter", "0").toInt() } catch(e: NumberFormatException) { 0 } fun Response.setCounter(count: Int): Response = apply { cookies["counter"] = Cookie(count.toString(), duration = Cookie.Ttl.Expires(OffsetDateTime.now().plusDays(3))) } val handler: Handler = { request -> val count = request.parseCounter() + 1 Response().text("count: $count").setCounter(count) } fun main(args: Array<String>) { Server(handler).listen(9000) }

Demo:

$ http --session=kog-example --body localhost:9000 count: 1 $ http --session=kog-example --body localhost:9000 count: 2 $ http --session=kog-example --body localhost:9000 count: 3

Included Middleware

The com.danneu.kog.batteries package includes some useful middleware.

Development Logger

The logger middleware prints basic info about the request and response to stdout.

import com.danneu.kog.batteries.logger Server(logger(handler)).listen()

Static File Serving

The serveStatic middleware checks the request.path against a directory that you want to serve static files from.

import com.danneu.kog.batteries.serveStatic val middleware = serveStatic("public", maxAge = Duration.ofDays(365)) val handler = { Response().text(":)") } Server(middleware(handler)).listen()

If we have a public folder in our project root with a file message.txt, then the responses will look like this:

$ http localhost:3000/foo HTTP/1.1 404 Not Found $ http localhost:3000/message.txt HTTP/1.1 200 OK Content-Length: 38 Content-Type: text/plain This is a message from the file system $ http localhost:3000/../passwords.txt HTTP/1.1 400 Bad Request

Conditional-Get Caching

This middleware adds Last-Modified or ETag headers to each downstream response which the browser will echo back on subsequent requests.

If the response's Last-Modified/ETag matches the request, then this middleware instead responds with 304 Not Modified which tells the browser to use its cache.

ETag

notModified(etag = true) will generate an ETag header for each downstream response.

val router = Router(notModified(etag = true)) { get("/", fun(): Handler = { Response().text("Hello, world!) }) }

First request gives us an ETag.

$ http localhost:9000 HTTP/1.1 200 OK Content-Length: 13 Content-Type: text/plain ETag: "d-bNNVbesNpUvKBgtMOUeYOQ" Hello, world!

When we echo back the ETag, the server lets us know that the response hasn't changed:

$ http localhost:9000 If-None-Match:'"d-bNNVbesNpUvKBgtMOUeYOQ"' HTTP/1.1 304 Not Modified

Last-Modified

notModified(etag = false) will only add a Last-Modified header to downstream responses if response.body is ResponseBody.File since kog can read the mtime from the File's metadata.

If the response body is not a ResponseBody.File type, then no header will be added.

This is only useful for serving static assets from the filesystem since ETags are unnecessary to generate when you have a file's modification time.

val router = Router { // TODO: kog doesn't yet support mounting middleware on a prefix use("/assets", notModified(etag = false), serveStatic("public", maxAge = Duration.ofHours(4))) get("/") { Response().text("homepage") }

Multipart File Uploads

To handle file uploads, use the com.danneu.kog.batteries.multipart middleware.

This middleware parses file uploads out of "multipart/form-data" requests and populates request.uploads : MutableMap for your handler to access which is a mapping of field names to File representations.

package com.danneu.kog.batteries.multipart class SavedUpload(val file: java.io.File, val filename: String, val contentType: String, val length: Long)

In this early implementation, by the time your handler is executed, the file uploads have already been piped into temporary files in the file-system which will get automatically deleted.

File1: File2 (Ignored):
""") }) post("/upload", multipart(Whitelist.only(setOf("file1"))), fun(): Handler = { req -> val upload = req.uploads["file1"] Response().text("You uploaded ${upload?.length ?: "--"} bytes") }) } fun main(args: Array ) { Server(router.handler()).listen(3000) }">
import com.danneu.kog.Router import com.danneu.kog.batteries.multipart import com.danneu.kog.batteries.multipart.Whitelist val router = Router { get("/", fun(): Handler = { Response().html("""
File1: File2 (Ignored):
""") }) post("/upload", multipart(Whitelist.only(setOf("file1"))), fun(): Handler = { req -> val upload = req.uploads["file1"] Response().text("You uploaded ${upload?.length ?: "--"} bytes") }) } fun main(args: Array<String>) { Server(router.handler()).listen(3000) }

Pass a whitelist into multipart() to only process field names that you expect.

import com.danneu.kog.batteries.multipart import com.danneu.kog.batteries.multipart.Whitelist multipart(whitelist = Whitelist.all) multipart(whitelist = Whitelist.only(setOf("field1", "field2")))

Basic Auth

Just pass a (name, password) -> Boolean predicate to the basicAuth() middleware.

Your handler won't get called unless the user satisfies it.

import com.danneu.kog.batteries.basicAuth fun String.sha256(): ByteArray { return java.security.MessageDigest.getInstance("SHA-256").digest(this.toByteArray()) } val secretHash = "a man a plan a canal panama".sha256() fun isAuthenticated(name: String, pass: String): Boolean { return java.util.Arrays.equals(pass.sha256(), secretHash) } val router = Router { get("/", basicAuth(::isAuthenticated)) { Response().text("You are authenticated!") } }

Compression / Gzip

The compress middleware reads and manages the appropriate headers to determine if it should send a gzip-encoded response to the client.

Options:

compress(threshold: ByteLength) (Default = 1024 bytes) Only compress the response if it is at least this large.

compress(predicate = (String?) -> Boolean) (Default = Looks up mime in https://github.com/jshttp/mime-db file) Only compress the response if its Content-Type header passes predicate(type).

Some examples:

<-- Not compressed (not json) get("/b", fun(): Handler = { Response().html("
bar
") }) // <-- Not compressed (not json) get("/c", fun(): Handler = { Response().jsonArray(1, 2, 3) }) // <-- Compressed } // These responses will be compressed if they are at least 1024 bytes group(compress(threshold = ByteLength.ofBytes(1024))) { get("/d", fun(): Handler = { Response().text("qux") }) // <-- Not compressed (too small) } }">
import com.danneu.kog.batteries.compress import com.danneu.kog.ByteLength val router = Router() { // These responses will be compressed if they are JSON of any size group(compress(threshold = ByteLength.zero, predicate = { it == "application/json" })) { get("/a", fun(): Handler = { Response().text("foo") }) // <-- Not compressed (not json) get("/b", fun(): Handler = { Response().html("
bar
") }) // <-- Not compressed (not json) get("/c", fun(): Handler = { Response().jsonArray(1, 2, 3) }) // <-- Compressed } // These responses will be compressed if they are at least 1024 bytes group(compress(threshold = ByteLength.ofBytes(1024))) { get("/d", fun(): Handler = { Response().text("qux") }) // <-- Not compressed (too small) } }

HTML Templating

Templating libraries generally generate an HTML string. Just pass it to Response().html(html).

For example, tipsy/j2html is a simple templating library for generating HTML from your handlers.

compile "com.j2html:j2html:1.0.0"

Here's an example server with a "/" route that renders a file-upload form that posts to a "/upload" route.

) { Server(router.handler()).listen(9000) }">
import j2html.TagCreator.* import j2html.tags.ContainerTag import com.danneu.kog.Router import com.danneu.kog.Response import com.danneu.kog.Server import com.danneu.kog.batteries.multipart import com.danneu.kog.batteries.multipart.Whitelist fun layout(vararg tags: ContainerTag): String = document().render() + html().with( body().with(*tags) ).render() val router: Router = Router { get("/", fun(): Handler = { Response().html(layout( form().attr("enctype", "multipart/form-data").withMethod("POST").withAction("/upload").with( input().withType("file").withName("myFile"), button().withType("submit").withText("Upload File") ) )) }) post("/upload", multipart(Whitelist.only(setOf("myFile"))), fun(): Handler = { Response().text("Uploaded ${req.uploads["myFile"]?.length ?: "--"} bytes") }) } fun main(args: Array<String>) { Server(router.handler()).listen(9000) }

WebSockets

Check out examples/websockets.kt for a websocket example that demonstrates a websocket handler that echos back every message, and a websocket handler bound to a dynamic /ws/ route.

Take note of a few limitations explained in the comments that I'm working on fixing.

Idle Timeout

By default, Jetty (and thus kog) timeout connections that have idled for 30 seconds.

You can change this when initializing a kog Server:

import com.danneu.kog.Server import java.time.Duration fun main(args: Array<String>) { Server(handler, idleTimeout = Duration.ofMinutes(5)).listen(3000) }

However, instead of changing kog's idleTimeout, you probably want to have your websocket clients ping the server to keep the connections alive.

Often reverse proxies like nginx, Heroku's routing layer, and Cloudflare have their own idle timeout.

For example, here are Heroku's docs on the matter: https://devcenter.heroku.com/articles/websockets#timeouts

I believe this is also why websocket libraries like https://socket.io/ implement their own ping/pong.

Finally, it seems that Jetty's maximum idle timeout is 5 minutes, so passing in durations longer than 5 minutes seems to just max out at 5 minutes. If someone can correct me here, feel free to create an issue.

Caching

In-Memory Cache

I've been impressed with Ben Manes' ben-manes/caffeine library.

Easy to pick up and use in any project.

There's also Guava's Cache.

Environment Variables

Kog's Env object provides a central way to access any customizations passed into an application.

First it reads from an optional .env file, then it reads from system properties, and finally it reads from system environment variables (highest precedence). Any conflicts will be overwritten in that order.

For instance, if we had PORT=3000 in an .env file and then launched our application with:

PORT=9999 java -jar app.java

Then this is what we'd see in our code:

import com.danneu.kog.Env Env.int("PORT") == 9999

For example, when deploying an application to Heroku, you want to bind to the port that Heroku gives you via the "PORT" env variable. But you may want to default to port 3000 in development when there is no port configured:

import com.danneu.kog.Server import com.danneu.kog.Env fun main(args: Array<String>) { Server(router.handler()).listen(Env.int("PORT") ?: 3000) }

Env provides some conveniences:

Env.string(key)

Env.int(key)

Env.float(key)

Env.bool(key): True if the value is "true" or "1", e.g. VALUE=true java -jar app.java

If the parse fails, null is returned.

You can get a new, overridden env container with .fork():

3000 Env.fork(mapOf("PORT" to "8888")).int("PORT") //=> 8888 Env.int("PORT") //=> 3000">
Env.int("PORT") //=> 3000 Env.fork(mapOf("PORT" to "8888")).int("PORT") //=> 8888 Env.int("PORT") //=> 3000

Heroku Deploy

This example application will be called "com.danneu.kogtest".

I'm not sure what the minimal boilerplate is, but the following is what worked for me.

In ./build.gradle:

<--------------- CHANGE ME repositories { jcenter() maven { url "http://dl.bintray.com/kotlin/kotlin-eap-1.1" } maven { url 'https://jitpack.io' } } dependencies { compile "org.jetbrains.kotlin:kotlin-stdlib:$kotlin_version" compile 'com.danneu:kog:master-SNAPSHOT' } task stage(dependsOn: ['shadowJar', 'clean'])">
buildscript { ext.kotlin_version = "1.1-M03" ext.shadow_version = "1.2.3" repositories { jcenter() maven { url "http://dl.bintray.com/kotlin/kotlin-eap-1.1" } } dependencies { classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version" classpath "com.github.jengelman.gradle.plugins:shadow:$shadow_version" } } apply plugin: 'kotlin' apply plugin: 'com.github.johnrengelman.shadow' apply plugin: 'application' mainClassName = 'com.danneu.kogtest.MainKt' // <--------------- CHANGE ME repositories { jcenter() maven { url "http://dl.bintray.com/kotlin/kotlin-eap-1.1" } maven { url 'https://jitpack.io' } } dependencies { compile "org.jetbrains.kotlin:kotlin-stdlib:$kotlin_version" compile 'com.danneu:kog:master-SNAPSHOT' } task stage(dependsOn: ['shadowJar', 'clean'])

In ./src/main/kotlin/com/danneu/kogtest/main.kt:

package com.danneu.kogtest import com.danneu.kog.Env import com.danneu.kog.Handler import com.danneu.kog.Response import com.danneu.kog.Server fun main(args: Array<String>) { val handler: Handler = { Response().text("Hello, world!") } Server(handler).listen(ENV.int("PORT") ?: 3000) }

Reminder: Bind to the PORT env variable that Heroku will set.

In ./Procfile:

web: java -jar build/libs/kogtest-all.jar

Create and push to Heroku app:

heroku apps:create my-app commit -am 'Initial commit' git push heroku master

Example: Tiny Pastebin Server

I got this idea from: https://rocket.rs/guide/pastebin/.

This simple server will have two endpoints:

Upload file: curl --data-binary @example.txt http://localhost:3000.

Uploads binary stream to a "pastes" directory on the server.

Server responds with JSON { "url": "http://localhost:3000/ " }.

Fetch file: curl http://localhost:3000/.

Server responds with file or 404.

// Generate random ID for user's upload val id = UUID.randomUUID() // Ensure "pastes" directory is created val destFile = File(File("pastes").apply { mkdir() }, id.toString()) // Move user's upload into "pastes", bailing if their upload size is too large. try { req.body.limitedCopyTo(uploadLimit, destFile.outputStream()) } catch(e: CopyLimitExceeded) { destFile.delete() return@handler Response.badRequest().text("Cannot upload more than ${uploadLimit.byteLength} bytes") } // If stream was empty, delete the file and scold user if (destFile.length() == 0L) { destFile.delete() return@handler Response.badRequest().text("Paste file required") } println("A client uploaded ${destFile.length()} bytes to ${destFile.absolutePath}") // Tell user where they can find their uploaded file Response().json(JE.obj("url" to JE.str("http://localhost:${req.serverPort}/$id"))) }) // Fetch file get("/ ", fun(id: UUID): Handler = handler@ { req -> val file = File("pastes/$id") if (!file.exists()) return@handler Response.notFound() Response().file(file) }) } fun main(args: Array ) { Server(router.handler()).listen(3000) }">
import com.danneu.kog.Router import com.danneu.kog.Response import com.danneu.kog.Handler import com.danneu.kog.util.CopyLimitExceeded import com.danneu.kog.util.limitedCopyTo import java.io.File import java.util.UUID val uploadLimit = ByteLength.ofMegabytes(10) val router = Router { // Upload file post("/", fun(): Handler = handler@ { req -> // Generate random ID for user's upload val id = UUID.randomUUID() // Ensure "pastes" directory is created val destFile = File(File("pastes").apply { mkdir() }, id.toString()) // Move user's upload into "pastes", bailing if their upload size is too large. try { req.body.limitedCopyTo(uploadLimit, destFile.outputStream()) } catch(e: CopyLimitExceeded) { destFile.delete() return@handler Response.badRequest().text("Cannot upload more than ${uploadLimit.byteLength} bytes") } // If stream was empty, delete the file and scold user if (destFile.length() == 0L) { destFile.delete() return@handler Response.badRequest().text("Paste file required") } println("A client uploaded ${destFile.length()} bytes to ${destFile.absolutePath}") // Tell user where they can find their uploaded file Response().json(JE.obj("url" to JE.str("http://localhost:${req.serverPort}/$id"))) }) // Fetch file get("/ " , fun(id: UUID): Handler = handler@ { req -> val file = File("pastes/$id") if (!file.exists()) return@handler Response.notFound() Response().file(file) }) } fun main(args: Array<String>) { Server(router.handler()).listen(3000) }

Content Negotiation

TODO: Improve negotiation docs

Each request has a negotiator that parses the accept-* headers, returning a list of values in order of client preference.

request.negotiate.mediaTypes parses the accept header.

request.negotiate.languages parses the accept-language header.

request.negotiate.encodings parses the accept-encoding header.

Until the docs are fleshed out, here's a demo server that will illuminate this:

fun main(args: Array<String>) { val handler: Handler = { request -> println(request.headers.toString()) Response().text(""" languages: ${request.negotiate.languages()} encodings: ${request.negotiate.encodings()} mediaTypes: ${request.negotiate.mediaTypes()} """.trimIndent()) } Server(handler).listen(3000) }

An example curl request:

curl http://localhost:3000 \ --header 'Accept-Language:de;q=0.7, fr-CH, fr;q=0.9, en;q=0.8, *;q=0.5, de-CH;q=0.2' \ --header 'accept:application/json,TEXT/*' \ --header 'accept-encoding:gzip,DeFLaTE'

Corresponding response:

languages: [French[CH], French[*], English[*], German[*], *[*], German[CH]] encodings: [Encoding(name='gzip', q=1.0), Encoding(name='deflate', q=1.0)] mediaTypes: [MediaType(type='application', subtype='json', q=1.0), MediaType(type='text', subtype='*', q=1.0)]

Notice that values ("TEXT/*", "DeFLaTE") are always downcased for easy comparison.

Most acceptable language

Given a list of languages that you want to support, the negotiator can return a list that filters and sorts your available languages down in order of client preference, the first one being the client's highest preference.

import com.danneu.kog.Lang import com.danneu.kog.Locale // Request Accept-Language: "en-US, es" request.negotiate.acceptableLanguages(listOf( Lang.Spanish(), Lang.English(Locale.UnitedStates) )) == listOf( Lang.English(Locale.UnitedStates), Lang.Spanish() )

Also, note that we don't have to provide a locale. If the client asks for en-US, then of course Lang.English() without a locale should be acceptable if we have no more specific match.

// Request Accept-Language: "en-US, es" request.negotiate.acceptableLanguages(listOf( Lang.Spanish(), Lang.English() )) == listOf( Lang.English(), Lang.Spanish() )

The singular form, .acceptableLanguage(), is a helper that returns the first result (the most preferred language in common with the client).

// Request Accept-Language: "en-US, es" request.negotiate.acceptableLanguage(listOf( Lang.Spanish(), Lang.English() )) == Lang.English()

Here we write an extension function Request#lang() that returns the optimal lang between our available langs and the client's requested langs.

We define an internal OurLangs enum so that we can exhaust it with when expressions in our routes or middleware.

return when (request.lang()) { OurLangs.Spanish() -> Response().text("Les servimos en español") OurLangs.English() -> Response().text("We're serving you English") } })">
enum class OurLangs { Spanish, English } fun Request.lang(): OurLangs { val availableLangs = listOf( Lang.Spanish(), Lang.English() ) return when (this.negotiate.acceptableLanguage(availableLangs)) { Lang.English() -> OurLangs.English // Default to Spanish else -> OurLangs.Spanish } } router.get("/", fun(): Handler = { request -> return when (request.lang()) { OurLangs.Spanish() -> Response().text("Les servimos en español") OurLangs.English() -> Response().text("We're serving you English") } })

License

MIT

TODO: This should go in the wiki but I've disabled the wiki for now.

A list of git commits that I think will be useful to future me, like large refactorings that I would like to revisit in the future.

https://github.com/danneu/kog/commit/7429b6b79699a52a295f2603969d764de7845701 Reabstracting the accept-language Lang and Locale system.

I did this reabstraction to make it easier to match/compare langs. For instance, Lang.English() should match en but also all en-*. But it's still highly experimental since I'm not very opinionated on how i18n should work. I'll revisit this system when I want to play with real world i18n impl's since that will be the real test of flexibility, and I know the current system isn't there yet.
[14 July] https://github.com/danneu/kog/commit/f235a63fb8568fc362aa28e0d358894ead053ede Created ContentType(Mime, params) abstraction.

Trying to get rid of content-type string-typing, and trying to unify/canonicalize all instances of mimes in kog. Probably went too far, but won't be able to see where/why I need to reabstract until I upgrade one of my kog apps that uses content-type more extensively.

Not so convinced that having strings here is all that bad. But I do hate typing out "application/x-www-form-urlencoded" when I have to.

Another issue is that the user can potentially use both res.setHeader(Header.ContentType, "text/html") and res.contentType = ContentType(Mime.Html), but right now I make the res.contentType overwrite the content-type header.

One thing you can do now with the new ContentType(mime, params) abstraction is add a charset=uft-8 and potentially other params (how many are actually used? I can only think of multipart boundary ><). Previously you could not.

Websocket keepalive
Hi :)

it's a nice framework. But I've got a little issue with the connection timeout on websocket.

[70294c19-6a40-4f0e-93a2-84ef22679498] onError: Timeout on Read [70294c19-6a40-4f0e-93a2-84ef22679498] onClose: 1001 Idle Timeout

Do you know how to do a keepalive on websocket? Didn't find anything on Jetty side. Or how to do a ping/pong at least. (With JS side)

Thanks and best regards Gino
opened by ThraaxSession 5
Rebuild the websocket abstraction for use with new Router.kt
I built the websocket abstraction for use with the old/original Router.kt. I need to rebuild/rethink it now that I have the new Router.kt (formerly SafeRouter.kt).

For one, the websocket acceptor should be refactored from its lambda-heavy impl to something with an ad-hoc usage more like:

websocket("/foo", fun(): = object : WebsocketHandler { override fun onConnect() = ... ... })

Also, the websocket/jetty interop in Server.kt is nasty. But I really like the idea of nesting websocket handlers beneath existing middleware.
opened by danneu 4
String encoding - Broken with special chars like: äöüß etc.

Hi @danneu I have a problem with special chars. When I serialize data in JSON and transmit the data to the websocket service, I get only ? back.

Can you help me with it?

Thank you and best regards Gino

opened by ThraaxSession 3
[Router] Implement type-safe router
The API could look something like this:

Router { get("/users/:id", fun(id: Int): Handler = { req -> Response().text("User $id") }) }

TODO:

[x] Middleware tests

[x] Figure out a good group("/prefix") API.

[x] Support websockets

Could map SafeRouter patterns into jetty context paths with wildcards: http://stackoverflow.com/questions/4798120/is-it-possible-to-use-regular-expression-for-jettys-servlet-mapping

[x] Build router using reflection once so that the router isn't using reflection (slow) during dispatch. Right now it naively uses reflection on every request when instead it should compile a mapping at init. (Even though I improved it and checked the box, more work could be done)

[x] Consider compiling one regex that maps requests to handlers.
opened by danneu 2

[Router] Parse url /:params

This is a familiar way to parameterize segments of the route url:

get("/users/:id", fun(request: Request): Response {
    val user = database.getUser(request.params.get(":id"))
    user ?: return Response(Status.notFound)
    return Response().text("Hello, ${user.name}")
})

Just want something quick/dirty/working to start with.

opened by danneu 2

Add Java support

https://kotlinlang.org/docs/reference/java-to-kotlin-interop.html

I had trouble just importing kog into the gradle init java boilerplate project. Gradle seemed to download it, but the IDE didn't seem to be able to find import com.danneu....

opened by danneu 1
[Router] Add scope control
A current source of end-user mistakes and kog-level bugs is that you can accidentally reference top-level methods when you're inside a nested builder block:

val router = Router { group("/prefix") { mount(otherRouter) } }

In the above example, if RouteGroup didn't have the .mount() method, it'll call the top-level router .mount(otherRouter) which will mount the subrouter without a prefix.

To prevent these sorts of bugs in kog and for the end-user, kotlin 1.1 introduces dsl scope control (@DslMarker).
opened by danneu 0
[Router] Support router mounting
Haven't thought of the exact API, but it'd be nice to inject child routers which would help one organize router code.

val router = Router { mount(router1) mount("/prefix", router2) }
opened by danneu 0
Remove funktionale.Option if Result supports null values

I use funktionale's Option in one place where I'd rather just use a Result will a nullable value and remove the dependency.

https://github.com/kittinunf/Result/issues/18

opened by danneu 0
Remove one of the json deps

I'm using both https://github.com/ralfstx/minimal-json and https://github.com/SalomonBrys/Kotson for json decoding and encoding respectively since they were both obvious at those tasks when I started rapidly building kog.

But we only need one lib.

opened by danneu 0
serveStatic - does not find resources from Jar.
Hi @danneu

I've got following snippet:

val public = this::class.java.getResource("/public").path val static = serveStatic(public, maxAge = Duration.ofDays(365))

When I run it directly in gradle with ./gradlew run ... it works and I can gather the static files But when I build a fat Jar and run it, it doesn't work. Then I get this error:

WARN [serveStatic] Could not find public resource folder: "file:/home/gino/Programming/myapp/build/libs/myapp-SNAPSHOT.jar!/public". serveStatic skipped...

After unzipping the Jar, I can see that my public folder is there.

Do you have any ideas?

Thanks and best regards Gino
opened by ThraaxSession 2
[Router] Support prefix mounting
Note: Need more use-case ideas before actually implementing this.

Prefix mounting matches request.paths that start with a given prefix, except that the prefix is then removed from the request.path before being passed to the mounted middleware.

In other words, the mounted middleware do not know about the prefix.

The following router unfortunately queries the filesystem for every request to see if the request hits a file in the public static assets folder.

val router = Router (serveStatic("public")) { get("/") { Response().text("homepage") }

Prefix mounting would let us rewrite that router to limit the serveStatic middleware only to requests with paths that start with /assets, yet without requiring any other changes to our filesystem hierarchy or the public folder.

val router = Router { use("/assets", serveStatic("public") get("/", fun(): Handler = { Response().text("homepage") }) }

Now, a request GET /assets/img/flower.png would trigger the serveStatic middleware, yet it will look up public/img/flower.png (doesn't see the /assets prefix) and serve the file if it exists.

If serveStatic does not handle the request, then the request is passed down the chain as usual with downstream middleware/handlers seeing the prefixed route.
idea
opened by danneu 0
Historical commits
TODO: This should go in the wiki but I've disabled the wiki for now.

A list of git commits that I think will be useful to future me, like large refactorings that I would like to revisit in the future.

https://github.com/danneu/kog/commit/7429b6b79699a52a295f2603969d764de7845701 Reabstracting the accept-language Lang and Locale system.

I did this reabstraction to make it easier to match/compare langs. For instance, Lang.English() should match en but also all en-*. But it's still highly experimental since I'm not very opinionated on how i18n should work. I'll revisit this system when I want to play with real world i18n impl's since that will be the real test of flexibility, and I know the current system isn't there yet.

[14 July] https://github.com/danneu/kog/commit/f235a63fb8568fc362aa28e0d358894ead053ede Created ContentType(Mime, params) abstraction.

Trying to get rid of content-type string-typing, and trying to unify/canonicalize all instances of mimes in kog. Probably went too far, but won't be able to see where/why I need to reabstract until I upgrade one of my kog apps that uses content-type more extensively.

Not so convinced that having strings here is all that bad. But I do hate typing out "application/x-www-form-urlencoded" when I have to.

Another issue is that the user can potentially use both res.setHeader(Header.ContentType, "text/html") and res.contentType = ContentType(Mime.Html), but right now I make the res.contentType overwrite the content-type header.

One thing you can do now with the new ContentType(mime, params) abstraction is add a charset=uft-8 and potentially other params (how many are actually used? I can only think of multipart boundary ><). Previously you could not.
opened by danneu 0
Handle ResponseBody.Writer failure
Since a ResponseBody.Writer is piped to the jetty response object after the kog response is returned from the handler, if the writer throws an error, then kog (e.g. the user's custom error handler) does not handle it. Jetty handles it and displays a blank 500 response.

This is the logic right now:

val request = kog.Request.fromServletRequest(jettyServletRequest) val response = handler(request) response.body.pipe(jettyServletResponse)

For instance, Pebble (http://www.mitchellbosecke.com/pebble/home) writes a to Writer.

val template: PebbleTemplate = engine.getTemplate(path) return Response().writer("text/html") { writer -> template.evaluate(writer, data) }

So if there's an error in the template, like {{ doesNotExist() }}, then you'll get a blank Jetty 500 page.
opened by danneu 0
Support websocket handlers on dynamic endpoints
Right now, returning Response.websocket("/foo/bar", wshandler) from a handler will add the "/foo/bar" -> wshandler mapping to Jetty's context mappings, so it must be a static path.

So, to mount a websocket handler on /users/<name>, you must do something like this:

val router = Router { get("/users/<name>", fun(name: String): Handler = { Response.websocket("/users/$name", /* websocket handler */) }) }

This means that a mapping could be added to Jetty's table for all possible values of /users/<name>.

Even if you ensure Response.websocket() only runs if, say, a user with the given name exists in the database, that's still pretty suboptimal.

The problem is my websocket Jetty code in general. It's a pretty big hack, but I'm not familiar enough with Jetty's API to improve it just yet.

Some objectives that drove my current approach that I want to maintain:

End-user should be able to wrap a websocket endpoint behind existing middleware stacks, like inside a group or router that ensures that the user is an admin.

The websocket handler should access upstream context (like values set by upstream middleware) and the kog.Request.
opened by danneu 1