A lightweight coroutine based persistent job/cron scheduler written in Kotlin

Overview

kjob

Bintray GitHub Build Status Coverage Status

A coroutine based persistent background (cron) scheduler written in Kotlin.

Features

  • Persist scheduled jobs (e.g. mongoDB)
  • Cron jobs
  • Nice DSL for registering and scheduling jobs
  • Multiple instances possible
  • Failed jobs will be rescheduled
  • Other instances will restart the work of a crashed one
  • Pool for blocking and non-blocking jobs
  • Pool size can be defined to go easy on resources
  • Add new features through your own extensions

Installation

Just add the following lines to your build.gradle. For <version> see the button above or releases/latest for the current version number.

repositories {
  jcenter()
}

dependencies {
  implementation "it.justwrote:kjob-core:<version>"
  implementation "it.justwrote:kjob-mongo:<version>" // for mongoDB persistence
  testImplementation "it.justwrote:kjob-inmem:<version>" // for in-memory 'persistence' (e.g. tests)
}

Using kjob

import it.justwrote.kjob.Mongo
import it.justwrote.kjob.Job
import it.justwrote.kjob.job.JobExecutionType
import it.justwrote.kjob.kjob

object OrderCreatedEmail : Job("order-created-email") {
    val recipient = string("recipient")
}

// ...

// start kjob with mongoDB persistence and default configuration
val kjob = kjob(Mongo).start()

// ...

kjob.register(OrderCreatedEmail) {
    executionType = JobExecutionType.NON_BLOCKING // our fake email client is non blocking
    maxRetries = 3
    execute {
        val to = props[it.recipient] // getting address from customer
        client.sendTo(to, subject, body)
    }.onError {
        // errors will automatically logged but we might want to do some metrics or something 
    }
}

// ...

kjob.schedule(OrderCreatedEmail) {
    props[it.recipient] = "[email protected]"
}

// or provide some delay for the scheduling
kjob.schedule(OrderCreatedEmail, 5.seconds) {
    props[it.recipient] = "[email protected]"
}
// this runs the job not immediately but - you may guess it already - in 5 seconds!

For more details please take a look at the examples

Starting kjob

Multiple schedulers are running in the background after starting kjob. There is one looking for new jobs every second (period can be defined in the configuration). If a job has been found that has not yet been started (or reset after an error) and the kjob instance is currently not executing too many other jobs of the same kind (there are blocking and non-blocking jobs) kjob will process it. The second scheduler is handling the locking. It indirectly tells the other kjob instances that this one is still alive. The last scheduler is cleaning up locked jobs of other not responding kjob instances to make the jobs available again for execution.

Multiple kjob instances

To be fault tolerant you sometimes want to have multiple instances of your job processor. This might be in the same app or on different nodes. Therefore, every kjob instances has a unique id which will be added to the job it is currently executing. This actually locks a job to a specific kjob instance. If the kjob instance is somehow dying while executing a job another kjob instance will remove the lock after a specific time (which can be defined in the configuration) and will pick it up again.

Changing Configuration

Changing the config is fairly easy. There is not another config file and everything will be done in code - so you can use your own configuration.

kjob(InMem) {
    nonBlockingMaxJobs = 10 // how many non-blocking jobs will be executed at max in parallel per instance
    blockingMaxJobs = 3 // same for blocking jobs
    maxRetries = 5 // how often will a job be retried until it fails
    defaultJobExecutor = JobExecutionType.BLOCKING // default job execution type
        
    exceptionHandler = { t: Throwable -> logger.error("Unhandled exception", t) } // default error handler for coroutines
    keepAliveExecutionPeriodInSeconds = 60 // the time between 'I am alive' notifications
    jobExecutionPeriodInSeconds = 1 // the time between new job executions
    cleanupPeriodInSeconds = 300 // the time between job clean ups
    cleanupSize = 50 // the amount of jobs that will be cleaned up per schedule
}.start()

MongoDB Configuration

Despite the configuration above there are some mongoDB specific settings that will be shown next.

kjob(Mongo) {
    // all the config above plus those:
    connectionString = "mongodb://localhost" // the mongoDB specific connection string 
    client = null // if a client is specified the 'connectionString' will be ignored
    databaseName = "kjob" // the database where the collections below will be created
    jobCollection = "kjob-jobs" // the collection for all jobs
    lockCollection = "kjob-locks" // the collection for the locking
    expireLockInMinutes = 5L // using the TTL feature of mongoDB to expire a lock
}.start()

Extensions

If you want to add new features to kjob you can do so with a kjob extension.

object ShowIdExtension : ExtensionId<ShowIdEx>

class ShowIdEx(private val config: Configuration, private val kjobConfig: BaseKJob.Configuration, private val kjob: BaseKJob<BaseKJob.Configuration>) : BaseExtension(ShowIdExtension) {
    class Configuration : BaseExtension.Configuration()

    fun showId() {
        // here you have access to some internal properties
        println("KJob has the following id: ${kjob.id}")
    }
}

object ShowIdModule : ExtensionModule<ShowIdEx, ShowIdEx.Configuration, BaseKJob<BaseKJob.Configuration>, BaseKJob.Configuration> {
    override val id: ExtensionId<ShowIdEx> = ShowIdExtension
    override fun create(configure: ShowIdEx.Configuration.() -> Unit, kjobConfig: BaseKJob.Configuration): (BaseKJob<BaseKJob.Configuration>) -> ShowIdEx {
        return { ShowIdEx(ShowIdEx.Configuration().apply(configure), kjobConfig, it) }
    }
}

// ...

val kjob = kjob(InMem) {
    extension(ShowIdModule) // register our extension and bind it to the kjob life cycle
}

// ...

kjob(ShowIdExtension).showId() // access our new extension method

To see a more advanced version take a look at this example

Cron

With kjob you are also able to schedule jobs with the familiar cron expression. To get Kron - the name of the extension to enable Cron scheduling in kjob - you need to add the following dependency:

dependencies {
  implementation "it.justwrote:kjob-kron:<version>"
}

After that you can schedule cron jobs as easy as every other job with kjob.

// define a Kron job with a name and a cron expression (e.g. 5 seconds)
object PrintStuff : KronJob("print-stuff", "*/5 * * ? * * *")

// ...

val kjob = kjob(InMem) {
    extension(KronModule) // enable the Kron extension
}

// ...

// define the executed code
kjob(Kron).kron(PrintStuff) {
    maxRetries = 3 // and you can access the already familiar settings you are used to
    execute {
        println("${Instant.now()}: executing kron task '${it.name}' with jobId '$jobId'")
    }
}

You can find more in this example

Roadmap

Here is an unordered list of features that I would like to see in kjob. If you consider one of them important please open an issue.

  • Priority support
  • Backoff algorithm for failed jobs
  • REST API
  • Dashboard

License

kjob is licensed under the Apache 2.0 License.

You might also like...
Kotools Assert is a lightweight assertions library for Kotlin

Kotools Assert is a lightweight assertions library for Kotlin

A simple, lightweight, non-bloated redis client for kotlin and other JVM languages

rekt is a lightweight, non-bloated redis client, primarily written for the kotlin programming language, while also supporting other JVM-based languages, such as Java, Scala, and obviously way more.

Depenject - a lightweight, minimalistic dependency injection library for Kotlin/JVM.

depenject depenject is a lightweight, minimalistic dependency injection library for Kotlin/JVM. Our goal is similar to flavor's to simplify the usage

A lightweight Kotlin library which converts images to PixelArt in Android.
A lightweight Kotlin library which converts images to PixelArt in Android.

PixelArtImageView PixelArtImageView is a lightweight library used to convert standalone images into pixel art. The view extends ImageView behaviour an

A pragmatic lightweight dependency injection framework for Kotlin developers.
A pragmatic lightweight dependency injection framework for Kotlin developers.

A pragmatic lightweight dependency injection framework for Kotlin developers. Koin is a DSL, a light container and a pragmatic API

A lightweight Trakt API.

This library is a lightweight Trakt API. It supports Swift, Kotlin, and JavaScript by setting up as a Kotlin Multiplatform project.

A lightweight library for requesting and consuming Activity Results using coroutines.

SuspendActivityResult A lightweight library for requesting and consuming Activity Results using coroutines, it's usage is as simple as: val uri = Acti

A simple, lightweight and powerful field validation library for Android.
A simple, lightweight and powerful field validation library for Android.

Convalida Convalida - (Italian for "validation") Convalida is a simple, lightweight and powerful field validation library for Android. Documentation G

Comments
  • InMem, missing docs?

    InMem, missing docs?

    Correct me if I'm wrong, but I don't see anything mentioned that InMem is not apart of the core libary and needs to be added to your dependcies list (ie, implementation "it.justwrote:kjob-inmem:version" ), just letting you know, so people don't get stuck and give up, and miss out on this amazing libary 😅

    opened by Yofou 1
  • only support UTC time?

    only support UTC time?

    when i add a kron job, cronExpression like this: "0 0 1 * * ?",it should have started at 1:00 every day,but not. I checked the code,I found this

    class InMemKJob(config: Configuration) : BaseKJob<InMemKJob.Configuration>(config) {
    
        class Configuration : BaseKJob.Configuration() {
            /**
             * The timeout until a kjob instance is considered dead if no 'I am alive' notification occurred
             */
            var expireLockInMinutes = 5L
        }
    
        override val jobRepository: JobRepository = InMemJobRepository(Clock.systemUTC())
    
        override val lockRepository: LockRepository = InMemLockRepository(config, Clock.systemUTC())
    }
    

    It seems to only be supported UTC time, Can I change the time zone?

    opened by fantasyding 0
  • Add option to configure periods in milliseconds

    Add option to configure periods in milliseconds

    For example, you anyway convert the jobExecutionPeriodInSeconds to milliseconds, so why not provide user a possibility to configure jobExecutionPeriod, cleanupPeriod, etc. in ms?

    https://github.com/justwrote/kjob/blob/880b175516ce5f3e9edbb0b6fe7b50820c332026/kjob-core/src/main/kotlin/it/justwrote/kjob/BaseKJob.kt#L116

    opened by attebjorner 0
Releases(0.2.0)
Owner
null
Kreds - a thread-safe, idiomatic, coroutine based Redis client written in 100% Kotlin

Kreds Kreds is a thread-safe, idiomatic, coroutine based Redis client written in 100% Kotlin. Why Kreds? Kreds is designed to be EASY to use. Kreds ha

Abhijith Shivaswamy 117 Dec 23, 2022
🚀 🥳 MVVM based sample currency converter application using Room, Koin, ViewModel, LiveData, Coroutine

Currency Converter A demo currency converter app using Modern Android App Development techniques Tech stack & Open-source libraries Minimum SDK level

Abinash Neupane 2 Jul 17, 2022
New Relic Kotlin Instrumentation for Kotlin Coroutine. It successfully handles thread changes in suspend states.

new-relic-kotlin-coroutine New Relic Kotlin Instrumentation for Kotlin Coroutine. It successfully handles thread changes in suspend states. Usage 1- U

Mehmet Sezer 7 Nov 17, 2022
Android Multi Theme Switch Library ,use kotlin language ,coroutine ,and so on ...

Magic Mistletoe Android多主题(换肤)切换框架 背景 时隔四年,在网易换肤之前的思路下,做了几点改进,现在完全通过反射创建View,并且在SkinLoadManager中提供一个configCustomAttrs以支持自定义View的属性插队替换 摈弃了之前的AsyncTask

Mistletoe 18 Jun 17, 2022
R2DBC Sharding Example with Kotlin Coroutine

R2DBC Sharding Example with Kotlin Coroutine A minimal sharding example with R2DBC and coroutine, where user table is divided into 2 different shards.

K.S. Yim 0 Oct 4, 2021
Demonstration of Object Pool Design Pattern using Kotlin language and Coroutine

Object Pool Design Pattern with Kotlin Demonstration of Thread Safe Object Pool Design Pattern using Kotlin language and Coroutine. Abstract The objec

Enes Kayıklık 7 Apr 12, 2022
MVVM ,Hilt DI ,LiveData ,Flow ,SharedFlow ,Room ,Retrofit ,Coroutine , Navigation Component ,DataStore ,DataBinding , ViewBinding, Coil

RickMorty This is a simple app which has been implemented using Clean Architecture alongside MVVM design to run (online/offline) using : [ MVVM ,Hilt

Ali Assalem 13 Jan 5, 2023
Spring Boot project scaffold written in Kotlin, which is based on the Official Guide.

Kotlin-Spring-Boot Spring Boot project scaffold written in Kotlin, which is based on the Official Guide. Development environment Windows choco install

idea2app 1 Feb 27, 2022
YAML-based source-based kotlin module descriptors

kproject - Liberate your Kotlin projects YAML-based source-based kotlin module descriptors that runs on top of gradle. Define your kotlin multiplatfor

KorGE Game Engine & Korlibs 16 Jan 31, 2023
A lightweight tool for managing and building Kotlin projects.

kpm kpm (Kotlin Project Manager) is a lightweight tool for managing and building Kotlin projects. What is kpm? Essentially, kpm is going to be a light

Conor Byrne 5 Apr 23, 2022