PufferDB
PufferDB is a
The purpose of this library is to provide an efficient, reliable and Android independent storage.
Why Android independent? The SharedPreferences and many great third-party libraries (like Paper and MMKV) requires the Android Context to work. But if you are like me and want a kotlin-only data module (following the principles of Clean Architecture), this library is for you!
This project started as a library module in one of my personal projects, but I decided to open source it and add more features for general use. Hope you like!
About Protobuf
Protocol Buffers are a language-neutral, platform-neutral extensible mechanism for serializing structured data. Compared to JSON, Protobuf files are smaller and faster to read/write because the data is stored in an efficient binary format.
Features
- Fast
- Works on Android and JVM
- Simple API
- Thread-safe
- Wrappers for Coroutines and RxJava
Supported types
So far, PufferDB supports the following types:
-
DoubleandList<Double> -
FloatandList<Float> -
IntandList<Int> -
LongandList<Long> -
BooleanandList<Boolean> -
StringandList<String>
Getting Started
Import to your project
- Add the JitPack repository in your root build.gradle at the end of repositories:
allprojects {
repositories {
maven { url 'https://jitpack.io' }
}
}
- Next, add the desired dependencies to your module:
dependencies {
// Core library
implementation "com.github.adrielcafe.pufferdb:core:$currentVersion"
// Android helper
implementation "com.github.adrielcafe.pufferdb:android:$currentVersion"
// Coroutines wrapper
implementation "com.github.adrielcafe.pufferdb:coroutines:$currentVersion"
// RxJava wrapper
implementation "com.github.adrielcafe.pufferdb:rxjava:$currentVersion"
}
Current version:
Platform compatibility
core |
android |
coroutines |
rxjava |
|
|---|---|---|---|---|
| Android | ✓ | ✓ | ✓ | ✓ |
| JVM | ✓ | ✓ | ✓ |
Core
As the name suggests, Core is a standalone module and all other modules depends on it.
To create a new Puffer instance you must tell which file to use.
val pufferFile = File("path/to/puffer/file")
val puffer = PufferDB.with(pufferFile)
// or
val puffer = PufferDB.with(pufferFile, myCoroutineScope, myCoroutineDispatcher)
If you are on Android, I recommend to use the Context.filesDir as the parent folder. If you want to save in the external storage remember to ask for write permission first.
Its API is similar to SharedPreferences:
puffer.apply {
val myValue = get<String>("myKey")
val myValueWithDefault = get("myKey", "defaultValue")
put("myOtherKey", 123)
getKeys().forEach { key ->
// ...
}
if(contains("myKey")){
// ...
}
remove("myOtherKey")
removeAll()
}
But unlike SharedPreferences, there's no apply() or commit(). Changes are saved asynchronously every time a write operation (put(), remove() and removeAll()) happens.
Threading
PufferDB uses a ConcurrentHashMap to manage a thread-safe in-memory cache for fast read and write operations.
Changes are saved asynchronously with the help of a StateFlow (to save the most recent state in a race condition) and Mutex locker (to prevent simultaneous writes).
It is possible to run the API methods on the Android Main Thread, but you should avoid that. You can use one of the wrapper modules or built in extension functions for that (listed below).
Android
The Android module contains an AndroidPufferDB helper class:
class MyApp : Application() {
override fun onCreate() {
super.onCreate()
// Init the PufferDB when your app starts
AndroidPufferDB.init(this)
}
}
// Now you can use it anywhere in your app
class MyActivity : AppCompatActivity() {
// Returns a default Puffer instance, the file is located on Context.filesDir
val corePuffer = AndroidPufferDB.withDefault()
// Returns a File that should be used to create a Puffer instance
val pufferFile = AndroidPufferDB.getInternalFile("my.db")
val coroutinePuffer = CoroutinePufferDB.with(pufferFile)
}
Coroutines
The Coroutines module contains a CoroutinePufferDB wrapper class and some useful extension functions:
val pufferFile = File("path/to/puffer/file")
val puffer = CoroutinePufferDB.with(pufferFile)
// All methods are suspend functions that runs on Dispatchers.IO context
launch {
puffer.apply {
val myValue = get<String>("myKey")
val myValueWithDefault = get("myKey", "defaultValue")
put("myOtherKey", 123)
getKeys().forEach { key ->
// ...
}
if(contains("myKey")){
// ...
}
remove("myOtherKey")
removeAll()
}
}
If you don't want to use this wrapper class, there's some built in extension functions that can be used with the Core module:
val pufferFile = File("path/to/puffer/file")
val puffer = PufferDB.with(pufferFile) // <- Note that we're using the Core PufferDB
launch {
puffer.apply {
val myValue = getSuspend<String>("myKey")
val myValue = getAsync<String>("myKey").await()
// You can use your own coroutine scope and dispatcher
putSuspend("myOtherKey", 123, myCoroutineScope, myCoroutineDispatcher)
putAsync("myOtherKey", 123, myActivityScope).await()
}
}
RxJava
The RxJava module contains a RxPufferDB wrapper class and some useful extension functions:
val pufferFile = File("path/to/puffer/file")
val puffer = RxPufferDB.with(pufferFile)
puffer.apply {
// Some methods returns Single<T>...
get<String>("myKey") // OR get("myKey", "defaultValue")
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe { myValue ->
// ...
}
// ... And others returns Completable
put("myOtherKey", 123)
// ...
.subscribe {
// ...
}
getKeys()
// ...
.subscribe { keys ->
// ...
}
contains("myKey")
// ...
.subscribe { contains ->
// ...
}
remove("myOtherKey")
// ...
.subscribe {
// ...
}
removeAll()
// ...
.subscribe {
// ...
}
}
Like the Coroutines module, the RxJava module also provides some useful built in extension functions that can be used with the Core module:
val pufferFile = File("path/to/puffer/file")
val puffer = PufferDB.with(pufferFile) // <- Note that we're using the Core PufferDB
puffer.apply {
val myValue = getSingle<String>("myKey").blockingGet()
putCompletable("myOtherKey", 123).blockingAwait()
getKeysObservable().blockingSubscribe { keys ->
// ...
}
}
Benchmark
Write & Read
| Write 1k strings (ms) | Read 1k strings (ms) | |
|---|---|---|
| PufferDB | 20 | 5 |
| SharedPreferences | 278 | 7 |
| MMKV | 13 | 8 |
| Paper | 818 | 169 |
| Binary Prefs | 121 | 9 |
| Hawk | 15183 | 207 |
| Write 100k strings (ms) | Read 100k strings (ms) | |
|---|---|---|
| PufferDB | 259 | 32 |
| SharedPreferences |
|
|
| MMKV | 871 | 516 |
| Paper |
|
|
| Binary Prefs | 1082 | 101 |
| Hawk |
|
|
File size
| 1k strings (kb) | |
|---|---|
| PufferDB | 25 |
| SharedPreferences | 20 |
| MMKV | 40 |
| Paper | 61 |
| Binary Prefs | 53 |
| Hawk | 27 |
| 100k strings (kb) | |
|---|---|
| PufferDB | 2.907 |
| SharedPreferences |
|
| MMKV | 4.104 |
| Paper |
|
| Binary Prefs | 5.175 |
| Hawk |
|
Tested on Moto Z2 Plus
You can run the Benchmark through the sample app.
8 Nov 26, 2022
20 Oct 30, 2022
1.6k Jan 8, 2023
9 Oct 22, 2022
8 Dec 1, 2022
132 Jan 8, 2023
1 Jun 30, 2022
0 Dec 19, 2021
1 Feb 13, 2022
35 Oct 4, 2022
4 Nov 10, 2022
2 Nov 7, 2022
3.4k Jan 3, 2023
130 Jan 7, 2023
2.8k Dec 30, 2022
7 Sep 17, 2022
14 Nov 5, 2022