General purpose utilities and hash functions for Android and Java (aka java-common)

Last update: Jun 15, 2022

Essentials

Build Status

Essentials are a collection of general-purpose classes we found useful in many occasions.

  • Beats standard Java API performance, e.g. LongHashMap can be twice as fast as HashMap<Long, Object>.
  • Adds missing pieces without pulling in heavy-weights like Guava
  • Improved convenience: do more with less code
  • Super lightweight: < 100k in size
  • Compatible with Android and Java

This project is bare bones compared to a rich menu offered by Guava or Apache Commons. Essentials is not a framework, it's rather a small set of utilities to make Java standard approaches more convenient or more efficient.

Website | JavaDoc | Changelog

Features

  • Hash set and map for primitive long keys outperform the generic versions of the Java Collection APIs
  • Multimaps provide a map of lists or sets to simplify storing multiple values for a single key
  • Object cache with powerful configuration options: soft/weak/strong references, maximum size, and time-based expiration
  • IO utilities help with streams (byte and character based)
  • File utilities simplify reading and writing strings/bytes/objects from or to files. Also includes getting hashes from files and copying files.
  • String utilities allow efficient splitting and joining of strings, fast hex creation, and other useful string helpers.
  • Date utilities
  • Better hash functions: our Murmur3 implementation provides superior hash quality and outperforms standard Java hash functions
  • Specialized Streams: for example an optimized PipedOutputStream replacement (based on a circular byte buffer)

Read more on our website.

Performance

Some classes where motivated by less than optimal performance offered by standard Java.

For long keys (also works for int), Essentials provides a specialized implementation, that can be twice as fast.

Here are some (completely non-scientific) benchmarking results running on Ubuntu 20.04 LTS using OpenJDK 11.0.9:

Essentials Class Java (seconds) Essentials (seconds) Speed up
LongHashSet (Dynamic) 19.756 13.079 + 51%
LongHashSet (Prealloc) 16.480 8.171 + 102%
LongHashMap (Dynamic) 20.311 14.659 + 39%
LongHashMap (Prealloc) 17.496 8.677 + 102%
PipelineStream (1024KB) 8.036 1.424 + 564%
StringHex (vs. Guava) 6.849 3.732 + 84%

The benchmarking sources are available in the java-essentials-performance directory.

Add the dependency to your project

For Gradle, you add this dependency (from repository mavenCentral()):

implementation 'org.greenrobot:essentials:3.1.0'

And for Maven:

<dependency>
    <groupId>org.greenrobot</groupId>
    <artifactId>essentials</artifactId>
    <version>3.1.0</version>
</dependency>

Code samples

Example code for some of the utility classes:

// Get all bytes from stream and close the stream safely
byte[] bytes = IoUtils.readAllBytesAndClose(inputStream);

// Read the contents of an file as a string (use readBytes to get byte[])
String contents = FileUtils.readUtf8(file);

// How many days until new year's eve?
long time2 = DateUtils.getTimeForDay(2015, 12, 31);
int daysToNewYear = DateUtils.getDayDifference(time, time2);

Multimaps:

ListMap<String,String> multimap = new ListMap<>();
multimap.putElement("a", "1");
multimap.putElement("a", "2");
multimap.putElement("a", "3");
List<String> strings = multimap.get("a"); // Contains "1", "2", and "3"

Our hash functions implement java.util.zip.Checksum, so this code might look familiar to you:

Murmur3A murmur = new Murmur3A();
murmur.update(bytes);
long hash = murmur.getValue();

All hashes can be calculated progressively by calling update(...) multiple times. Our Murmur3A implementation goes a step further by offering updates with primitive data in a very efficient way:

// reuse the previous instance and start over to calculate a new hash
murmur.reset();

murmur.updateLong(42L);

// Varargs and arrays are supported natively, too  
murmur.updateInt(2014, 2015, 2016);

// Hash for the previous update calls. No conversion to byte[] necessary.
hash = murmur.getValue();

The utility classes are straight forward and don't have dependencies, so you should be fine to grasp them by having a look at their source code. Most of the method names should be self-explaining, and often you'll find JavaDocs where needed.

Build setup

We use Gradle as a primary build system. Previously, Maven is used to build greenrobot-common. Inside of build-common, there are two parent POMs defined that might be useful: parent-pom and parent-pom-with-checks. The latter integrates FindBugs and Checkstyle in your build. Use it like this:

<parent>
    <groupId>de.greenrobot</groupId>
    <artifactId>parent-pom-with-checks</artifactId>
    <version>2.0.0</version>
    <relativePath></relativePath>
</parent>

License

Copyright (C) 2012-2020 Markus Junginger, greenrobot (https://greenrobot.org)

EventBus binaries and source code can be used according to the Apache License, Version 2.0.

More by greenrobot

EventBus is a central publish/subscribe bus for Android with optional delivery threads, priorities, and sticky events. A great tool to decouple components (e.g. Activities, Fragments, logic components) from each other.

ObjectBox super-fast object database.

GitHub

https://github.com/greenrobot/essentials
Comments
  • 1. Murmur 3F collision

    I might not be using the api correctly, but we're getting collisons for the following case:

    var m1 = new Murmur3F();
    m1.update(0);
    m1.update(-17664);
    
    var m2 = new Murmur3F();
    m2.update(0);
    m2.update(-16640);
    
    assert !Objects.equals(new UUID(m1.getValueHigh(), m1.getValue()), new UUID(m2.getValueHigh(), m2.getValue()));
    

    Do we need a finalize or something?

    Reviewed by paplorinc at 2020-07-10 07:00
  • 2. An illegal reflective access operation has occurred

    JDK 11

    WARNING: An illegal reflective access operation has occurred
    WARNING: Illegal reflective access by org.greenrobot.essentials.PrimitiveArrayUtils$UnsafeImpl (file:/<REDACTED>/essentials-3.0.0-RC1.jar) to method java.nio.Bits.unaligned()
    WARNING: Please consider reporting this to the maintainers of org.greenrobot.essentials.PrimitiveArrayUtils$UnsafeImpl
    WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
    WARNING: All illegal access operations will be denied in a future release
    
    Reviewed by greenrobot-team at 2019-01-08 07:10
  • 3. Make fields in CircularByteBuffer protected

    I want to extend its functionality (like, direct access to Buffers without copying byte arrays twice), but its fields are private. This forces me to make a copy of the class just to add a few new methods instead of being able to extend it. Can you please change this?

    Reviewed by piegamesde at 2018-04-19 15:57
  • 4. Murmur3A.updateUtf8 not exists

    Trying to use Murmur implementation from this library, and got the error for:

            Murmur3A murmur = new Murmur3A();
            murmur.updateUtf8("My test string");
    

    error: cannot find symbol method updateUtf8(String)

    Tried with both versions: org.greenrobot:essentials:3.0.0-RC1 and de.greenrobot:java-common:2.3.1

    Reviewed by Kostanos at 2019-04-12 17:01
  • 5. Large File MurMurHash is much more slower than SHA1 (Android)

    Test file size: 1.09GB

    Here is the code:

        try {
            Log.d(TAG, "SHA1");
            Log.d(TAG, FileUtils.getSha1(file));
        } catch (IOException e) {
            e.printStackTrace();
        }
        try {
            Murmur3F checksum = new Murmur3F(123);
            Log.d(TAG, "murmur");
            FileUtils.updateChecksum(file, checksum);
            Log.d(TAG, "" + checksum.getValueHexString());
        } catch (IOException e) {
            e.printStackTrace();
        }
    

    Here is the result: 09-20 18:18:53.021 D/HashTest: SHA1 09-20 18:19:12.721 D/HashTest: a21321f03c0463927f51f2febaf2ea1e66acff21 09-20 18:19:12.731 D/HashTest: murmur 09-20 18:19:47.951 D/HashTest: 6e838ff35a6d624310df1607cc5892f6

    SHA1: ~20s MurmurHash: ~40s

    Is this result reasonable?

    Reviewed by MatoMA at 2016-09-20 10:28
  • 6. Question: Required java version

    Question: What is the required minimum java version of the project? Officially, unofficially? It's easy enough to test, but is there a decision regarding this? Looking into using it in an embedded environment where some hardware still only has java 1.5. Looks like the collections, dates, files etc can manage with only 1.5. At least not any 1.7. sun.misc.Unsafe might be an issue but isolated use of that.

    Reviewed by mattiasbe at 2016-02-24 18:56
  • 7. Issues when building with proguard (3.0.0-RC1)

    Building with proguard enabled would result in this error: PrimitiveArrayUtils$UnsafeImpl: can't find referenced class sun.misc.Unsafe

    adding the follow to my proguard configs seemed to help: -dontwarn sun.misc.Unsafe

    Reviewed by mhiew at 2017-08-09 19:00
  • 8. LinkedListMultimap

    Why not use LinkedHashMap for the key to preserve order when iterating through the map ?

    Or provide a LinkedListMultimap class like

    public class LinkedListMultimap<K, V> extends AbstractMultimap<K, V, List<V>> {
    	public static <K, V> LinkedListMultimap<K, V> create() {
    		return new LinkedListMultimap<>(new LinkedHashMap<K, List<V>>());
    	}
    
    	protected LinkedListMultimap(LinkedHashMap<K, List<V>> map) {
    		super(map);
    	}
    
    	@Override protected List<V> createNewCollection() {
    		return new ArrayList<>();
    	}
    }
    

    I'd be happy to make a PR with this as I'm using this in my own project with great success

    Reviewed by slott at 2016-11-11 23:47
  • 9. reference links are broken. documentation link missing.

    Most of the link in README.md section is broken. Like: stream-based IO

    Your website says:

    For more details, please have a look at the documentation.

    But I can't find a visible documentation link anywhere.

    Reviewed by oronno at 2016-05-09 15:27
  • 10. No subscribers for service

    I post EventBus.getDefault().post(new SendPlayer(player)); from a services, which is running in non main thread

    <service
            android:name=".player.PlayerService"
            android:process=":player"
            android:enabled="true"
            android:exported="true">
        </service>
    

    I expect to recieve even in my fragment

      @Subscribe(threadMode = ThreadMode.MAIN) public void onEvent(SendPlayer event) {
        Log.w("mcheck", "onEvent");
      }
    

    However, i get message D/EventBus: No subscribers registered for event class yarh.com.tryexo.player.SendPlayer. Events are delivered only if i remove android:process=":player". Is it a bug or I misunderstud flow of posting events between bachground thread and main thread?

    Reviewed by frakc at 2016-04-15 09:42
  • 11. Create InputReader

    Using this class, user will be able to directly read integer, long, string, character datatype. it is faster than Scanner class of java.io. In Java programming language, input output stream takes much time than other language. so if we are able to make library which is efficient for input and output, that would be great.

    Reviewed by Bhavik3 at 2014-12-09 08:59
  • 12. ObjectCache should use phone uptime instead of time

    Currently ObjectCache depends on the system time never changing, which is not optimal in a setup where the app runs for a long time, the user or the system might change the time.

    a solution would be not to use the time but the uptime of the phone.

    https://developer.android.com/reference/android/os/SystemClock#uptimeMillis()

    Reviewed by andrew-ld at 2021-11-05 10:52
Related tags
A general purpose compiler heart

Jamtree A general purpose compiler heart. This is a component of the compiler of the jamplate programming language.

Mar 21, 2022
FractalUtils - A collection of utility functions and classes, with an emphasis on game related utilities

A collection of utility functions and classes written in Kotlin. There is some emphasis on utilities useful for games (Geometry, Random, Time, Updating, etc).

Mar 28, 2022
[Deprecated] AKA VectorDrawableCompat: A 7+ backport of VectorDrawable
[Deprecated] AKA VectorDrawableCompat: A 7+ backport of VectorDrawable

###@Deprecated Unfortunatenly this library is no longer maintained, we encourage you to use first party VectorDrawableCompat coming soon to support li

Jun 3, 2022
Collection of source codes, utilities, templates and snippets for Android development.

Android Templates and Utilities [DEPRECATED] Android Templates and Utilities are deprecated. I started with this project in 2012. Android ecosystem ha

May 30, 2022
Utilities I wish Android had but doesn't

wishlist Library of helpers and utilities that I wish were included in the Android SDK but aren't. If you think something in this library is already h

Jul 2, 2021
Various useful utilities for Android apps development

Android Commons Various useful utilities for Android apps development. API documentation provided as Javadoc. Usage Add dependency to your build.gradl

May 8, 2021
Android library that regroup bunch of dateTime utilities

DateTimeUtils This library is a package of functions that let you manipulate objects and or java date string. it combine the most common functions use

Apr 24, 2022
CreditCardHelper 🖊️ A Jetpack-Compose library providing useful credit card utilities such as card type recognition and TextField ViewTransformations
CreditCardHelper 🖊️  A Jetpack-Compose library providing useful credit card utilities such as card type recognition and TextField ViewTransformations

CreditCardHelper ??️ A Jetpack-Compose library providing useful credit card utilities such as card type recognition and TextField ViewTransformations

Jun 5, 2022
A Telegram bot utilities that help to reduce the code amount

Flume Party A Telegram bot utilities that help to reduce code amount. Real project examples Pull Party Bot: 19% of code has been reduced. Resistance B

Jun 8, 2022
A set of lint rules to check for common mistakes when styling and theming on Android
A set of lint rules to check for common mistakes when styling and theming on Android

A set of lint rules to check for common mistakes when styling and theming on Android

Aug 29, 2021
Native solution for common React Native problem of focused views being covered by soft input view.

react-native-avoid-softinput Native solution for common React Native problem of focused views being covered by soft input view. It is solved by listen

Jun 15, 2022
A collection of small utility functions to make it easier to deal with some otherwise nullable APIs on Android.

requireKTX is a collection of small utility functions to make it easier to deal with some otherwise nullable APIs on Android, using the same idea as requireContext, requireArguments, and other similar Android SDK methods.

Apr 30, 2022
Catch `dd`, `ddd`, `dump`, `sleep` and `ray` functions in your code

catch-debug-code Template ToDo list Create a new IntelliJ Platform Plugin Template project. Get familiar with the template documentation. Verify the p

Apr 29, 2022
λRPC allows using code with high-order functions as a service
λRPC allows using code with high-order functions as a service

λRPC Simple native RPC with high order functions support. Inspired by @altavir and Communicator. λRPC allows using code with high-order functions as a

May 18, 2022
WebSocket & WAMP in Java for Android and Java 8

Autobahn|Java Client library providing WAMP on Java 8 (Netty) and Android, plus (secure) WebSocket for Android. Autobahn|Java is a subproject of the A

Jun 10, 2022
WebSocket & WAMP in Java for Android and Java 8

Autobahn|Java Client library providing WAMP on Java 8 (Netty) and Android, plus (secure) WebSocket for Android. Autobahn|Java is a subproject of the A

Jun 10, 2022
gRPC and protocol buffers for Android, Kotlin, and Java.

Wire “A man got to have a code!” - Omar Little See the project website for documentation and APIs. As our teams and programs grow, the variety and vol

Jun 25, 2022
Trail is a simple logging system for Java and Android. Create logs using the same API and the library will detect automatically in which platform the code is running.

Trail Trail is a simple logging system for Java and Android. Create logs using the same API and the library will detect automatically in which platfor

Apr 8, 2019