General purpose utilities and hash functions for Android and Java (aka java-common)

Related tags

Utility essentials
Overview

Essentials

Build Status

Essentials are a collection of general-purpose classes we found useful in many occasions.

  • Beats standard Java API performance, e.g. LongHashMap can be twice as fast as HashMap<Long, Object>.
  • Adds missing pieces without pulling in heavy-weights like Guava
  • Improved convenience: do more with less code
  • Super lightweight: < 100k in size
  • Compatible with Android and Java

This project is bare bones compared to a rich menu offered by Guava or Apache Commons. Essentials is not a framework, it's rather a small set of utilities to make Java standard approaches more convenient or more efficient.

Website | JavaDoc | Changelog

Features

  • Hash set and map for primitive long keys outperform the generic versions of the Java Collection APIs
  • Multimaps provide a map of lists or sets to simplify storing multiple values for a single key
  • Object cache with powerful configuration options: soft/weak/strong references, maximum size, and time-based expiration
  • IO utilities help with streams (byte and character based)
  • File utilities simplify reading and writing strings/bytes/objects from or to files. Also includes getting hashes from files and copying files.
  • String utilities allow efficient splitting and joining of strings, fast hex creation, and other useful string helpers.
  • Date utilities
  • Better hash functions: our Murmur3 implementation provides superior hash quality and outperforms standard Java hash functions
  • Specialized Streams: for example an optimized PipedOutputStream replacement (based on a circular byte buffer)

Read more on our website.

Performance

Some classes where motivated by less than optimal performance offered by standard Java.

For long keys (also works for int), Essentials provides a specialized implementation, that can be twice as fast.

Here are some (completely non-scientific) benchmarking results running on Ubuntu 20.04 LTS using OpenJDK 11.0.9:

Essentials Class Java (seconds) Essentials (seconds) Speed up
LongHashSet (Dynamic) 19.756 13.079 + 51%
LongHashSet (Prealloc) 16.480 8.171 + 102%
LongHashMap (Dynamic) 20.311 14.659 + 39%
LongHashMap (Prealloc) 17.496 8.677 + 102%
PipelineStream (1024KB) 8.036 1.424 + 564%
StringHex (vs. Guava) 6.849 3.732 + 84%

The benchmarking sources are available in the java-essentials-performance directory.

Add the dependency to your project

For Gradle, you add this dependency (from repository mavenCentral()):

implementation 'org.greenrobot:essentials:3.1.0'

And for Maven:

<dependency>
    <groupId>org.greenrobot</groupId>
    <artifactId>essentials</artifactId>
    <version>3.1.0</version>
</dependency>

Code samples

Example code for some of the utility classes:

// Get all bytes from stream and close the stream safely
byte[] bytes = IoUtils.readAllBytesAndClose(inputStream);

// Read the contents of an file as a string (use readBytes to get byte[])
String contents = FileUtils.readUtf8(file);

// How many days until new year's eve?
long time2 = DateUtils.getTimeForDay(2015, 12, 31);
int daysToNewYear = DateUtils.getDayDifference(time, time2);

Multimaps:

ListMap<String,String> multimap = new ListMap<>();
multimap.putElement("a", "1");
multimap.putElement("a", "2");
multimap.putElement("a", "3");
List<String> strings = multimap.get("a"); // Contains "1", "2", and "3"

Our hash functions implement java.util.zip.Checksum, so this code might look familiar to you:

Murmur3A murmur = new Murmur3A();
murmur.update(bytes);
long hash = murmur.getValue();

All hashes can be calculated progressively by calling update(...) multiple times. Our Murmur3A implementation goes a step further by offering updates with primitive data in a very efficient way:

// reuse the previous instance and start over to calculate a new hash
murmur.reset();

murmur.updateLong(42L);

// Varargs and arrays are supported natively, too  
murmur.updateInt(2014, 2015, 2016);

// Hash for the previous update calls. No conversion to byte[] necessary.
hash = murmur.getValue();

The utility classes are straight forward and don't have dependencies, so you should be fine to grasp them by having a look at their source code. Most of the method names should be self-explaining, and often you'll find JavaDocs where needed.

Build setup

We use Gradle as a primary build system. Previously, Maven is used to build greenrobot-common. Inside of build-common, there are two parent POMs defined that might be useful: parent-pom and parent-pom-with-checks. The latter integrates FindBugs and Checkstyle in your build. Use it like this:

<parent>
    <groupId>de.greenrobot</groupId>
    <artifactId>parent-pom-with-checks</artifactId>
    <version>2.0.0</version>
    <relativePath></relativePath>
</parent>

License

Copyright (C) 2012-2020 Markus Junginger, greenrobot (https://greenrobot.org)

EventBus binaries and source code can be used according to the Apache License, Version 2.0.

More by greenrobot

EventBus is a central publish/subscribe bus for Android with optional delivery threads, priorities, and sticky events. A great tool to decouple components (e.g. Activities, Fragments, logic components) from each other.

ObjectBox super-fast object database.

Comments
  • Murmur 3F collision

    Murmur 3F collision

    I might not be using the api correctly, but we're getting collisons for the following case:

    var m1 = new Murmur3F();
    m1.update(0);
    m1.update(-17664);
    
    var m2 = new Murmur3F();
    m2.update(0);
    m2.update(-16640);
    
    assert !Objects.equals(new UUID(m1.getValueHigh(), m1.getValue()), new UUID(m2.getValueHigh(), m2.getValue()));
    

    Do we need a finalize or something?

    opened by paplorinc 5
  • An illegal reflective access operation has occurred

    An illegal reflective access operation has occurred

    JDK 11

    WARNING: An illegal reflective access operation has occurred
    WARNING: Illegal reflective access by org.greenrobot.essentials.PrimitiveArrayUtils$UnsafeImpl (file:/<REDACTED>/essentials-3.0.0-RC1.jar) to method java.nio.Bits.unaligned()
    WARNING: Please consider reporting this to the maintainers of org.greenrobot.essentials.PrimitiveArrayUtils$UnsafeImpl
    WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
    WARNING: All illegal access operations will be denied in a future release
    
    opened by greenrobot-team 5
  • Make fields in CircularByteBuffer protected

    Make fields in CircularByteBuffer protected

    I want to extend its functionality (like, direct access to Buffers without copying byte arrays twice), but its fields are private. This forces me to make a copy of the class just to add a few new methods instead of being able to extend it. Can you please change this?

    opened by piegamesde 4
  • Murmur3A.updateUtf8 not exists

    Murmur3A.updateUtf8 not exists

    Trying to use Murmur implementation from this library, and got the error for:

            Murmur3A murmur = new Murmur3A();
            murmur.updateUtf8("My test string");
    

    error: cannot find symbol method updateUtf8(String)

    Tried with both versions: org.greenrobot:essentials:3.0.0-RC1 and de.greenrobot:java-common:2.3.1

    opened by Kostanos 3
  • Large File MurMurHash is much more slower than SHA1 (Android)

    Large File MurMurHash is much more slower than SHA1 (Android)

    Test file size: 1.09GB

    Here is the code:

        try {
            Log.d(TAG, "SHA1");
            Log.d(TAG, FileUtils.getSha1(file));
        } catch (IOException e) {
            e.printStackTrace();
        }
        try {
            Murmur3F checksum = new Murmur3F(123);
            Log.d(TAG, "murmur");
            FileUtils.updateChecksum(file, checksum);
            Log.d(TAG, "" + checksum.getValueHexString());
        } catch (IOException e) {
            e.printStackTrace();
        }
    

    Here is the result: 09-20 18:18:53.021 D/HashTest: SHA1 09-20 18:19:12.721 D/HashTest: a21321f03c0463927f51f2febaf2ea1e66acff21 09-20 18:19:12.731 D/HashTest: murmur 09-20 18:19:47.951 D/HashTest: 6e838ff35a6d624310df1607cc5892f6

    SHA1: ~20s MurmurHash: ~40s

    Is this result reasonable?

    opened by MatoMA 3
  • Question: Required java version

    Question: Required java version

    Question: What is the required minimum java version of the project? Officially, unofficially? It's easy enough to test, but is there a decision regarding this? Looking into using it in an embedded environment where some hardware still only has java 1.5. Looks like the collections, dates, files etc can manage with only 1.5. At least not any 1.7. sun.misc.Unsafe might be an issue but isolated use of that.

    opened by mattiasbe 3
  • Issues when building with proguard (3.0.0-RC1)

    Issues when building with proguard (3.0.0-RC1)

    Building with proguard enabled would result in this error: PrimitiveArrayUtils$UnsafeImpl: can't find referenced class sun.misc.Unsafe

    adding the follow to my proguard configs seemed to help: -dontwarn sun.misc.Unsafe

    opened by mhiew 1
  • LinkedListMultimap

    LinkedListMultimap

    Why not use LinkedHashMap for the key to preserve order when iterating through the map ?

    Or provide a LinkedListMultimap class like

    public class LinkedListMultimap<K, V> extends AbstractMultimap<K, V, List<V>> {
    	public static <K, V> LinkedListMultimap<K, V> create() {
    		return new LinkedListMultimap<>(new LinkedHashMap<K, List<V>>());
    	}
    
    	protected LinkedListMultimap(LinkedHashMap<K, List<V>> map) {
    		super(map);
    	}
    
    	@Override protected List<V> createNewCollection() {
    		return new ArrayList<>();
    	}
    }
    

    I'd be happy to make a PR with this as I'm using this in my own project with great success

    opened by slott 1
  • reference links are broken. documentation link missing.

    reference links are broken. documentation link missing.

    Most of the link in README.md section is broken. Like: stream-based IO

    Your website says:

    For more details, please have a look at the documentation.

    But I can't find a visible documentation link anywhere.

    opened by oronno 1
  • No subscribers for  service

    No subscribers for service

    I post EventBus.getDefault().post(new SendPlayer(player)); from a services, which is running in non main thread

    <service
            android:name=".player.PlayerService"
            android:process=":player"
            android:enabled="true"
            android:exported="true">
        </service>
    

    I expect to recieve even in my fragment

      @Subscribe(threadMode = ThreadMode.MAIN) public void onEvent(SendPlayer event) {
        Log.w("mcheck", "onEvent");
      }
    

    However, i get message D/EventBus: No subscribers registered for event class yarh.com.tryexo.player.SendPlayer. Events are delivered only if i remove android:process=":player". Is it a bug or I misunderstud flow of posting events between bachground thread and main thread?

    opened by frakc 1
  • Create InputReader

    Create InputReader

    Using this class, user will be able to directly read integer, long, string, character datatype. it is faster than Scanner class of java.io. In Java programming language, input output stream takes much time than other language. so if we are able to make library which is efficient for input and output, that would be great.

    opened by Bhavik3 1
  • ObjectCache should use phone uptime instead of time

    ObjectCache should use phone uptime instead of time

    Currently ObjectCache depends on the system time never changing, which is not optimal in a setup where the app runs for a long time, the user or the system might change the time.

    a solution would be not to use the time but the uptime of the phone.

    https://developer.android.com/reference/android/os/SystemClock#uptimeMillis()

    opened by andrew-ld 2
Releases(V3.0.0)
  • V3.0.0(Nov 3, 2020)

Owner
Markus Junginger
CTO and co-founder at objectbox.io, creator of EventBus and greenDAO.
Markus Junginger
A general purpose compiler heart

Jamtree A general purpose compiler heart. This is a component of the compiler of the jamplate programming language.

Jamplate 3 Nov 22, 2022
FractalUtils - A collection of utility functions and classes, with an emphasis on game related utilities

A collection of utility functions and classes written in Kotlin. There is some emphasis on utilities useful for games (Geometry, Random, Time, Updating, etc).

null 2 Nov 11, 2022
[Deprecated] AKA VectorDrawableCompat: A 7+ backport of VectorDrawable

###@Deprecated Unfortunatenly this library is no longer maintained, we encourage you to use first party VectorDrawableCompat coming soon to support li

Telly 658 Oct 9, 2022
Android Utilities Library build in kotlin Provide user 100 of pre defined method to create advanced native android app.

Android Utilities Library build in kotlin Provide user 100 of pre defined method to create advanced native android app.

Shahid Iqbal 4 Nov 29, 2022
Collection of source codes, utilities, templates and snippets for Android development.

Android Templates and Utilities [DEPRECATED] Android Templates and Utilities are deprecated. I started with this project in 2012. Android ecosystem ha

Petr Nohejl 1.1k Nov 30, 2022
Utilities I wish Android had but doesn't

wishlist Library of helpers and utilities that I wish were included in the Android SDK but aren't. If you think something in this library is already h

Kevin Sawicki 386 Nov 21, 2022
Various useful utilities for Android apps development

Android Commons Various useful utilities for Android apps development. API documentation provided as Javadoc. Usage Add dependency to your build.gradl

Alex Vasilkov 112 Nov 14, 2022
Android library that regroup bunch of dateTime utilities

DateTimeUtils This library is a package of functions that let you manipulate objects and or java date string. it combine the most common functions use

Thunder413 98 Nov 16, 2022
CreditCardHelper 🖊️ A Jetpack-Compose library providing useful credit card utilities such as card type recognition and TextField ViewTransformations

CreditCardHelper ??️ A Jetpack-Compose library providing useful credit card utilities such as card type recognition and TextField ViewTransformations

Stelios Papamichail 18 Dec 19, 2022
A Telegram bot utilities that help to reduce the code amount

Flume Party A Telegram bot utilities that help to reduce code amount. Real project examples Pull Party Bot: 19% of code has been reduced. Resistance B

pool party 1 Jun 8, 2022
Little utilities for more pleasant immutable data in Kotlin

What can KopyKat do? Mutable copy Nested mutation Nested collections Mapping copyMap copy for sealed hierarchies copy from supertypes copy for type al

KopyKat 193 Dec 19, 2022
A set of lint rules to check for common mistakes when styling and theming on Android

A set of lint rules to check for common mistakes when styling and theming on Android

GuilhE 6 Aug 29, 2022
Native solution for common React Native problem of focused views being covered by soft input view.

react-native-avoid-softinput Native solution for common React Native problem of focused views being covered by soft input view. It is solved by listen

Mateusz Mędrek 312 Jan 2, 2023
A collection of small utility functions to make it easier to deal with some otherwise nullable APIs on Android.

requireKTX is a collection of small utility functions to make it easier to deal with some otherwise nullable APIs on Android, using the same idea as requireContext, requireArguments, and other similar Android SDK methods.

Márton Braun 82 Oct 1, 2022
Catch `dd`, `ddd`, `dump`, `sleep` and `ray` functions in your code

catch-debug-code Template ToDo list Create a new IntelliJ Platform Plugin Template project. Get familiar with the template documentation. Verify the p

Moath Alhajri 1 Apr 29, 2022
λRPC allows using code with high-order functions as a service

λRPC Simple native RPC with high order functions support. Inspired by @altavir and Communicator. λRPC allows using code with high-order functions as a

Andrey Stoyan 5 May 18, 2022
WebSocket & WAMP in Java for Android and Java 8

Autobahn|Java Client library providing WAMP on Java 8 (Netty) and Android, plus (secure) WebSocket for Android. Autobahn|Java is a subproject of the A

Crossbar.io 1.5k Dec 9, 2022
WebSocket & WAMP in Java for Android and Java 8

Autobahn|Java Client library providing WAMP on Java 8 (Netty) and Android, plus (secure) WebSocket for Android. Autobahn|Java is a subproject of the A

Crossbar.io 1.5k Dec 9, 2022