Emotion recognition by speech in android.

Overview

Vokaturi - Android Library

API Android Arsenal

Android port of the Vokaturi emotion recognition API.

Overview

Vokaturi is an emotion recognition software, that can understand the emotions in a speaker's voice. Currently Vokaturi is available for iOS, Windows, MacOS. This project adds up to support for Android platform as well. Vokaturi maintains three separate versions of its software library for recognizing emotions. The android library provided in this project is implemented using JNI framework and built up on the OpenVokaturi that is distributed under General Public License (GPL).

Currently the community version of the Vokaturi is able to detect five different types of emotions.

  • Neutrality
  • Happiness
  • Sadness
  • Anger
  • Fear

Demo

To have a check on the library, download the demo apk

Demo application shows the following emoji's based on the results detected from voice.

emoji_neutral emoji_happiness emoji_sadness emoji_anger emoji_fear

🔧 Installation

To add emotion recognition by speech functionality in your app, Add the library in your Project build.gradle :

allprojects {
	repositories {
		...
		maven { url 'https://jitpack.io' }
	}
}

Add the dependency in the build.gradle (Module: app) :

dependencies {
	compile 'com.github.alshell7:VokaturiAndroid:{version-number}'
}

Check out the latest release or from the badge above for the version number.

💡 Usage

Permissions

  • To be able to record and analyze for emotions from a given audio file, your app must tell the user that it will access the device's audio input and read from external storage. You must include these permission tags in the app's manifest file :
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>

Java

  • To instantiate the library and get access to its functions, retrieve the singleton instance of the Vokaturi class :
Vokaturi vokaturiApi;

try {            
      vokaturiApi = Vokaturi.getInstance(getApplicationContext());
 } catch (VokaturiException e) {
      e.printStackTrace();
}
  • Start recording the voice from the device :
vokaturiApi.startListeningForSpeech();
  • Stop recording and analyze for emotions :
EmotionProbabilities emotionProbabilities = vokaturiApi.stopListeningAndAnalyze();

Recording of voice from the audio input hardware of the device is automatically handled by the library, so you need not to worry about using AudioRecord and MediaRecord classes.

  • Access the metrics of emotions :
logD("Neutrality: " + emotionProbabilities.Neutrality);
logD("Happiness: " + emotionProbabilities.Happiness);
logD("Sadness: " + emotionProbabilities.Sadness);
logD("Anger: " + emotionProbabilities.Anger);
logD("Fear: " + emotionProbabilities.Fear);

Since the method stopListeningAndAnalyze() returns back highly accurate metrics of emotions analyzed from the speech data captured, you can trim the length of values in EmotionProbabilities by calling the method :

EmotionProbabilities.scaledValues(int scale);

For example, the value 0.0780483331548E-15, after calling scaledValues(5), value will change to 0.07805

  • Detect the exact dominating emotion from the metrics 🔎 :
Emotion capturedEmotion = Vokaturi.extractEmotion(EmotionProbabilities emotionProbabilities)

The returned value can be any of the below encapsulated values :

public enum Emotion
{
   Neutral,
   Happy,
   Sad,
   Angry,
   Feared
}

Analyze asynchronously

You can use async method to analyze for emotions on a background thread. But only if you wish to handle voice recording by yourself, or want to voluntarily process any audio file to extract emotions.

vokaturiApi.AnalyzeForEmotionsAsync(MainActivity.this, fileName, new VokaturiAsyncResult()
            {
                @Override
                public void onSuccess(EmotionProbabilities emotionProbabilities)
                {
                    //If there were no exceptions thrown by the native code
                }
    
                @Override
                public void onError(VokaturiException e)
                {
                   //If there was some problem that occurred while processing
                }
            });
  • Important Note : Vokaturi currently supports audio file of WAV format only. Trying to process audio file of any other format will result in receiving an exception 👻 .

💬 Error codes

The exception class thrown by the library is common for both the exceptions triggered from the native and java implementation as well. The exception comes along with the error codes, so that you can take actions based on the type of the exception.

The error codes associated with the exception are as follows :

  • VOKATURI_ERROR - Generic error.

  • VOKATURI_NOT_ENOUGH_SONORANCY - If the speech sound that is produced is not continuous, and has turbulent audio.

  • VOKATURI_DENIED_PERMISSIONS - File Read Write/Recording audio permissions not granted for the application.

  • VOKATURI_INVALID_WAV_FILE - The given file is not of a valid WAV format.

💤 Few Tips

  • You can use in built recorder to record audio in WAV format, in case you wish to handle the voice recording by yourself.

  • Need some more info?, you can download the java documentation for the library.

  • Found some problems in the library?, feel easy to report it in the issues.

How to Contribute?

  1. Fork it :trollface:
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create new Pull Request 🙉

Licenses

Copyright 2017 alshell7

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
 Copyright (C) 2016,2017 Paul Boersma, Johnny Ip, Toni Gojani
 version 2.0, 2017-01-02
 
 OpenVokaturi is free software; you can redistribute it and/or modify
 it under the terms of the GNU General Public License as published by
 the Free Software Foundation; either version 3 of the License, or (at
 your option) any later version.
 
 OpenVokaturi is distributed in the hope that it will be useful, but
 WITHOUT ANY WARRANTY; without even the implied warranty of
 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
 See the GNU General Public License for more details.
You might also like...
A better Android VideoView with more Media Controller customization. 一个更好用的Android VideoView
A better Android VideoView with more Media Controller customization. 一个更好用的Android VideoView

Android UniversalVideoView 中文版说明请点击这里 UniversalVideoView is a Android widget helps playing video easier, which is similar with the Android system nati

Android Texture VideoView having a variety of scale types like the scale types of ImageView such as fitCenter, centerCrop, centerTopCrop and more
Android Texture VideoView having a variety of scale types like the scale types of ImageView such as fitCenter, centerCrop, centerTopCrop and more

Android-ScalableVideoView Looking for the extra scale types of ImageView? Check out ScalableImageView. Android Texture VideoView having a variety of s

Android Video Crop
Android Video Crop

🔺 Before using this library, read information below 🔺 This library is not more supported. If you want to add new feature or fix a bug, grab source

Android music player example.
Android music player example.

Android music player example.

A elegant and light weight music player for android
A elegant and light weight music player for android

A elegant and light weight music player for android

NOVA is an open source video player for Android

NOVA: opeN sOurce Video plAyer Overview NOVA is an open source video player for Android. It consists in a fork of the original Archos Video Player Com

Simple and lightweight, yet polished and powerful Android video player based on ExoPlayer
Simple and lightweight, yet polished and powerful Android video player based on ExoPlayer

Just (Video) Player Android video player based on ExoPlayer It uses ExoPlayer's extension-ffmpeg with all its audio formats enabled (it can handle eve

Best material design music player for Android
Best material design music player for Android

Metro Material Design music player for Android music lovers Table of contents Downloads Differences between Metro and RetroMusicPlayer Screenshots App

Shuttle is an open source, local music player for Android.

Shuttle Music Player Shuttle is an open source, local music player for Android. Shuttle comes in two flavours: Shuttle (free) Shuttle+ The free versio

Comments
  • Chatbot integration

    Chatbot integration

    I am looking for an API that will integrate emotion detection capability for my chatbot application. Can I integrate an emotion detection without measuring it's own emotion given by the user?

    opened by jyambao 0
Releases(1.0)
Owner
Owais
Love coding for fun
Owais
mpv-android is a video player for Android based on libmpv.

mpv-android is a video player for Android based on libmpv.

null 1.1k Jan 6, 2023
FFmpeg compiled for Android. Execute FFmpeg commands with ease in your Android app.

FFMPEG video operations FFmpeg compiled for Android. Execute FFmpeg commands with ease in your Android app. Getting Started This project is provide in

Simform Solutions 277 Jan 2, 2023
Echo is a lightweight and minimal music player for Android, built with Android Studio and written in Kotlin

Echo - Echo, A light-weight, minimal music player for Android, with shuffle, favorites and audio visualization

Divins Mathew 0 Feb 7, 2022
Youtube Android Clone 🚀an Android Youtube Clone made out of XML and Kotlin

Youtube Android Clone ?? This app consumes The Youtube Api to fetch and display a list of popular videos, The app uses MVVM design pattern to allow se

Breens Robert 38 Dec 13, 2022
An extensible media player for Android

ExoPlayer ExoPlayer is an application level media player for Android. It provides an alternative to Android’s MediaPlayer API for playing audio and vi

Google 20.2k Jan 1, 2023
Android/iOS video player based on FFmpeg n3.4, with MediaCodec, VideoToolbox support.

ijkplayer Platform Build Status Android iOS Video player based on ffplay Download Android: Gradle # required allprojects { repositories {

bilibili 31k Jan 3, 2023
Custom Android view with video player, loader and placeholder image

VideoPlayerView Custom Android view with video player, loader and placeholder image. To stay up-to-date with news about the library Usage Here is an e

Marcin Moskała 89 Nov 18, 2022
[] FFmpeg build for android random architectures with example jni

AndroidFFmpegLibrary This project aims to create working library providing playing video files in android via ffmpeg libraries. With some effort and N

AppUnite Sp. z o.o. Spk. 1.1k Dec 27, 2022
Free p2p cdn android github sdk to reduce video streaming costs of live and on demand video using webrtc by upto 90% and improve scalability by 6x - 🚀 Vadootv 🚀

Android p2p cdn sdk to distribute load and reduce costs(https://peervadoo.com) Vadootv is a p2p sdk integration to reduce your video streaming costs b

Vadootv 40 Oct 5, 2022