CRYLOGGER: Detecting Crypto Misuses for Android and Java Apps Dynamically

Overview

Introduction

CRYLOGGER detects cryptographic (crypto) misuses in Android apps. A crypto misuse is an invocation to a crypto API that does not respect common security guidelines, such as those suggested by cryptographers or organizations like NIST and IETF. For instance, CRYLOGGER can tell you if your Android app uses AES in ECB mode to encrypt multiple data blocks, which is bad in cryptography.

CRYLOGGER detects crypto misuses for you automatically, without requiring to analyze a single line of your code. First, CRYLOGGER runs your Android app on the official Android Emulator, whose Java libraries have been instrumented to log the parameters passed to the relevant crypto APIs. Then, it analyzes the log file offline and reports all the crypto misuses. Differently from other approaches, it does not employ static analysis. CRYLOGGER runs your app by using Monkey or the user-interface events you send to the emulator.

If you want to know more about CRYLOGGER, please contact me at [email protected] or read our Oakland paper:

Luca Piccolboni, Giuseppe Di Guglielmo, Luca P. Carloni and Simha Sethumadhavan, "CRYLOGGER:
Detecting Crypto Misuses Dynamically", in IEEE Symposium on Security and Privacy (SP), 2021.

Requirements

There are not a lot of requirements that are specific to CRYLOGGER. If you satisfy the requirements of the AOSP and you install all the Python packages required for scripts/run.py and scripts/check.py you are good to go! CRYLOGGER has been tested on Android-9.0.0_r36 (this is the default version of the emulator that is installed as explained below). It should be easy to adapt it to other Android builds. For the host machine, we used a machine with a clean installation of Ubuntu 18.04.1.

Make sure you install the Android SDK if you want to compile the test app included in this repository (directory test-app) and set the environment variable ANDROID_SDK_ROOT to point to it.

Emulator Setup

Once you satisfy the requirements of the AOSP, it is sufficient to run the following command to setup the emulator and CRYLOGGER:

cd scripts/setup
./setup_emu.py

This scripts downloads the AOSP in a new directory android-emu from the official Google repositories, installs CRYLOGGER by copying the files from the directory scripts/deltas, and builds it. Please refer to the scripts in the directory scripts/setup for more information. By default, it uses all the available cores to compile the AOSP.

In addition, if you want to install apps from the Google Play Store, you need to install the OpenGApps. You can do so by running the following commands:

cd scripts/setup
./setup_opengapps.sh

The OpenGApps (x86-9.0-super-20210127) are downloaded in script/opengapps so they can be installed on the emulator.


Verify your App

You are now ready to run your app on the Android emulator and collect the log that contains information about the crypto APIs that are invoked. We call this log "cryptolog". Here, we verify a simple Android app that you can find in the directory app-test. If you have your own APK to test, you can skip the compilation of app-test, otherwise:

cd app-test
./gradlew build

If the compilation is successfull, you should find a file named com.example.aes_0.apk in the folder test-app that points to the APK of the test app. If you use your APK, make sure you use the following naming convention: _ .apk , where is the package name of the Android app and is its version number. Copy the APK in the directory scripts/data/apks:

cp app-test/com.example.aes_0.apk scripts/data/apks/

You need to start the emulator by passing the option -writable-system (this option is only used to install the OpenGApps):

# Setup the env variables
cd android-emu
source build/envsetup.sh
lunch sdk_phone_x86-userdebug
# Now start the emulator
emulator -writable-system

Collect the logs

The emulator should be now running. Wait for the completion of the boot process, and then run the following script to execute your app:

cd scripts
python run.py --work_dir data --session emulator-<number>

where is the emulator session number (you can find it in the title bar of the emulator window). By default the script run.py (1) installs the OpenGApps, if they have not been installed in a previous run, (2) configures the emulator, so that your app can be tested with Monkey, (3) installs your app on the emulator, (4) runs your app with Monkey by using a fixed number of user-interface events (default 100), (5) collects the cryptolog, which contains information about the use of the crypto APIs, and (6) uninstalls the app. You can easily modify the script run.py if you want to use your own user-generated events.

Analyze the logs

After 'run.py' completes, you should find the log in the directory scripts/data/crypto_logs. Now you can analyze them by running the following command:

cd scripts
python check.py --work_dir data/crypto_logs --rule_ID <number>

where is the number of the crypto rule you want to check. CRYLOGGER support 26 rules that are explained in the paper as well as in the script check.py. These rules are suggested by cryptographers or organizations like NIST and IETF. Try for example to check rule R-03 by using the following command:

cd scripts
python check.py --work_dir data/crypto_logs --rule_ID 03

You should obtain a file with extension .rules in the directory scripts/data/crypto_logs that tells you if rule R-03 is violated. For the app included in this repository the rule should be violated because the app performs encryptions and decryptions by using the insecure ECB mode. Note that some rules require two executions of your app, thus you need to run the script run.py twice. The second run should look like this:

# Start emulator
cd android-emu
emulator -writable-system
# Run the script
cd ../scripts
python run.py --work_dir data --session emulator-<number> --suffix 2

This command runs again your app on the emulator and appends '2' to the cryptolog files (extension .cryptolog2 instead of simply .cryptolog). You can check rules that require two executions, for example rule R-05 with the following command:

cd scripts
python check.py --work_dir data/crypto_logs --rule_ID 05

If you want to check all the rules supported by CRYLOGGER, omit the flag --rule_ID.

Comments
  • How to solve the INSTALL_FAILED_NO_MATCHING_ABIS failure?

    How to solve the INSTALL_FAILED_NO_MATCHING_ABIS failure?

    I fetched some apks from some Android application stores, but met some problem when installing

    ---------------adb: failed to install /home/zhang/documents/code/crylogger/scripts/data/apks/com.ss.android.article.news_8.3.3.apk: Failure [INSTALL_FAILED_NO_MATCHING_ABIS: Failed to extract native libraries, res=-113]
    

    It seems that the problem was caused because it uses the arm binaries.

    However, support_for_arm_binaries_on_android_9_and_11_system_images says that

    If you were previously unable to use the Android Emulator because your app depended on ARM binaries, you can now use the Android 9 x86 system image or any Android 11 system image to run your app – it is no longer necessary to download a specific system image to run ARM binaries. These Android 9 and Android 11 system images support ARM by default and provide dramatically improved performance when compared to those with full ARM emulation.

    I verified the document by downloading the android 9 and android 11 image from Android Studio, and then ran the Android Studio emulator and installed apk com.ss.android.article.news_8.3.3.apk successfully by running adb install -g com.ss.android.article.news_8.3.3.apk.

    It seems that the android 9/11 x86 image could provided ARM compatibility but my compiled crylogger failed to do that.

    Have you met the INSTALL_FAILED_NO_MATCHING_ABIS failure? And how do you solve it? Thanks in advance.

    opened by darwintree 4
  • cryptolog not found

    cryptolog not found

    I have reviewed the other two issues about cryptolog, which should be located in /data/data/com.example.aes/cache, (I am using the example app right now) but still didn't figure out how to solve the problem I am facing. I have two problems listed below.

    1. I am using a remote server and could run the emulator successfully. However, while pinning the application, I got stuck on the part "get the ID of the task of the application to pin". There is no output from grep command. Do you have any ideas what could cause this? Why there is no task ID of the app?
    Screen Shot 2021-06-06 at 4 20 26 PM
    1. I have omitted the last two steps of the part of pinning the application, and started to run monkey. The automated test was successful.
    Screen Shot 2021-06-06 at 4 31 16 PM But I was not able to find the cryptolog in /data/data/com.example.aes/cache. I have seen "aes.log" in one of the subdirectories (which implies that the app is actually running as expected??): Screen Shot 2021-06-06 at 4 36 50 PM and I tried `git status` in android-emu/libcore directory. Screen Shot 2021-06-06 at 4 47 27 PM Do you have any suggestions on this or have you been in the similar situation before? I would really appreciate your help!!

    Thank you!

    opened by Ellakyli 4
  • Why the FileOutputStream is used instead of the native

    Why the FileOutputStream is used instead of the native "android.util.Log" library ?

    I noticed in the implementation of CRYLOGGER, you used

    private static FileOutputStream stream;
    

    to record the log instead of using the android.util.Log library. What is the intent of this design?

    Thanks in advance.

    opened by darwintree 1
  • How to migrate CRYLOGGER delta to other Android versions?

    How to migrate CRYLOGGER delta to other Android versions?

    It seems simply apply the changes in scripts/delta folder to AOSP source code would work. The problem is how to correctly modify current.txt and openjdk_java_files.bp.

    Could you provide some advice about how to check the correction of the migration? By the way, is there any method to only build the changed module and apply the changes instead of building the whole AOSP from scratch ? Thanks in advance

    opened by darwintree 1
  • Cound ' sdk_phone_x86-userdebug' be substituted by other target platforms?

    Cound ' sdk_phone_x86-userdebug' be substituted by other target platforms?

    In the setup file, lunch sdk_phone_x86-userdebug is used for compiling android. Could sdk_phone_x86-userdebug be substituted by other target platforms? And why this target platform is chosen?

    Thanks in advance.

    opened by darwintree 1
  • OpenGapps is not available

    OpenGapps is not available

    The OpenGapps version used in the setup_opengapps.sh is not available anymore.

    https://sourceforge.net/projects/opengapps/files/x86/20210127/open_gapps-x86-9.0-super-20210127.zip

    However, https://sourceforge.net/projects/opengapps/files/x86/20210126/open_gapps-x86-9.0-super-20210126.zip is available. Do you think this version can be used?

    opened by tausiq 1
  • I have some question,please help me~~~

    I have some question,please help me~~~

    I do all the things , but when i install my apk on emulator and run monkey, it told me no souch file "/tmp/application.cryptolog".I found it on deltas\libcore\ojluni\src\main\java\java\security\CRYLogger.java. I can not create that file, so i change the path to "/data/local/tmp/application.cryptolog". Will it works or something else i should do.Please~~~~~help~~~~help~~~~~

    opened by Followwind4567 1
  • Analysis gets stuck in pinning step

    Analysis gets stuck in pinning step

    After few successful apps analysis the process either fails in pinning step or gets stuck in pinning step.

    shahab-Precision-T1700 2021-07-12 13-37-44

    When the process is stuck in pinning step the emulator displays a blank/white screen

    shahab-Precision-T1700 2021-07-12 13-40-53

    I think this is a limitation of the emulator itself and may not be solvable from project end. I am wondering if you faced similar issue and if yes, how did you deal with it?

    Right now I am restarting the emulator and then re-execute run.py. For a large number of app analysis this is not a suitable solution.

    To overcome this, I am planning to execute multiple emulators and parallel the analysis process. Multiple emulators cannot be run from same source I think, I got the following error,

    emulator: ERROR: There's another emulator instance running with the current AVD '<build>'. Exiting...
    

    How about I copy crylogger source into multiple location and execute emulators from each of them but pointing work_dir to the same location. Do you foresee any potential issues with this idea? Thanks!

    opened by tausiq 1
  • Documentation improvement

    Documentation improvement

    Issue 01

    Before running the "Emulator Setup" the distutils package is required.

    Distutils package is available only for python3 and AOSP requires python 2.7. Still the package is required. Otherwise getting the following error,

    no module named distutils.dir_util
    

    Please add the following line in documentaiton,

    sudo apt-get install python3-distutils -y
    

    Issue 02

    Repo tool is required. Please add the following link in documentation, https://source.android.com/setup/develop#installing-repo

    Additionally,

    sudo apt-get install curl -y
    

    Issue 03

    sudo apt-get install lzip

    Issue 04

    sudo apt-get install m4
    

    Issue 05

    sudo apt install python-pip -y
    pip install zxcvbn
    pip install numpy
    

    Issue 06

    This repo should be cloned with --recursive flag, otherwise dj-on-github/sp800_22_tests files will not get downloaded.

    sudo apt-get install git -y
    git clone --recursive https://github.com/lucapiccolboni/crylogger.git
    

    Issue 07

    sudo apt-get install cpu-checker
    kvm-ok
    sudo chown $USER /dev/kvm
    
    opened by tausiq 5
  • There is no application.cryptolog. in /data/data/com.example.aes/cache/

    There is no application.cryptolog. in /data/data/com.example.aes/cache/

    I followed the instructions in the README and executed the commands in order, but when I reached the 'Collect the logs' step, 'cryptolog | failed' was printed. By looking at the script file, I found the problem was that there was no no application.cryptolog. in /data/data/com.example.aes/cache/, which led to The run_cmd function returns Error, I want to know how to solve it.

    Thanks in advance.

    image

    opened by Rocksnake 39
  • Emulator stops after couple of minutes with segmentation fault (core dump)

    Emulator stops after couple of minutes with segmentation fault (core dump)

    I have used a fresh install of Ubuntu 18.04 and and followed the procedure mentioned in readme. When I execute emulator -writable-system, the emulator gets started but stops with Segmentation fault.

    I have also tried with macOS but it didn't help. Have you faced any similar issue? If yes, how do you resolve this?

    Thanks in advance.

    opened by tausiq 6
Releases(v1.0)
Owner
Luca Piccolboni
Luca Piccolboni
CLI tool for decompiling Android apps to Java. It does resources! It does Java! Its real easy!

Easy-as-pie Android Decompiler Why One stop shop I got pretty tired of decompiling Android apps with a bunch of steps that I had to remember all the t

Alex Davis 619 Dec 27, 2022
enjarify 8.6 0.0 L5 Python Enjarify is a tool for translating Dalvik bytecode to equivalent Java bytecode. This allows Java analysis tools to analyze Android applications.

Note: This repository may be out of date. Future development will occur at https://github.com/Storyyeller/enjarify. Introduction Enjarify is a tool fo

Google 2.7k Jan 8, 2023
Grab’n Run, a simple and effective Java Library for Android projects to secure dynamic code loading.

Grab’n Run, a simple and effective Java Library for Android projects to secure dynamic code loading.

Luca Falsina 418 Dec 29, 2022
A Java ePub reader and parser framework for Android.

FolioReader-Android is an EPUB reader written in Java and Kotlin. Features Custom Fonts Custom Text Size Themes / Day mode / Night mode Text Highlight

FolioReader 2.1k Jan 3, 2023
A program analysis tool to find cryptographic misuse in Java and Android.

A program analysis tool to find cryptographic misuse in Java and Android.

null 92 Dec 15, 2022
Signal Protocol library for Java/Android

Overview A ratcheting forward secrecy protocol that works in synchronous and asynchronous messaging environments. PreKeys This protocol uses a concept

Signal 1.8k Dec 24, 2022
Analyze any Android/Java based app or game

ClassyShark Introduction ClassyShark is a standalone binary inspection tool for Android developers. It can reliably browse any Android executable and

Google 7.2k Jan 3, 2023
Appdbg - make it possible to run android dex file in original Java Virtual Machine

Appdbg - make it possible to run android dex file in original Java Virtual Machine

null 137 Dec 20, 2022
Smali/baksmali is an assembler/disassembler for the dex format used by dalvik, Android's Java VM implementation

About smali/baksmali is an assembler/disassembler for the dex format used by dalvik, Android's Java VM implementation. The syntax is loosely based on

Ben Gruver 5.7k Dec 27, 2022
TweetNaCl in Java - a port of TweetNaCl-js

TweetNacl in Java: port of tweetnacl-js API/Usage Suggest always use TweetNaclFast implementation Public key authenticated encryption get key pair: Bo

AppNet.Link 40 Nov 10, 2022
Dex to Java decompiler

JADX jadx - Dex to Java decompiler Command line and GUI tools for producing Java source code from Android Dex and Apk files Main features: decompile D

null 32.8k Jan 2, 2023
Java bytecode obfuscator created by x4e.

Binscure Java bytecode obfuscator created by x4e. Usage First, create a config file, (example config here). When you have a config file, run binscure

null 35 Nov 22, 2022
A RSA Cipher implementation for Kotlin/Java

A RSA Cipher implementation for Kotlin/Java.

Lyzev 3 Aug 22, 2022
Soot - A Java optimization framework

Using Soot? Let us know about it! We are regularly applying for funding to help us maintain Soot. You can help us immensely by letting us know about p

Soot Program Analysis Framework 2.5k Jan 2, 2023
Find Security Bugs is the SpotBugs plugin for security audits of Java web applications

The SpotBugs plugin for security audits of Java web applications and Android applications. (Also work with Kotlin, Groovy and Scala projects)

OWASP Find Security Bugs 2k Jan 6, 2023
Catch common Java mistakes as compile-time errors

Error Prone Error Prone is a static analysis tool for Java that catches common programming mistakes at compile-time. public class ShortSet { public

Google 6.3k Dec 23, 2022
MiHawk 🦅👁️ is simple and secure 🔒 Android Library to store and retrieve pair of key-value data with encryption , internally it use jetpack DataStore Preferences 💽 to store data.

MiHawk MiHawk ?? ??️ is simple and secure ?? Android Library to store and retrieve pair of key-value data with encryption , internally it use jetpack

Nedal Hasan Ibrahem 5 Sep 3, 2022
BlackDex is an Android unpack tool, it supports Android 5.0~12 and need not rely to any environment. BlackDex can run on any Android mobile phones or emulators, you can unpack APK File in several seconds.

BlackDex is an Android unpack tool, it supports Android 5.0~12 and need not rely to any environment. BlackDex can run on any Android mobile phones or emulators, you can unpack APK File in several seconds.

null 4.3k Jan 2, 2023