A Gradle plugin to help analyse the dependency between modules and run tasks only on modules impacted by specific set of changes.

Overview

Change Tracker Plugin

A Gradle plugin to help analyse the dependency between modules and run tasks only on modules impacted by specific set of changes.

The idea behind this plugin is to optimize running verification tasks on large modularized projects. There is no need to run these tasks on all modules every time we apply a change. Instead, we could run them only on modules affected by the changes and their dependents.


Take the following project structure as an example:



If you apply changes to the :profile-service module, you only need to run your verification tasks on :profile-service:profile-feature, and :app. It's safe to skip the tasks on all the other modules since they don't depend on :profile-service and can't be affected by these changes.

Setup

Apply the plugin and the configuration to your root project build.gradle

plugins {
    id 'com.ismaeldivita.changetracker' version '0.7.4'
}

// ...

changeTracker {
    tasks = ['lint','testDebugUnitTest']
    whitelist = [':app']
    blacklist = [':network',':featureA']
    reevaluate = [':sharedTest']
    branch = "master"
    remote = "origin"
}
  • tasks: List of the tasks the plugin will need to create.
  • branch: Name of the branch that should be used to extract the diff.
  • whitelist (optional): List of modules that should always run.
  • blacklist (optional): List of modules that should never run.
  • reevaluate (optional): List of modules that will trigger the task for all modules
  • remote (optional): Name of the remote repository.

Usage

The plugin will generate new tasks on the root project for each task provided on the configuration with the following name ${taskName}ChangedModules.

Taking as an example the configuration above the plugin will generate two new tasks lintChangedModules and testDebugUnitTestChangedModules.

To run your task:

./gradlew testDebugUnitTestChangedModules

You can override the default branch used for the comparison when running your command. This is useful when you're using the plugin on pull requests and each pull request may have different base branches.

./gradlew testDebugUnitTestChangedModules -Pbranch=dev

Notes

  • This plugin will assume you use GIT as your VCS.
  • Any changes to the root project or buildSrc will trigger the task for all modules.
  • Test configurations will not be tracked since dependency cycles could be created for test purposes and this is not supported by this plugin. If you have a shared test library project and want to trigger the tasks on their dependents check the reevaluate configuration.

License

MIT License

Copyright (c) 2019 Ismael Di Vita

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Comments
  • Affected Projects listing all modules

    Affected Projects listing all modules

    Hi,

    Thanks for this project, I think it's going to make a huge difference to our CI environment.

    At the minute running changedModules lists all the modules in the app as affected. Is there anyway to debug what is going on? I can see logger.isInfoEnabled but not sure how to toggle that.

    opened by mshearer123 11
  • Changed 3 modules and got  java.lang.StackOverflowError

    Changed 3 modules and got java.lang.StackOverflowError

    I've changed 3 modules and got such a problem. Could @ismaeldivita you take a look?

    > Task :changedModules FAILED
    FAILURE: Build failed with an exception.
    * What went wrong:
    Execution failed for task ':changedModules'.
    > java.lang.StackOverflowError (no error message)
    * Try:
    Run with --info or --debug option to get more log output. Run with --scan to get full insights.
    * Exception is:
    org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':changedModules'.
    	at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.lambda$executeIfValid$1(ExecuteActionsTaskExecuter.java:207)
    	at org.gradle.internal.Try$Failure.ifSuccessfulOrElse(Try.java:263)
    	at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeIfValid(ExecuteActionsTaskExecuter.java:205)
    	at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:186)
    	at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:114)
    	at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:46)
    	at org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter.execute(ResolveTaskExecutionModeExecuter.java:62)
    	at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:57)
    	at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:56)
    	at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:36)
    	at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.executeTask(EventFiringTaskExecuter.java:77)
    	at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:55)
    	at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:52)
    	at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:416)
    	at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:406)
    	at org.gradle.internal.operations.DefaultBuildOperationExecutor$1.execute(DefaultBuildOperationExecutor.java:165)
    	at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:250)
    	at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:158)
    	at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:102)
    	at org.gradle.internal.operations.DelegatingBuildOperationExecutor.call(DelegatingBuildOperationExecutor.java:36)
    	at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:52)
    	at org.gradle.execution.plan.LocalTaskNodeExecutor.execute(LocalTaskNodeExecutor.java:41)
    	at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:370)
    	at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:357)
    	at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:350)
    	at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:336)
    	at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.lambda$run$0(DefaultPlanExecutor.java:127)
    	at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:191)
    	at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.executeNextNode(DefaultPlanExecutor.java:182)
    	at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.run(DefaultPlanExecutor.java:124)
    	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
    	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
    	at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
    Caused by: java.lang.StackOverflowError
    	at kotlin.collections.CollectionsKt__IterablesKt.collectionSizeOrNull(Iterables.kt:33)
    	at kotlin.collections.SetsKt___SetsKt.plus(_Sets.kt:119)
    	at com.ismaeldivita.changetracker.project.ProjectDependents.getAllDependents(ProjectDependents.kt:16)
    	at com.ismaeldivita.changetracker.project.ProjectDependents.getAllDependents(ProjectDependents.kt:16)
    	at com.ismaeldivita.changetracker.project.ProjectDependents.getAllDependents(ProjectDependents.kt:16)
    	at com.ismaeldivita.changetracker.project.ProjectDependents.getAllDependents(ProjectDependents.kt:16)
    	at com.ismaeldivita.changetracker.project.ProjectDependents.getAllDependents(ProjectDependents.kt:16)
    	at com.ismaeldivita.changetracker.project.ProjectDependents.getAllDependents(ProjectDependents.kt:16)
    	at com.ismaeldivita.changetracker.project.ProjectDependents.getAllDependents(ProjectDependents.kt:16)
    
    bug 
    opened by lion4ik 4
  • Compatibility with gradle 7

    Compatibility with gradle 7

    After updating the gradel version of my project to gradle 7, build using tasks from this plugin fail with the following message:

    FAILURE: Build failed with an exception.
    
    * What went wrong:
    A problem was found with the configuration of task ':changedModules' (type 'ChangedModulesTask').
      - Type 'com.ismaeldivita.changetracker.ChangedModulesTask' property 'group' is missing an input or output annotation.
        
        Reason: A property without annotation isn't considered during up-to-date checking.
        
        Possible solutions:
          1. Add an input or output annotation.
          2. Mark it as @Internal.
        
        Please refer to https://docs.gradle.org/7.0.1/userguide/validation_problems.html#missing_annotation for more details about this problem.
    
    
    opened by SanderSmee 3
  • Let reevaluate option take paths

    Let reevaluate option take paths

    Sometimes there are buildsystem, config or buildSrc directories (not modules) that contains dependency versions for all libs. Changes in them should behave like the reevaluate option. Having module names in reevaluate is great but it'd be awesome if we can also include paths there as well.

    Something like this https://github.com/dropbox/AffectedModuleDetector#installation

    reevaluate = ["config/", ":core", ":sharedTest"]
    

    They could be different options as well

    opened by nodejsjs 3
  • Add new option `reevaluate`

    Add new option `reevaluate`

    Thank you very much for this awesome plugin. It improved our CI pipeline by far.

    This PR adds the following changes:

    • Remove blacklist projects before evaluating the dependents: In our specific case, we have a module defined which only has lint rules. As this module is not a com.android.library or com.android.application it misses tasks like unit tests. This results in warnings, even when this module is added to the blacklist
    • Exclude test configuration when evaluating project dependents: This fixes #3 as you could run into cyclic dependencies because the configuration is not considered. Example: databaseModule: testImplementation(project(:sharedTestModule)) sharedTestModule: implementation(project(:databaseModule)) This is possible, as the databaseModule has only a dependency in its tests, while the sharedTestModule needs database functionality for testing purposes.
    • Reevaluate option: As the test depedencies are not considered for evaluation, there has to be an option to trigger all tasks in case the sharedTestModule has changes.

    I was unable to test these changes locally, as I am not versed enough in gradle to get this plugin to run for development purposes in our project. Hopefully, @ismaeldivita can confirm the changes work.

    opened by vtuan10 3
  • Branch not found on CI

    Branch not found on CI

    I have a master branch and I'm trying to test this plugin on the CI. I branched off of master into a new branch called a and installed the plugin in branch a.

    in branch a:

    changeTracker {
        tasks = ['testDebugUnitTest']
        branch = "a"
    }
    

    Then I branched off of branch a into branch b to test the changedModules task. In this branch I just made a small change in one of the modules. I pushed both branches and triggered a CI build on branch b and ran the command gradlew changedModules on CI. The branches look like the following:

    ------  master
                 \
                   a ----- commit (install the plugin)
                                \
                                 b ------ commit (minor change in a module)
    

    The CI build failed with the following error:

    Caused by: org.gradle.api.GradleException: Branch  a not found
    

    I've tried adding remote in the config block and tried other variations of the branch name like origin/a or remotes/origin/a, etc, but nothing works. Locally, if I'm on branch b and run ./gradlew changedModules, then it runs fine and prints out a nice report. Not sure what's going on in the CI. Somehow the branch name is the issue. Any ideas?

    opened by nodejsjs 2
  • Logging diffs

    Logging diffs

    Hi,

    No rush at all, but if you do another release, do you think it would be possible to add more logging for the diffs found? We're having trouble identifying what is changing in our root gradle in CI

    cheers

    opened by mshearer123 2
  • Add ability to toggle between changed modules & affected dependencies

    Add ability to toggle between changed modules & affected dependencies

    It would be nice to be able to specify a command-line flag to toggle between changed modules + affected dependencies and only changed modules. This comes useful for tasks like linting, which won't affect top-level modules in case of changes.

    The AffectedModuleDetector plugin has an example here on a similar feature, but I believe your plugin is still more powerful, which is why it would be nice to get such a feature here!

    enhancement 
    opened by GabrielBottari 1
  • The plugin always detekt all modules as changes

    The plugin always detekt all modules as changes

    I've integrated the plugin and has problem in the next condition:

    • all changes are committed
    • current branch is develop
    • Configuration:
    changeTracker {
        // List of tasks the plugin will need to create
        tasks = ['lintDebug','testDebugUnitTest', 'detekt']
    
        // List of modules that should always run
        whitelist = []
    
        // List of modules that should never run.
        blacklist = []
    
        // List of modules that will trigger the task for all modules
        reevaluate = []
    
        // Name of the branch that should compare to the current one to extract the diff
        // Branch can be specified dynamically via Gradle property 'branch'. E.g. -Pbranch=dev
        branch = "develop"
    }
    

    Task changedModules always return all modules as changed. What is incorrect?

    opened by kirich1409 8
Releases(0.7.4)
Owner
Ismael Di Vita
@transferwise
Ismael Di Vita
Gradle Plugin to automatically upgrade your gradle project dependencies and send a GitHub pull request with the changes

Gradle Plugin to automatically upgrade your gradle project dependencies and send a GitHub pull request with the changes

Dipien 142 Dec 29, 2022
Graphfity is a Gradle Plugin which creates a dependency node diagram graph about your internal modules dependencies, specially useful if you are developing a multi-module application

Graphfity creates a dependency nodes diagram graph about your internal modules dependencies, specially useful if you are developing a multi-module app

Iván Carrasco 27 Dec 20, 2022
A Gradle Plugin to determine which modules were affected by a set of files in a commit.

A Gradle Plugin to determine which modules were affected by a set of files in a commit. One use case for this plugin is for developers who would like to only run tests in modules which have changed in a given commit.

Dropbox 491 Dec 23, 2022
Gradle plugin to ease Kotlin IR plugin development and usage in multimodule gradle projects

Gradle plugin to ease Kotlin IR plugin development and usage in multimodule gradle projects. Former: kotlin-ir-plugin-adapter-gradle

null 2 Mar 8, 2022
A Gradle plugin that helps you speed up builds by excluding unnecessary modules.

?? Focus A Gradle plugin that generates module-specific settings.gradle files, allowing you to focus on a specific feature or module without needing t

Dropbox 314 Jan 2, 2023
Gradle Plugin that determines if modules are Kotlin Multiplatform (KMP) ready.

Gradle Plugin that determines if modules are Kotlin Multiplatform (KMP) ready. KMP Ready means that the code is Kotlin Multiplatform compatible.

Sam Edwards 58 Dec 22, 2022
A Gradle plugin that generates plugin.yml for Bukkit/BungeeCord/Nukkit plugins based on the Gradle project

plugin-yml is a simple Gradle plugin that generates the plugin.yml plugin description file for Bukkit plugins, bungee.yml for Bungee plugins or nukkit.yml for Nukkit plugins based on the Gradle project. Various properties are set automatically (e.g. project name, version or description) and additional properties can be added using a simple DSL.

Plexus 0 Apr 10, 2022
Kmp4free - A Gradle Plugin that allows seamless switching between Kotlin JVM and the Kotlin Multiplatform Plugins

?? kmp4free Allows you to toggle between Kotlin JVM Plugin -> Kotlin Multiplatform with a Gradle Property kmp4free=true. This Gradle Plugin was built

Sam Edwards 61 Oct 14, 2022
Gradle plugin which validates the licenses of your dependency graph match what you expect

Gradle plugin which validates the licenses of your dependency graph match what you expect

Cash App 502 Dec 31, 2022
Gradle Plugin that allows you to decompile bytecode compiled with Jetpack Compose Compiler Plugin into Java and check it

decomposer Gradle Plugin that allows you to decompile bytecode compiled with Jetpack Compose Compiler Plugin into Java and check it How to use Run bui

Takahiro Menju 56 Nov 18, 2022
Gradle Plugin to enable auto-completion and symbol resolution for all Kotlin/Native platforms.

CompleteKotlin Gradle Plugin to enable auto-completion and symbol resolution for all Kotlin/Native platforms. What this plugin provides This zero-conf

Louis CAD 235 Jan 3, 2023
Android Gradle Plugin -- Auto Check big image and compress image in building.

McImage I will continue to update, please rest assured to use 中文文档 Android优雅的打包时自动化获取全部res资源 McImage is a Non-invasive plugin for compress all res in

smallSohoSolo 1.1k Dec 28, 2022
Gradle plugin that generates a Swift Package Manager manifest and an XCFramework to distribute a Kotlin Multiplatform library for Apple platforms.

Multiplatform Swift Package This is a Gradle plugin for Kotlin Multiplatform projects that generates an XCFramework for your native Apple targets and

Georg Dresler 262 Jan 5, 2023
Gradle Plugin for publishing artifacts to Sonatype and Nexus

Introduction Due to Sonatype's strict validation rules, the publishing requirement must be satisfied by every artifact which wants to be published to

Johnson Lee 21 Oct 14, 2022
Gradle plugin that parses version updates and assigns them to groups of people.

Notifier Gradle Plugin This gradle plugin serves the need of automating how dependencies are handles in a project. More specifically, it functions usi

Plum 4 Oct 27, 2022
A Gradle plugin enforcing pre-commit and commit-msg Git hooks configuration

A Gradle plugin enforcing pre-commit and commit-msg Git hooks configuration. Conventional-commits-ready.

Danilo Pianini 24 Dec 16, 2022
Klinker is a gradle plugin making it possible to link kotlin native executables with custom linkers and options.

Klinker is a gradle plugin making it possible to link kotlin native executables with custom linkers and options. It does this by creating a static library for kotlin compilation, then generates a c+kotlin wrapper that calls into kotlin to start the app, finally using the specified compiler to compile and link the c code and kotlin library into a binary.

Jason Monk 4 Apr 14, 2022
Gradle plugin to check, if rest-controllers are used by clients and clients have suitable rest-interfaces

Verify-Feign Gradle Plugin This plugin will help you to verify all rest-clients and -controllers in a multimodule spring project. It will check, if al

null 3 May 11, 2022
The core Gradle plugin and associated logic used for Slack's Android app

slack-gradle-plugin This repository contains the core Gradle plugin and associated logic used for Slack's Android app. This repo is effectively read-o

Slack 306 Dec 30, 2022