Hi!

The title says it basically. How do you get the entire AST, with DefaultAstNodes converted to the Klass Data classes, from a kotlinfile. In the example the following function is invoked:

kotlinFile.summary()

This function does not include the body of a function for example. I tried to create my own function:

kotlinFile.complete()

private val completeTreeMap: TreeMap = DefaultTreeMap(
    annotationsMapper,
    declarationsMapper,
    expressionsMapper.flattenNames("functionDeclaration"),
    globalsMapper,
    kotlinDefaultMapper,
    modifierMapper,
    stringMapper,
    typesMapper
)

fun List<Ast>.complete(): AstResult<List<Ast>> {
    return completeTreeMap.treeMap(this)
}

fun Ast.complete(): AstResult<List<Ast>> {
    return listOf(this).complete()
}

I tried to change the Mappers in the DefaultTreeMap, but couldn't get a configuration to be able to visit the statements in the body of functions.

Hopefully this is enough information to be able to resolve my problem.

Thanks in advance!

Hi! I tried to parse a very simple file: class Sample (private val propertyA: String)

using the code from example

fun main() {
    val source = AstSource.File(
        "src/jvmMain/kotlin/sample/Sample.kt"
    )
    val kotlinFile = KotlinGrammarAntlrKotlinParser.parseKotlinFile(source)
    kotlinFile.summary()
        .onSuccess { astList ->
            astList.forEach(Ast::print)
        }.onFailure { errors ->
            errors.forEach(::println)
        }
}

And got such error

Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 170
	at kotlinx.ast.parser.antlr.kotlin.AntlrKotlinAstParserImpl.toAstTerminal(AntlrKotlinAstParserImpl.kt:27)
	at kotlinx.ast.parser.antlr.kotlin.AntlrKotlinAstParserImpl.parse(AntlrKotlinAstParserImpl.kt:82)
	at kotlinx.ast.parser.antlr.kotlin.AntlrKotlinAstParserImpl.parse(AntlrKotlinAstParserImpl.kt:75)
	at kotlinx.ast.parser.antlr.kotlin.AntlrKotlinAstParserImplKt.antlrKotlinParser(AntlrKotlinAstParserImpl.kt:138)
	at kotlinx.ast.parser.antlr.kotlin.AntlrKotlinAstParserImplKt.antlrKotlinParser(AntlrKotlinAstParserImpl.kt:98)
	at kotlinx.ast.parser.antlr.kotlin.AntlrKotlinParser.parse(AntlrKotlinParser.kt:20)
	at kotlinx.ast.grammar.kotlin.common.KotlinGrammarParser$DefaultImpls.parseKotlinFile(KotlinGrammarParser.kt:9)
	at kotlinx.ast.grammar.kotlin.target.antlr.kotlin.KotlinGrammarAntlrKotlinParser.parseKotlinFile(KotlinGrammarAntlrKotlinParser.kt:10)
	at sample.MainKt.main(Main.kt:14)
	at sample.MainKt.main(Main.kt)

My project is a Gradle Plugin that parses Kotlin code to generate flat files.

I am trying to replace kastree with this library, but i encounter the captioned error when I run my plugin in another project. My code works fine within my Gradle plugin project and my tests though. I suppose this is a missing transitive dependency problem, but I can't figure out what is missing exactly.

In my Gradle plugin project I have the following dep:

api 'com.github.kotlinx.ast:grammar-kotlin-parser-antlr-kotlin-jvm:a96e681f90'

Note that the Gradle project uses subprojects. The parsing is implemented in one of the subproject, and the Gradle plugin project has an implementation dependency on that subproject. I also tried with an api dependency to no avail.

Should i add a runtime dependency on the kotlin parser ?

Example below will crash when calling summary() i.e. KotlinAstParser.parseKotlinFile(source).summary() throws java.lang.UnsupportedOperationException: Empty collection can't be reduced..

Parsing the code below, with the String? in a generic, leads to a new structure and summary() does not work anymore.

package com.example.test

class TestClass {
    fun function(record: Map<String?, String>) {
    }
}

Hi, I'm not sure that is it ast library issue but I've got following file:

package com.example.app.ui

internal class FeatureFragment :
    BaseFragment<FeatureViewModel, FeatureFragmentBinding>() {

    private fun handleViewStateChanges(viewState: FeatureViewModel.ViewState): Unit =
        with(
            requireBinding
        ) {
            dateRangePropertyView.text =
                "${monthDayYearDateFormat.format(viewState.dateRange.startDate)} - ${
                    monthDayYearDateFormat.format(viewState.dateRange.endDate)
                }"
        }
}

I'm testing the file as usual:

kotlinFile.summary(attachRawAst = false)
    .onSuccess { astList ->
        astList.forEach(Ast::print)
    }.onFailure { errors ->
        errors.forEach(::println)
    }

and parser throws an exception:

Exception in thread "main" org.antlr.v4.kotlinruntime.misc.ParseCancellationException at kotlinx.ast.parser.antlr.kotlin.AntlrKotlinErrorListener.syntaxError(AntlrKotlinErrorListener.kt:17) at org.antlr.v4.kotlinruntime.ProxyErrorListener.syntaxError(ProxyErrorListener.kt:34) at org.antlr.v4.kotlinruntime.Parser.notifyErrorListeners(Parser.kt:564) at org.antlr.v4.kotlinruntime.DefaultErrorStrategy.reportUnwantedToken(DefaultErrorStrategy.kt:376) at org.antlr.v4.kotlinruntime.DefaultErrorStrategy.sync(DefaultErrorStrategy.kt:271) at kotlinx.ast.grammar.kotlin.target.antlr.kotlin.generated.KotlinParser.kotlinFile(KotlinParser.kt:1108) at kotlinx.ast.grammar.kotlin.target.antlr.kotlin.KotlinGrammarAntlrKotlinParserExtractor$extractor$1.invoke(KotlinGrammarAntlrKotlinParserAstParserType.kt:18) at kotlinx.ast.grammar.kotlin.target.antlr.kotlin.KotlinGrammarAntlrKotlinParserExtractor$extractor$1.invoke(KotlinGrammarAntlrKotlinParserAstParserType.kt:9) at kotlinx.ast.common.AstParserExtractor$DefaultImpls.extract(AstParserExtractor.kt:5) at kotlinx.ast.parser.antlr.kotlin.AntlrKotlinParserExtractor$DefaultImpls.extract(AntlrKotlinParserExtractor.kt) at kotlinx.ast.grammar.kotlin.target.antlr.kotlin.KotlinGrammarAntlrKotlinParserExtractor.extract(KotlinGrammarAntlrKotlinParserAstParserType.kt:9) at kotlinx.ast.grammar.kotlin.target.antlr.kotlin.KotlinGrammarAntlrKotlinParserExtractor.extract(KotlinGrammarAntlrKotlinParserAstParserType.kt:9) at kotlinx.ast.parser.antlr.kotlin.AntlrKotlinAstParserImplKt.antlrKotlinParser(AntlrKotlinAstParserImpl.kt:165) at kotlinx.ast.parser.antlr.kotlin.AntlrKotlinAstParserImplKt.antlrKotlinParser(AntlrKotlinAstParserImpl.kt:121) at kotlinx.ast.parser.antlr.kotlin.AntlrKotlinParser.parse(AntlrKotlinParser.kt:20) at kotlinx.ast.grammar.kotlin.common.KotlinGrammarParser$DefaultImpls.parseKotlinFile(KotlinGrammarParser.kt:9) at kotlinx.ast.grammar.kotlin.target.antlr.kotlin.KotlinGrammarAntlrKotlinParser.parseKotlinFile(KotlinGrammarAntlrKotlinParser.kt:10) at org.jetbrains.kotlin.idea.scratch.generated.ScratchFileRunnerGenerated$ScratchFileRunnerGenerated.(tmp.kt:22) at org.jetbrains.kotlin.idea.scratch.generated.ScratchFileRunnerGenerated.main(tmp.kt:33)

The problem has to be the line break after ${ because, without that lines, the execution goes smooth

            dateRangePropertyView.text =
                "${monthDayYearDateFormat.format(viewState.dateRange.startDate)} - ${
                    monthDayYearDateFormat.format(viewState.dateRange.endDate)
                }"

Library version

com.github.kotlinx.ast:grammar-kotlin-parser-antlr-kotlin-jvm:a96e681f90

The parser now handles "escaped" characters inside annotations' values but I have found another missing node.

The AST does not contain constructor parameters and properties declared in constructor e.g.:

class TestClass(
    @Value("test value")
    private val propertyA: String
)

Hi I would like to add some additional keyword (labels) knows from Spock library: Given, When, Then, Expect, And. These labels would be enabled only when junit test class is annotated with my custom annotation for instance @BddScenario.

I think that currently is not possible or maybe I am wrong?

Creating a file like this:

package testpackage.

val LARGE_JSON = """
     <very large json with 77000 lines>
"""

Running

val source = AstSource.String("", stringSource)
val kotlinFile = KotlinGrammarAntlrKotlinParser.parseKotlinFile(source)

takes very long on a file like this. If I replace the JSON in the string with base64, it will be much faster. Does the grammar somehow inspect the braces inside the templating?

I'm parsing the following Kotlin code kotlinx.ast and the KotlinGrammarAntlrKotlinParser. But weirdly, I have some nullable types which are not marked as nullable.

package foo.bar
class Spam {
    fun doSomething(i: Int, s: String?, l : List<String>) {}
} 

Here's what I get when printing the AST nodes

PackageHeader(foo.bar)
importList
KlassDeclaration(object Spam)
PackageHeader(foo.bar)
importList
KlassDeclaration(class Spam)
  classBody
    KlassDeclaration(fun doSomething)
      KlassDeclaration(parameter i Int)
      KlassDeclaration(parameter s String)
      KlassDeclaration(parameter l List<String>)

I'm using version c35b50fa44 from JitPack :

implementation("com.github.kotlinx.ast:grammar-kotlin-parser-antlr-kotlin-jvm:c35b50fa44")

Hi, Please help in what I am doing wrong I am unable to use this AstSource

Dependencies used -

    implementation 'com.github.drieks.antlr-kotlin:antlr-kotlin-runtime-jvm:b09d76328'
    implementation 'com.github.drieks.antlr-kotlin:antlr-kotlin-runtime:b09d76328'
    api("com.github.kotlinx.ast:grammar-kotlin-parser-antlr-kotlin:b09d76328")

Sample code -

    val source = AstSource.File(
        "grammar-kotlin-parser-antlr-kotlin/src/jvmTest/kotlin/kotlinx/ast/example/ExampleMain.kt"
    )
    val kotlinFile = KotlinGrammarAntlrKotlinParser.parseKotlinFile(source)
    kotlinFile.summary(attachRawAst = false)
        .onSuccess { astList ->
            astList.forEach(Ast::print)
        }.onFailure { errors ->
            errors.forEach(::println)
        }
}

Error - Unable to import class - AstSource

Hi, thanks for the work!

I am fresh to Kotlin, and I want to convert a Kotlin file to AST.

Here is my Kotlin file:

package com.kickstarter.libs.utils

import android.content.Context
import android.net.Uri
import androidx.browser.customtabs.CustomTabsIntent
import androidx.core.content.ContextCompat
import com.kickstarter.R

object UrlUtils {

    private const val KEY_REF = "ref"

    fun appendPath(baseUrl: String, path: String): String {
        val uriBuilder = Uri.parse(baseUrl).buildUpon()
        uriBuilder.appendEncodedPath(path)

        return uriBuilder.build().toString()
    }

    fun appendQueryParameter(baseUrl: String, key: String, value: String): String {
        val uriBuilder = Uri.parse(baseUrl).buildUpon()
        uriBuilder.appendQueryParameter(key, value)

        return uriBuilder.build().toString()
    }

    fun appendRefTag(baseUrl: String, tag: String): String {
        return appendQueryParameter(baseUrl, KEY_REF, tag)
    }

    fun baseCustomTabsIntent(context: Context): CustomTabsIntent {
        val builder = CustomTabsIntent.Builder()

        builder.setShowTitle(true)
        builder.setToolbarColor(ContextCompat.getColor(context, R.color.primary))

        return builder.build()
    }

    fun refTag(url: String): String? {
        return Uri.parse(url).getQueryParameter(KEY_REF)
    }
}

Here is the test code:

package kotlinx.ast.example

import kotlinx.ast.common.AstSource
import kotlinx.ast.common.ast.Ast
import kotlinx.ast.common.print
import kotlinx.ast.grammar.kotlin.common.summary
import kotlinx.ast.grammar.kotlin.target.antlr.kotlin.KotlinGrammarAntlrKotlinParser

fun main() {
    val source = AstSource.File(
            "gen.kotlin/src/main/resources/2.kt.txt"
    )
    val kotlinFile = KotlinGrammarAntlrKotlinParser.parseKotlinFile(source)
    kotlinFile.summary(attachRawAst = false)
            .onSuccess { astList ->
                astList.forEach(Ast::print)
            }.onFailure { errors ->
                errors.forEach(::println)
            }
}

And here is the result, which only prints the top-level ones:

PackageHeader(com.kickstarter.libs.utils)
importList
  Import(android.content.Context)
  Import(android.net.Uri)
  Import(androidx.browser.customtabs.CustomTabsIntent)
  Import(androidx.core.content.ContextCompat)
  Import(com.kickstarter.R)
KlassDeclaration(object UrlUtils)

However, I found that the children for the top-level object AST is empty, then how to print the whole structure of the file?

Bumps multiplatform from Versions.kotlin to 1.8.0.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

I am just coming from https://github.com/cretz/kastree as it points me to here.

Bascially what I want to do is to insert a new element. kastree can modify the node, but does not seem to be able to insert a node. So I came here and take a look. Loos like this library can only parse but cannot insert an element? https://github.com/kotlinx/ast/issues/43

Bumps logback-classic from 1.2.10 to 1.4.5.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Due to unreliability of jitpack.io repository (https://github.com/jitpack/jitpack.io/issues/5239) I wanted to ask if it is possible to publish somewhere else, like maven central or anything. Our builds are constantly failing due to this.

Bumps gradle/wrapper-validation-action from 1.0.4 to 1.0.5.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.