SceneView Android - 3D/AR Android View with ARCore and Google Filament

SceneView Open Community

Last update: Dec 28, 2022

Related tags

App android augmented-reality 3d filament arcore

Overview

SceneView Android

3D/AR Android View with ARCore and Google Filament

This is a Sceneform replacement in Kotlin

Features

Use SceneView for 3D only or ArSceneView for 3D and ARCore.
Everything is accessible at the SceneView/ArSceneView level. For example, no more ~~ArFragment~~ and code like ~~arFragment.arSceneView.scene~~, ~~arFragment.session.config~~, etc.
Just add the tag to your layout or call the ArSceneview(context: Context) constructor in your code. Compose is coming next ...
Requesting the camera permission and installing/updating the Google Play Services for AR is handled automatically in the ArSceneView.
Support for the latest ARCore features (the upcoming features will be integrated quicker thanks to Kotlin).
Lifecycle-aware components = Better memory management and performance.
Resources are loaded using coroutines launched in the LifecycleCoroutineScope of the SceneView/ArSceneView. This means that loading is started when the view is created and cancelled when it is destroyed.
Multiple instances are now possible.
Much easier to use. For example, the local and world position, rotation and scale of the Node are now directly accessible without creating ~~Vector3~~ objects (position.x = 1f, rotation = Rotation(90f, 180f, 0f), scale = Scale(0.5f), etc.).

Dependency

app/build.gradle

3D only

dependencies {
    // 3D only
    implementation 'io.github.sceneview:sceneview:0.6.0'
}

API Reference

3D and ARCore

dependencies {
    // 3D and ARCore
    implementation 'io.github.sceneview:arsceneview:0.6.0'
}

API Reference

Usage

res/layout/main_fragment.xml

3D only

<io.github.sceneview.SceneView
    android:id="@+id/sceneView"
    android:layout_width="match_parent"
    android:layout_height="match_parent" />

3D and ARCore

<io.github.sceneview.ar.ArSceneView
    android:id="@+id/sceneView"
    android:layout_width="match_parent"
    android:layout_height="match_parent" />

ARCore Geospatial API

Configure session

arSceneView.configureSession { session, config ->
    // Enable Geospatial Mode.
    config.geospatialMode = Config.GeospatialMode.ENABLED
}

Create an Anchor

val earth = arSceneView.session?.earth ?: return
if (earth.trackingState == TrackingState.TRACKING) {
    // Place the earth anchor at the same altitude as that of the camera to make it easier to view.
    val altitude = earth.cameraGeospatialPose.altitudeMeters - 1
    // The rotation quaternion of the anchor in the East-Up-South (EUS) coordinate system.
    val qx = 0f
    val qy = 0f
    val qz = 0f
    val qw = 1f
    earthAnchor = earth.createAnchor(latLng.latitude, latLng.longitude, altitude, qx, qy, qz, qw)
}

Camera Permission and ARCore install/update/unavailable

ArSceneView automatically handles the camera permission prompt and the ARCore requirements checks. Everything is proceed when the attached view Activity/Fragment is resumed but you can also add your ArSceneView at any time, the prompt will then occure when first addView(arSceneView) is called.

If you need it, you can add a listener on both ARCore success or failed session creation (including camera permission denied since a session cannot be created without it)

Camera permission has been granted and latest ARCore Services version are already installed or have been installed during the auto check

sceneView.onArSessionCreated = { arSession: ArSession ->
}

Handle a fallback in case of camera permission denied or AR unavailable and possibly move to 3D only usage

sceneView.onArSessionFailed = { exception: Exception ->
    // If AR is not available, we add the model directly to the scene for a 3D only usage
    sceneView.addChild(modelNode)
}

The exception contains the failure reason. e.g. SecurityException in case of camera permission denied

Customizing the instructions

The default instruction nodes have a ViewRenderable with a TextView or ImageView
The text and images of the instruction nodes can be overridden at the resource level (in the strings.xml file and drawable directory of your project).
Custom instruction nodes can have an arbitrary number of child nodes with ModelRenderables and ViewRenderables. It is even possible to play animation for a ModelRenderable if it is defined in a .glb file or a video using the VideoNode
The infoNode can have one of the following values depending on the ARCore features used and the current ARCore state: searchPlaneInfoNode, tapArPlaneInfoNode and augmentedImageInfoNode. Alternatively, it is possible to create your own instruction nodes.
The SearchPlaneInfoNode displays messages related to the ARCore state. For example, Searching for surfaces..., Too dark. Try moving to a well-lit area, Moving too fast. Slow down, etc.
The TapArPlaneInfoNode displays a message that helps users to understand how an object can be placed in AR when no objects are currently present in the scene.
The AugmentedImageInfoNode displays a frame with white corners when no augmented image is currently tracked.

💡 Idea for future: when access to the flashlight is finally available with the ARCore shared CameraManager, it will be great to add a button to the SearchPlaneInfoNode to enable the flashlight when there isn't enough light.

Why have we included the Kotlin-Math library in SceneView?

Earlier versions of OpenGL had a fixed rendering pipeline and provided an API for setting positions of vertices, transformation and projection matrices, etc. However, with the new rendering pipeline it is required to prepare this data before passing it to GLSL shaders and OpenGL doesn't provide any mathematical functions to do that.

It is possible to implement the required functions yourself like in Sceneform or use an existing library. For example, C++ supports operator overloading and benefits from the excellent GLM library that allows to use the same syntax and features as GLSL.

We use the Kotlin-Math library to rely on a well-tested functions and get an advantage of using Kotlin operators for vector, matrix and quaternion operations too.

Migration from Sceneform

You will have a little work to do if you are using the ArFragment in Sceneform. However, there is the Deprecated.kt file to help you with the migration.

Using the migration suggestions

Remove the Sceneform import for the class you want to migrate.
Import this class from the io.github.sceneview.ar package.
Use Alt+Enter/the light bulb icon to view and apply the suggestions for replacing the deprecated method calls.

After the migration you should get cleaner code and all of the benefits described in the Features section 🎉

Requesting the camera permission and installing/updating the Google Play Services for AR

This is handled automatically in the ArSceneView. You can use the ArSceneView.onArSessionFailed property to register a callback to be invoked when the ARCore Session cannot be initialized because ARCore is not available on the device or the camera permission has been denied.

Instructions for AR

The InstructionsController in the BaseArFragment has been replaced with the Instructions in the ArSceneView.

The Instructions use a Node that is a part of the scene instead of a View, as opposed to the InstructionsController. This provides more flexibility for customizing the instructions. The Instructions have the main Node that can be accessed through the Instructions.infoNode property.

Comments

SceneView multiple gesture detectors

I added a gesture detector but it is not working as it should. I created a sample app called sample-model-gestures to test out my gesture detector but it is very buggy. Below is the modification I made to SceneView.kt:

inner class OnGestureListener : GestureDetector.SimpleOnGestureListener() {
    override fun onSingleTapUp(motionEvent: MotionEvent): Boolean {
        val hitTestResult = surfaceGestureDetector.pickHitResult
        onTouch(hitTestResult.node, motionEvent)
        return true
    }

    override fun onDown(e: MotionEvent): Boolean {
        return true
    }

    override fun onSingleTapConfirmed(e: MotionEvent?): Boolean {
        gestureType = GestureType.SINGLE_TAP
        return super.onSingleTapConfirmed(e)
    }

    override fun onDoubleTap(e: MotionEvent?): Boolean {
        gestureType = GestureType.DOUBLE_TAP
        return super.onDoubleTap(e)
    }

    override fun onLongPress(e: MotionEvent?) {
        super.onLongPress(e)
        gestureType = GestureType.LONG_PRESS
    }

    // https://gist.github.com/TheIcemanCometh/57d0f9ae7b70dcdcc4d9
    override fun onFling(
        e1: MotionEvent?,
        e2: MotionEvent?,
        velocityX: Float,
        velocityY: Float
    ): Boolean {
        if (e1 != null && e2 != null) {
            val diffY = abs(e1.y - e2.y)
            val diffX = abs(e1.x - e2.x)
            if (diffY > SWIPE_MAX_OFF_PATH) {
                if (diffX > SWIPE_MAX_OFF_PATH || abs(velocityY) < SWIPE_THRESHOLD_VELOCITY) {
                    return false
                }
                if (e1.y - e2.y > SWIPE_MIN_DISTANCE) {
                    gestureType = GestureType.FLING_UP
                }
                else if (e2.y - e1.y > SWIPE_MIN_DISTANCE) {
                    gestureType = GestureType.FLING_DOWN
                }
            }
            else {
                if (abs(velocityX) < SWIPE_THRESHOLD_VELOCITY) {
                    return false
                }
                if (e1.x - e2.x > SWIPE_MIN_DISTANCE) {
                    gestureType = GestureType.FLING_LEFT
                }
                else if (e2.x - e1.x > SWIPE_MIN_DISTANCE) {
                    gestureType = GestureType.FLING_RIGHT
                }
            }
        }
        return super.onFling(e1, e2, velocityX, velocityY)
    }
}

Please have a look and let me know what I could be doing wrong. Thanks!

enhancement

opened by sarimmehdi 43

Add CloudAnchorNode
I have been having a very strange issue in my project that occurs after resolving AR Core Cloud Anchors, where I am not entirely sure if this error is related to sceneview, but since I am unable to find the root cause of this error, I figured you might know something or have had a similar problem like this before.

So my app does not crash with any "normal" exceptions or stacktrace, but when navigating away from my AR fragment, the app will just close with the following error message that doesn't really help too much:

I/native: I0304 19:19:53.140083 16305 session_lite_c_api.cc:37] Deleting ArSession... A/libc: Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x6c00000018 in tid 25038 (Thread-20), pid 16305 (nn.arsamplecode)

While trying to find the cause, I could pinpoint it on callling session.resolveCloudAnchor() with an existing anchor ID. Turns out, while AR Core is still trying to resolve a cloud anchor and hasn't found it yet, it will crash when trying to navigate away from the fragment. As soon as the anchor is successfully resolved, navigating works perfectly fine again.

I created a small sample to hopefully reproduce the issue, though for that you will have to use your own Cloud Anchor API Key and an existing cloud anchor ID. In the sample, tapping on the AR plane will call the resolveCloudAnchor, and then simply navigating back will crash the app.

Since I really am out of approaches of how to fix this, I was hoping that maybe y'all have some experience with these kind of errors or potentially have an idea how I can solve this issue. Thanks alot in advance
enhancement
opened by morhenny 24

Augmented Face - Not Rendering Model and Still in "surface mode"

Hello, I was trying to understand learn and understand "sceneview" usages, I already discovered how to get Augmented face in "sceneform-android" but when I try sceneview, I encountered several problems,

First camera settings are seems to be not working properly because some options act wrong. (For example MESH3D mode or I was able to enable Deph mode (normally I shouldn't be able to do that in Front Camera) )
Model size is too small
Model moving opposite side when I move my head (If I go right, model goes left)
Model not render properly and shows me "black material"
I'm always seeing "Searching for surfaces..." (front camera or back camera) It should not be visible in front camera

Here my minimal code (only MainActivity)


    private lateinit var binding: ActivityMainBinding
    private val facesNodes = HashMap<AugmentedFace, AugmentedFaceNode>()
    private var faceModel: ModelRenderable? = null
    
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        binding = ActivityMainBinding.inflate(layoutInflater)
        setContentView(binding.root)
        loadModels()

    }
    
    private fun loadModels() {

        lifecycleScope.launchWhenCreated {
            binding.sceneView.configureSession { arSession, config ->
                val filter =
                    CameraConfigFilter(arSession).setFacingDirection(CameraConfig.FacingDirection.FRONT)
                val cameraConfig = arSession.getSupportedCameraConfigs(filter)[0]
                arSession.cameraConfig = cameraConfig
                config.instantPlacementMode = Config.InstantPlacementMode.DISABLED

                config.augmentedFaceMode = Config.AugmentedFaceMode.MESH3D
                arSession.configure(config)
            }


            faceModel = loadModel(
                context = this@MainActivity,
                lifecycle = lifecycle,
                glbFileLocation = "models/fox.glb",
            )

            binding.sceneView.onAugmentedFaceUpdate = {
                val existingFaceNode = facesNodes[it]
                when (it.trackingState) {
                    TrackingState.TRACKING ->
                        if (existingFaceNode == null) {
                            val faceNode = AugmentedFaceNode(binding.sceneView.lifecycle, it)
                            val modelInstance = faceNode.setFaceRegionsRenderable(faceModel)
                            modelInstance.isShadowCaster = false
                            modelInstance.isShadowReceiver = true
                            faceNode.faceRegionsRenderable = faceModel!!
                            binding.sceneView.addChild(faceNode)
                            facesNodes[it] = faceNode
                        }
                    TrackingState.STOPPED -> {
                        if (existingFaceNode != null) {
                            binding.sceneView.removeChild(existingFaceNode)
                            facesNodes.remove(it)
                        }
                        facesNodes.remove(it)
                    }
                    else -> {
                        facesNodes.remove(it)
                    }
                }
            }

        }
    }

activity_main.xml snippet as well

    <io.github.sceneview.ar.ArSceneView
        android:id="@+id/sceneView"
        android:layout_width="0dp"
        android:layout_height="0dp"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintLeft_toLeftOf="parent"
        app:layout_constraintRight_toRightOf="parent"
        app:layout_constraintTop_toTopOf="parent" />

PS : I used "fox and "face" glb files from sceneform for quick testing.

If you need additional information please let me know.

Thank you.

Stale

opened by onuralpszr 23

FootprintSelectionVisualizer not displaying footprint

I'm trying to set a foot print shadow but I'm not seeing one when I try to add a FootprintSelectionVisualizer.

Am I missing a step?

    private fun onModelLoaded(modelRenderable: ModelRenderable) {

        val footprintSelectionVisualizer = FootprintSelectionVisualizer().apply {
            footprintRenderable = modelRenderable // this doesn't seem to do anything
        }
        val transformationSystem = TransformationSystem(context.resources.displayMetrics, footprintSelectionVisualizer)

        val node = DragTransformableNode(1f, transformationSystem).also {
            it.renderable = modelRenderable
            scene.addChild(it)
            it.select()
        }

        scene.addOnPeekTouchListener { hitTestResult: HitTestResult?, motionEvent: MotionEvent? ->
            transformationSystem.onTouch(hitTestResult, motionEvent)
        }

    }

opened by kelvinwatson 17

ArNodeManipulator + CameraManipulator (Filament Manipulator)
I'd like to have an ARModelNode that can be moved and rotated using gestures. I guess I can't use the ARModelNode directly as the TransformableNode needs a ArNode as its parent. The way to go would be this node hierachy:

ArNode

TransformableNode

ModelNode

This results in my model rotating around the anchor instead of rotating around itself. Could you help me to achieve the right result?

Thanks!
enhancement
opened by StevenMOG 13

Memory leak

I just realized in my application that the ArSceneView.renderer is leaking so I've tried in the sample-ar-model-viewer demo and it seems that the leak is coming from the library. I could reproduce it by simply adding the leak-canary library to the sample. I've tried to clear the ArSceneView in the onDestroyView() of the fragment by setting it to null, but it didn't help. Here is the leak-canary heap analysis result:

HEAP ANALYSIS RESULT
====================================
1 APPLICATION LEAKS
References underlined with "~~~" are likely causes.
Learn more at https://squ.re/leaks.
1029 bytes retained by leaking objects
Signature: 99bf7c9da341a49ffdcbea9aef692c7f6673b96e
┬───
│ GC Root: System class
│
├─ android.view.inputmethod.InputMethodManager class
│    Leaking: NO (InputMethodManager↓ is not leaking and a class is never
│    leaking)
│    ↓ static InputMethodManager.sInstance
├─ android.view.inputmethod.InputMethodManager instance
│    Leaking: NO (ArSceneView↓ is not leaking and InputMethodManager is a
│    singleton)
│    ↓ InputMethodManager.mCurRootView
├─ android.view.ViewRootImpl instance
│    Leaking: NO (ArSceneView↓ is not leaking)
│    mContext instance of com.android.internal.policy.DecorContext, wrapping
│    activity io.github.sceneview.sample.armodelviewer.Activity with mDestroyed
│    = false
│    ViewRootImpl#mView is not null
│    mWindowAttributes.mTitle = "io.github.sceneview.sample.armodelviewer/io.
│    github.sceneview.sample.armodelviewer.Activity"
│    mWindowAttributes.type = 1
│    ↓ ViewRootImpl.mSurfaceChangedCallbacks
├─ java.util.ArrayList instance
│    Leaking: NO (ArSceneView↓ is not leaking)
│    ↓ ArrayList[0]
├─ io.github.sceneview.ar.ArSceneView instance
│    Leaking: NO (View attached)
│    View is part of a window view hierarchy
│    View.mAttachInfo is not null (view attached)
│    View.mID = R.id.sceneView
│    View.mWindowAttachCount = 1
│    mContext instance of io.github.sceneview.sample.armodelviewer.Activity
│    with mDestroyed = false
│    ↓ ArSceneView.renderer
│                  ~~~~~~~~
├─ com.google.ar.sceneform.rendering.ArRenderer instance
│    Leaking: UNKNOWN
│    Retaining 2.1 kB in 40 objects
│    ↓ Renderer.viewAttachmentManager
│               ~~~~~~~~~~~~~~~~~~~~~
├─ com.google.ar.sceneform.rendering.ViewAttachmentManager instance
│    Leaking: UNKNOWN
│    Retaining 1.3 kB in 21 objects
│    ↓ ViewAttachmentManager.frameLayout
│                            ~~~~~~~~~~~
╰→ android.widget.FrameLayout instance
     Leaking: YES (ObjectWatcher was watching this because android.widget.
     FrameLayout received View#onDetachedFromWindow() callback)
     Retaining 1.0 kB in 16 objects
     key = 974cd537-0dbb-4f32-9f2e-0a60998aa9d8
     watchDurationMillis = 5934
     retainedDurationMillis = 887
     View not part of a window view hierarchy
     View.mAttachInfo is null (view detached)
     View.mWindowAttachCount = 1
     mContext instance of io.github.sceneview.sample.armodelviewer.Activity
     with mDestroyed = false
====================================
0 LIBRARY LEAKS
A Library Leak is a leak caused by a known bug in 3rd party code that you do
not have control over.
See https://square.github/.
io/leakcanary/fundamentals-how-leakcanary-works/#4-categorizing-leaks
====================================
0 UNREACHABLE OBJECTS
An unreachable object is still in memory but LeakCanary could not find a strong
reference path
from GC roots.
====================================
METADATA
Please include this in bug reports and Stack Overflow questions.
Build.VERSION.SDK_INT: 30
Build.MANUFACTURER: Google
LeakCanary version: 2.9.1
App process name: io.github.sceneview.sample.armodelviewer
Class count: 19745
Instance count: 175695
Primitive array count: 92345
Object array count: 22352
Thread count: 43
Heap total bytes: 77566459
Bitmap count: 2
Bitmap total bytes: 180002
Large bitmap count: 0
Large bitmap total bytes: 0
Stats: LruCache[maxSize=3000,hits=38513,misses=102077,hitRate=27%]
RandomAccess[bytes=4736305,reads=102077,travel=32772117401,range=25257049,size=8
7653658]
Analysis duration: 22954 ms
Heap dump file path: /storage/emulated/0/Download/leakcanary-io.github.
sceneview.sample.armodelviewer/2022-05-23_15-36-58_886.hprof
Heap dump timestamp: 1653309451124
Heap dump duration: Unknown
====================================

Stale

opened by tamashalcyon 11

Unable to get a model to show in Java

I'm trying to get the sample model displayed in my Java code. I've got everything working in SceneForm but I'm trying to migrate to this project for the memory management improvements. What am I doing wrong here? (This is a native android component I'm bridging for use in React Native so this class is being rendered inside of a Fragment.)

Here is my code:

class RNTExerciseView extends FrameLayout {

    private SceneView sceneView;
    private ModelNode characterModel;
    private Node cameraOrbit;
    private Camera camera;

    private ObjectAnimator currentlyPlaying;

    public RNTExerciseView(@NonNull Context context, ReactContext reactContext, int reactNativeViewId) {
        super(context);

        sceneView = new SceneView(context);
        sceneView.setBackgroundColor(Color.rgb(255, 255, 255));
        sceneView.setTransparent(true);
        sceneView.setZOrderOnTop(false);
        this.addView(sceneView);

        characterModel = new ModelNode();
        sceneView.addChild(characterModel);
        characterModel.setWorldPosition(new Float3(0, 0, -5));

        characterModel.loadModelAsync(
            context,
            "https://sceneview.github.io/assets/models/MaterialSuite.glb",
            null,
            true,
            true,
            new Float3(0, 0, 0), 
            null, 
            instance -> {
                return null;
            });
    }
}

opened by ninjz 11

3D Model gets re-rendered after the user moves the 3D object

Hi I am still new to the whole AR thing and I don't know how this framework actually work but it's great now I am facing a problem that the shape is added over and over again and it's not re-rendered upon here is the photo of the shape : here is the photo of the shape after trying to move it down and up : I am doing something wrong I don't know what it is here is the source code that I wrote :
Stale

opened by ayasser-identity 10

How to use with Geospatial API

Hey there! Thank you for this useful library which saves a lot of time when working with ARCore :)

I'm trying to create an android app that shows AR markers by specific location (LatLng). I'd like to create something like in this video (from README AR Geospatial API), but with simple ViewNode instead of ArModelNode.

I also can't place AR model at a specific location with a sample code from README. The model spawns just in front of the current camera position.

Here is my code:

val earth = binding.sceneView.arSession?.earth ?: return@launchWhenResumed
if (earth.trackingState == TrackingState.TRACKING) {
    val cameraGeospatialPose = earth.cameraGeospatialPose
    val earthAnchor = earth.createAnchor(49.844141, 24.024532, cameraGeospatialPose.altitude, 0f, 0f, 0f, 1f)
    val arModelNode = ArModelNode(
        context = this@MainActivity,
        lifecycle = lifecycle,
        modelFileLocation = "https://sceneview.github.io/assets/models/Gumball.glb"
    )
    arModelNode.anchor = earthAnchor
    binding.sceneView.addChild(arModelNode)
}

I would be grateful for your help!

Stale

opened by lincollincol 10

Sceneview visual glitch, multiplying objects in android

(1)This is link of a video of third party model viewer app where how model loaded

https://drive.google.com/file/d/1SelSOpWpNfQIGYIquvCqT3yeOHzk_8hJ/view?usp=sharing

(2) This is the link of video where same model is loaded in sceneview in my app

https://drive.google.com/file/d/1sreSZbrUgxSwO9-Nq7uYRjCwT1vL_epQ/view?usp=sharing

(3) I Found this warning in My XML File

(4) This warning in sceneview.kt

Please Help me

opened by vicode32 10
Changed the default mode for the PlaneRenderer.

PlaneRenderMode.RENDER_ALL was set as the default mode but the doc states clearly that this mode is very expensive. A short check on the Profile verified this statement with 500ms spikes. The default mode is now set to PlaneRenderMode.RENDER_TOP_MOST.

opened by RGregat 10
"Sceneview" and "Sceneform" dependencies cannot be added together

When I added them together I got this error

Duplicate class com.google.ar.sceneform.ArHelpers found in modules arsceneview-0.9.1-runtime (io.github.sceneview:arsceneview:0.9.1) and core-1.21.0-runtime (com.gorisse.thomas.sceneform:core:1.21.0) Duplicate class com.google.ar.sceneform.Sceneform found in modules core-1.21.0-runtime (com.gorisse.thomas.sceneform:core:1.21.0) and sceneview-0.9.0-runtime (io.github.sceneview:sceneview:0.9.0) Duplicate class com.google.ar.sceneform.SequentialTask found in modules core-1.21.0-runtime (com.gorisse.thomas.sceneform:core:1.21.0) and sceneview-0.9.0-runtime (io.github.sceneview:sceneview:0.9.0) Duplicate class com.google.ar.sceneform.Trackables found in modules arsceneview-0.9.1-runtime (io.github.sceneview:arsceneview:0.9.1) and core-1.21.0-runtime (com.gorisse.thomas.sceneform:core:1.21.0) Duplicate class com.google.ar.sceneform.animation.AnimatableModel found in modules core-1.21.0-runtime (com.gorisse.thomas.sceneform:core:1.21.0) and sceneview-0.9.0-runtime (io.github.sceneview:sceneview:0.9.0) Duplicate class com.google.ar.sceneform.animation.ModelAnimation found in modules core-1.21.0-runtime (com.gorisse.thomas.sceneform:core:1.21.0) and sceneview-0.9.0-runtime (io.github.sceneview:sceneview:0.9.0) Duplicate class com.google.ar.sceneform.animation.ModelAnimation$1 found in modules core-1.21.0-runtime (com.gorisse.thomas.sceneform:core:1.21.0) and sceneview-0.9.0-runtime (io.github.sceneview:sceneview:0.9.0) Duplicate class com.google.ar.sceneform.animation.ModelAnimation$2 found in modules core-1.21.0-runtime (com.gorisse.thomas.sceneform:core:1.21.0) and sceneview-0.9.0-runtime (io.github.sceneview:sceneview:0.9.0) Duplicate class com.google.ar.sceneform.animation.ModelAnimation$3 found in modules core-1.21.0-runtime (com.gorisse.thomas.sceneform:core:1.21.0) and sceneview-0.9.0-runtime (io.github.sceneview:sceneview:0.9.0) Duplicate class com.google.ar.sceneform.animation.ModelAnimation$PropertyValuesHolder found in modules core-1.21.0-runtime (com.gorisse.thomas.sceneform:core:1.21.0) and sceneview-0.9.0-runtime (io.github.sceneview:sceneview:0.9.0) Duplicate class com.google.ar.sceneform.animation.ModelAnimator found in modules core-1.21.0-runtime (com.gorisse.thomas.sceneform:core:1.21.0) and sceneview-0.9.0-runtime (io.github.sceneview:sceneview:0.9.0) Duplicate class com.google.ar.sceneform.animation.ModelAnimator$1 found in modules core-1.21.0-runtime (com.gorisse.thomas.sceneform:core:1.21.0) and sceneview-0.9.0-runtime (io.github.sceneview:sceneview:0.9.0) Duplicate class com.google.ar.sceneform.animation.ModelAnimator$2 found in modules core-1.21.0-runtime (com.gorisse.thomas.sceneform:core:1.21.0) and sceneview-0.9.0-runtime (io.github.sceneview:sceneview:0.9.0) Duplicate class com.google.ar.sceneform.animation.ModelAnimator$PropertyValuesHolder found in modules core-1.21.0-runtime (com.gorisse.thomas.sceneform:core:1.21.0) and sceneview-0.9.0-runtime (io.github.sceneview:sceneview:0.9.0) Duplicate class com.google.ar.sceneform.animation.ModelAnimator$PropertyValuesHolder$AnimationProperty found in modules core-1.21.0-runtime (com.gorisse.thomas.sceneform:core:1.21.0) and sceneview-0.9.0-runtime (io.github.sceneview:sceneview:0.9.0) Duplicate class com.google.ar.sceneform.collision.Box found in modules core-1.21.0-runtime (com.gorisse.thomas.sceneform:core:1.21.0) and sceneview-0.9.0-runtime (io.github.sceneview:sceneview:0.9.0) Duplicate class com.google.ar.sceneform.collision.Collider found in modules core-1.21.0-runtime (com.gorisse.thomas.sceneform:core:1.21.0) and sceneview-0.9.0-runtime (io.github.sceneview:sceneview:0.9.0) Duplicate class com.google.ar.sceneform.collision.CollisionShape found in modules core-1.21.0-runtime (com.gorisse.thomas.sceneform:core:1.21.0) and sceneview-0.9.0-runtime (io.github.sceneview:sceneview:0.9.0) Duplicate class com.google.ar.sceneform.collision.CollisionSystem found in modules core-1.21.0-runtime (com.gorisse.thomas.sceneform:core:1.21.0) and sceneview-0.9.0-runtime (io.github.sceneview:sceneview:0.9.0) Duplicate class com.google.ar.sceneform.collision.Intersections found in modules core-1.21.0-runtime (com.gorisse.thomas.sceneform:core:1.21.0) and sceneview-0.9.0-runtime (io.github.sceneview:sceneview:0.9.0) ....................

opened by MohamedFouad999 4
SceneView 1.0 Release Date

Hi, do you have a timeline for the 1.0 release date? I am considering entering the ARCore Geospatial API Challenge, but wanted the TerrainAnchor support for the app I have.

opened by ryust 2
Crashed with creation ViewNode with Plane object

When I need to create a Plane object, I also have to use a ViewNode. But the Plane object, when created, lost some parameters that will be needed in the future, and I have a crash.

And I have to use ugly workarounds to create the Plane properly.

ViewNode:

I think the main source of the problem is the Plane.kt arch, because the Plane class has no parameters from Geometry.

@ThomasGorisse Maybe you should reconsider creating a Plane.kt class?

opened by ozh-dev 3
Model 3d always top view different and LoadingNode tags are too blurry
hi @ThomasGorisse @grassydragon, when scroll model 3d alway top view different, how to set model 3d equal view different?

I using LoadingNode loading tag but too blurry. How to improve quality view? Can you help me resolve it? thank so much
opened by thanhphong190 2

Releases(v0.9.1)

v0.9.1(Dec 19, 2022)
Dependencies update

ARCore 1.35.0

Fixes

Source code(tar.gz)
Source code(zip)
v0.9.0(Aug 18, 2022)
Features

RIP RenderableInstance for ModelNode directly access FilamentAsset

Added RenderableNode for all types of rendered nodes

Added hitTest functions at ArSceneView level

Get hitTest from Position at ArFrame level

Added cameraFacingDirection directly to ArSceneView

Added model animator extensions

Changed ModelNode to use applyAnimation and updateBoneMatrices directly with frame time instead of using an ObjectAnimator

Fixes

Fixed wrongs AR placement

Moved placementMode and onMove() from ArNode to ArModelNode for coming next AugmentedFaceNode and AugmentedImageNode

Refactor approximateDistanceMeters to approximateDistance

Fix wrong model animations interpolations

Documentation

Added more documentation in the source code

Source code(tar.gz)
Source code(zip)
v0.7.0(Jun 24, 2022)
Temporary Changes :warning:

The anchorPoseUpdateInterval property is set to null by default. This means that the node pose is never updated to the anchor actual pose. Set this property to one second, for example, to benefit from the anchor pose updates.

ARCore was rolled back to 1.30.0 since there were some performance issues encountered. You can still use the latest version of ARCore in your build.gradle file.

Features

Made the pan pan gesture configurable (it can be now disabled)

Replaced ArNode.onTrackingChanged with ArNode.onPoseChanged

Made the click duration in the CursorNode configurable

Added constructors for the ModelNode and ArModelNode that accept a path to a glTF model

Supported using multiple listeners for the Augmented Image updates

Improved node selection

Refactored materials to use the Filament MaterialInstance

Added a listener for the smooth transformation end

Added a listener for the model change in the ModelNode

Added an extension function to convert a Pose to a Transform (a transformation matrix)

Supported limiting the number of hit tests in the PlaneRenderer

Replaced ArModelNode.autoAnchor with ArModelNode.instantAnchor

Supported disabling the ArModelNode pose updates based on hit tests

Replaced TextureLoader.loadTexture with TextureLoader.loadImageTexture

Fixes

Fixed model picking (the touch coordinates are now correct)

Fixed synchronization of the 3D scene state and the camera frames

Fixed the default node position (while also fixing the selection visualizer position)

Fixed tap events and listeners for the SceneView and models (they are now using Filament picking)

Fixed destroying the PlaneVisualizer

Fixed setting the isShadowCaster and isShadowReceiver properties

Fixed canceling smooth transformations

Fixed the move gesture

Documentation

Added more documentation in the source code

Added more samples to README.md

Source code(tar.gz)
Source code(zip)
v0.6.0(May 11, 2022)
ARCore Geospatial API The new ARCore Geospatial API utilizes data from Google Earth 3D models and Street View image data from Google Maps to enable your app for immersive, global-scale, location-based augmented reality experiences.

Earth provides localization ability in geospatial coordinates. Earth.createAnchor() creates a new Anchor at the specified latitude, longitude, altitude, and orientation.

Earth.Earthstate describes the current state of Earth, including its TrackingState.

GeospatialPose describes a specific latitude, longitude, altitude, and compass heading.

ARCore Depth API improvements The maximum range of Depth measurements has increased from 8191mm to 65535mm. The Depth API now uses all 16-bits to represent depth, where previously only 13 bits were used for depth values.

Source code(tar.gz)
Source code(zip)
v0.5.2(May 6, 2022)
Features

Completely renewed ARCameraStream by moving it to Kotlin and 100% Filament

Cleaned and resource consumption improved

Moved to multithreaded rendering pipeline to improve quality and move to Filament stream less texture creation

Improved gesture rotation

Improved LightEstimation consumption

Changed ArNode anchor update interval

Material synchronous loading

Added Node custom name

Added Renderable extensions

Access SceneView within the Lifecycle

KTXLoader synchronous loading

Allow gesture detectors to be changed at runtime (#69)

Fixes

Fixed some possible wrong session config applying

Source code(tar.gz)
Source code(zip)
v0.5.1(Apr 20, 2022)
Features

RIP CleanupRegistry welcome LifecycleObservers: Filament 3D Entities (Models, Materials, Textures, Lights,...), Engine and Loaders/Managers creation and destroy are now completely made around Android Lifecycle observers. They were previously managed by the Garbage collector. We now insure that every single consuming resource is destroyed when the Activity/Fragment/SceneView is destroyed and when it is not needed anymore internally. The only counterpart is that you have to give a lifecycle parameter when building a Filament entity that can be any of the Activity/Fragment/SceneView one.

e.g. You now have to give a Lifecycle parameter instead of the CoroutineScope when loading a model

RIP IEngine, FilamentEngineWrapper, EngineInstance,... welcome Filament single retainable object that contains the engine instance and every linked Filament Loaders/Managers. The Filament Engine can now be shared between different SceneViews. The last releasing retainer will destroy the Engine and the Loaders/Managers.

Allowed gesture detectors to be changed at runtime: GestureDetectors are vars and can be changed at any time

Fixes

Fixed wrong SceneView MainLight and Environment initialisation because of NullPointerException when getting ArSceneView lifecycle

Fixed Camera Stream init

Fixed Filament Engine not destroyed

Fixed move gesture: Some conditions for the move gesture are inverted

Fixed some double Filament entity destroys causing exception when the SceneView was destroyed

Source code(tar.gz)
Source code(zip)
v0.4.1(Apr 14, 2022)
Features

Added a precision parameter for updating the ArNode pose from the anchor

Fixes

Allow setting gesture listeners without inheriting the SceneView or ArSceneView

Improved performance for estimation of AR environmental lighting

Source code(tar.gz)
Source code(zip)
v0.4.0(Apr 11, 2022)
Features

Added an ArNode manipulator for moving, rotating and scaling nodes in the ArSceneView and a Camera manipulator for panning and zooming the camera in SceneView

Improved AR environmental lighting and reflections by removing spectacular filtering

Added a precision parameter for estimation of AR environmental lighting to increase performance by skipping frames

Made the ARCore session configuration simpler

Fixes

Fixed the order of destroying the Filament resources to avoid an exception

Enabling the auto focus without pausing and resuming the ARCore session

Source code(tar.gz)
Source code(zip)
v0.3.0(Apr 7, 2022)
Features

Added Cloud Anchor support to the ArNode and a sample application

Supported setting the world position, quaternion, rotation and scale of the Node

Source code(tar.gz)
Source code(zip)
v0.2.1(Mar 30, 2022)
Features

ARCore 1.30.0

Fix Android SourceSets for sources.jar = You should now have access to SceneView sources from your project when using maven dependency

Add Gltf folder model source: You can use a .gltf and its folder as 3D model from Renderable call

Add instant placement fallback to PlacementMode: You can enable an instant plcament fallback when your current placement mode is not available yet or at all.

Added code style and contributing guidelines

Gradle 7.1.2

Fixes

Fix Instructions enable/disable

Source code(tar.gz)
Source code(zip)
v0.2.0(Mar 17, 2022)
Features

Improved rendering quality: Removed 3D rendering frames skipped based on FPS

Improved AR Precision: Removed and cleaned AR frame skipped

Reduced PlaneRender consumption when not visible

Reduced ArFrame consumption when no updated trackables (Plane, AugmentedImage, AugmentFace) are required

Refactored FrameTime to Kotlin

Retrieve current frame FPS

Added ModelNode.doOnModelLoaded to handle modifying the model when loaded

New cleaned Node smooth thresholds

No AR maxHitsPerSecond by default = Max precision

Added configurable session features

Added some colorOf functions with different parameters

Create an Anchor from an HitResult

Pose multiple rotations types

Added RenderableInstance object culling setter

Improved CursorNode

Added NodeSelection default SelectionVisualizer

Refactored session -> arSession and sessionConfig -> arSessionConfig

Change material parameter to Color

Retrieve Pose Euler Angles

Changed IblGenerator to handle hdr environment directory input instead of file

AR model viewer sample: Hide the plane renderer when the node is anchored

Public RenderableInstance.entity

Fixes

Fixed the too yellow model default reflection: Changed the default environment to a white studio

Fixed top most plane rendering and restored previous parameters (#46)

PlaneRenderMode.RENDER_ALL was set as the default mode but the doc states clearly that this mode is very expensive (#39)

Fixed ArSceneView and SceneView lifecycle listeners

Fixed CursorNode default placement mode

Fixed AugmentedImageInfoNode alignment

Refactored HDREnvironment

Fixed HDREnvironment double cubemap

Fixed session config order

Renamed Node.onFrameUpdate to onFrame

Changed ModelViewer sample background

Fixed model loading error not thrown

Documentation

Dokka Doc Markdown GFM (GitHub)

Dokka published to SceneView/sceneview.github.io

API reference link inside README.md

Issue and PR Autoclose

Added dictionary with spelling for renderable and renderables (#41)

Source code(tar.gz)
Source code(zip)
v0.1.0(Feb 28, 2022)
We thank everyone who has participated in the SceneView development, used the early version and reported the issues! :tada:

This release is production ready and available on mavenCentral(). The API is now stable that means you won't encounter significant API changes from this version. Some parts of the API may be deprecated in future like in any other libraries but the core API won't change.

You can still use everything you could use in Sceneform. Some internal classes and specific ones (for example, AugmentedFaceNode, AugmentedImageNode, etc.) can still be used but haven't been converted to Kotlin yet. That means that they can change a little with time because of all the improvements available in SceneView compared with Sceneform (ARCore Depth, Instant placement, lifecycle-aware components, etc.).

Have a look at this list to see what features are supported only in SceneView. They are awesome!

Here is short summary of the new things in this release:

Improvements in the API, source code and documentation

Latest versions of ARCore and Filament (1.29.0 and 1.20.0)

kotlin-math for mathematical operations with vectors, transformation matrices and quaternions

Node system now includes LightNode, LoadingNode, ModelNode, ViewNode, ArNode, ArModelNode, and CursorNode

ModelNode supports scaling and centering the attached model automatically and transforming it separately from the node

ArModelNode can be placed using the ARCore Plane, Point, DepthPoint and InstantPlacementPoint

Even smoother model placement in AR

Automatic material and environment compilation using the Filament plugin that can be enabled by contributors

Better handling of requesting the camera permission and installing/updating the Google Play Services for AR

Callbacks for handling errors when the ARCore session can't be created or OpenGL isn't available

Avoiding using the previously bound texture if the view texture isn't ready (the problem when the camera texture is shown instead of the ViewRenderable)

Avoiding screen flickering by setting the display geometry for the ARCore session at the right time (the problem when the ArSceneView shows different colors instead of the camera texture)

Source code(tar.gz)
Source code(zip)
v0.0.3(Dec 10, 2021)

Source code(tar.gz)
Source code(zip)