ios - ARKIT: Move Object with PanGesture (the right way)

Question

I've been reading plenty of StackOverflow answers on how to move an object by dragging it across the screen. Some use hit tests against .featurePoints some use the gesture translation or just keeping track of the lastPosition of the object. But honestly.. none work the way everyone is expecting it to work.

Hit testing against .featurePoints just makes the object jump all around, because you dont always hit a featurepoint when dragging your finger. I dont understand why everyone keeps suggesting this.

Solutions like this one work: Dragging SCNNode in ARKit Using SceneKit

But the object doesnt really follow your finger, and the moment you take a few steps or change the angle of the object or the camera.. and try to move the object.. the x,z are all inverted.. and makes total sense to do that.

I really want to move objects as good as the Apple Demo, but I look at the code from Apple... and is insanely weird and overcomplicated I cant even understand a bit. Their technique to move the object so beautifly is not even close to what everyone propose online. https://developer.apple.com/documentation/arkit/handling_3d_interaction_and_ui_controls_in_augmented_reality

There's gotta be a simpler way to do it.

Answer 1

I added some of my ideas to Claessons's answer. I noticed some lag when dragging the node around. I found that the node cannot follow the finger's movement.

To make the node move more smoothly, I added a variable that keeps track of the node that is currently being moved, and set the position to the location of the touch.

    var selectedNode: SCNNode?

Also, I set a .categoryBitMask value to specify the category of nodes that I want to edit(move). The default bit mask value is 1.

The reason why we set the category bit mask is to distinguish between different kinds of nodes, and specify those that you wish to select (to move around, etc).

    enum CategoryBitMask: Int {
        case categoryToSelect = 2        // 010
        case otherCategoryToSelect = 4   // 100
        // you can add more bit masks below . . .
    }

Then, I added a UILongPressGestureRecognizer in viewDidLoad().

        let longPressRecognizer = UILongPressGestureRecognizer(target: self, action: #selector(longPressed))
        self.sceneView.addGestureRecognizer(longPressRecognizer)

The following is the UILongPressGestureRecognizer I used to detect a long press, which initiates the dragging of the node.

First, obtain the touch location from the recognizerView

    @objc func longPressed(recognizer: UILongPressGestureRecognizer) {

       guard let recognizerView = recognizer.view as? ARSCNView else { return }
       let touch = recognizer.location(in: recognizerView)

The following code runs once when a long press is detected.

Here, we perform a hitTest to select the node that has been touched. Note that here, we specify a .categoryBitMask option to select only nodes of the following category: CategoryBitMask.categoryToSelect

       // Runs once when long press is detected.
       if recognizer.state == .began {
            // perform a hitTest
            let hitTestResult = self.sceneView.hitTest(touch, options: [SCNHitTestOption.categoryBitMask: CategoryBitMask.categoryToSelect])

            guard let hitNode = hitTestResult.first?.node else { return }

            // Set hitNode as selected
            self.selectedNode = hitNode

The following code will run periodically until the user releases the finger. Here we perform another hitTest to obtain the plane you want the node to move along.

        // Runs periodically after .began
        } else if recognizer.state == .changed {
            // make sure a node has been selected from .began
            guard let hitNode = self.selectedNode else { return }

            // perform a hitTest to obtain the plane 
            let hitTestPlane = self.sceneView.hitTest(touch, types: .existingPlane)
            guard let hitPlane = hitTestPlane.first else { return }
            hitNode.position = SCNVector3(hitPlane.worldTransform.columns.3.x,
                                           hitNode.position.y,
                                           hitPlane.worldTransform.columns.3.z)

Make sure you deselect the node when the finger is removed from the screen.

        // Runs when finger is removed from screen. Only once.
        } else if recognizer.state == .ended || recognizer.state == .cancelled || recognizer.state == .failed{

            guard let hitNode = self.selectedNode else { return }

            // Undo selection
            self.selectedNode = nil
        }
    }