Styjun

I'm trying to visualize PlaneAnchorNodes detected by ARKit via an iOS app in a browser using three.js, and I can't get the planes that I'm visualizing to match up with the pointcloud data from the ARWorldMap that I'm also exporting.

I'm exporting each ARPlaneAnchor's position, length, width, and rotation. Here's a .debugDescription of an example PlaneAnchorNode:

0, <ClientCore.PlaneAnchorNode: 0x10170d540 'HorizontalPlaneAnchor' pos(-0.034809 -0.391279 0.130804) rot(0.000000 -1.002311 0.000000 0.011389) scale(1.000000 1.000000 1.000000) | 5 children>

I'm exporting as a CSV, which looks like this:

id,type,orientation,pos_x,pos_y,pos_z,rot_w,rot_x,rot_y,rot_z,length,width
0,anchorPlane,VerticalPlaneAnchor,4.4324,0.3275,-4.2946,3.0899,0.0277,-0.6809,0.7319,0.2359,1.0074
1,anchorPlane,VerticalPlaneAnchor,4.8228,-0.8363,-5.9102,3.1184,0.0118,-0.7006,0.7134,0.2723,0.4500
2,anchorPlane,HorizontalPlaneAnchor,5.5023,-1.1353,-2.8441,0.5499,0.0002,1.0000,0.0000,0.5798,0.4390
3,anchorPlane,HorizontalPlaneAnchor,5.0866,0.2323,-4.4964,0.0583,-0.0014,1.0001,-0.0003,0.2657,0.7294

For my visualization with three.js, I add the pointcloud and anchor planes to the scene, and set up the anchors like this:

// anchor planes
for (i = 0; i < planes_dataset.length; i++) 

 var pos_x = planes_dataset[i].pos_x;
 var pos_y = planes_dataset[i].pos_y;
 var pos_z = planes_dataset[i].pos_z;
 var rot_w = planes_dataset[i].rot_w;
 var rot_x = planes_dataset[i].rot_x;
 var rot_y = planes_dataset[i].rot_y;
 var rot_z = planes_dataset[i].rot_z;
 var width = planes_dataset[i].width;
 var length = planes_dataset[i].length;

 let cube_height = 0.1;

 var anchorGeometry = new THREE.BoxGeometry( width, cube_height, length );
 var anchorMaterial = new THREE.MeshPhongMaterial( color: 0x7a7978, opacity: 0.5, transparent: true );
 var anchorCube = new THREE.Mesh( anchorGeometry, anchorMaterial );

 if (planes_dataset[i].orientation == "VerticalPlaneAnchor") 
 anchorCube.rotateX( - Math.PI / 2 ); // make vertical
 

 //const quaternion = new THREE.Quaternion(rot_x, rot_y, rot_z, rot_w);
 //anchorCube.applyQuaternion(quaternion); // Apply Quaternion
 //anchorCube.quaternion.normalize(); // Normalize Quaternion

 scene.add(anchorCube);
 anchorCube.position.set(pos_x, pos_y, pos_z);

My challenge is that I haven't been able to figure out how to use the rotation correctly, and the planes are not rotated or positioned correctly vs. the pointcloud. Screen shot attached, using my staircase to line up point cloud data and detected planes. Any ideas would be much appreciated!

Full code in the "pointcloud visualization" branch on Github here: https://github.com/zredlined/control-my-mekamon/tree/pointcloud_visualization/pointcloud_visualization