Augmented Reality Measure App with WebXR and Three.js

The first choice for Augmented Reality (AR) applications for mobile devices were hitherto native APIs like Google’s ARCore or Apple’s ARKit. This changed with the introduction of WebXR - an AR and Virtual Reality (VR) API for the web. Even though WebXR is not as powerful as native APIs (yet), it has all the advantages web apps have compared to native apps:

• Availability: much more developers are familiar with the development of web apps than of native apps
• Interoperability: a web application can run on a multitude of devices with various operating systems
• Reachability: Users only need a browser and a URL and do not have to install an app

In this article I describe how I build a Measure app (like the ones available from Apple and Google) with WebXR and Three.js. The aim is to be able to measure the distance between two points by overlaying the image of the camera in real time with a reticle. The full code and a demo are accessible here.

The reticle is displayed, whenever the ray from the viewer’s point of view intersects with a physical object in the real world. In the image below, you see how the reticle is rendered on top of a small table.

The following code snippets shows, how the WebXR Hit Test module is used to update the reticle in the render function. To request a hitTestResult for a frame, we need a hitTestSource which is requested when an XRSession is started.

let referenceSpace = renderer.xr.getReferenceSpace();
let session = renderer.xr.getSession();
session.requestReferenceSpace(‘viewer’)
  .then(function (referenceSpace) {
    session.requestHitTestSource({space: referenceSpace })
      .then(function (source) {
        hitTestSource = source;
  });
});
session.addEventListener(‘end’, function () {
  hitTestSource = null;
});

After the hitTestSource is requested, we can request the hitTestResult and update the reticle’s pose.

if (hitTestSource) {
  let hitTestResults = frame.getHitTestResults(hitTestSource);
  if (hitTestResults.length) {
    let hit = hitTestResults[0];
    reticle.visible = true;
    let pose = hit.getPose(referenceSpace).transform.matrix;
    reticle.matrix.fromArray(pose);
  } else {
    reticle.visible = false;
  }
}

To provide accurate hit test result, a local map of the surrounding needs to be generated. When the XRSession is started, the user need to move the camera around, to allow for the generation of such a map.

The reticle is displayed, whenever we detect an intersection with the physical world. We now want to be able to tap on the display to start and stop measuring. We therefore add a controller to our scene, which listens to the select event.

let controller = renderer.xr.getController(0);
controller.addEventListener(‘select’, onSelect);
scene.add(controller);

Whenever the event is fired, we store the position of the reticle in a list. If there are two measurement points in the list, we compute the distance between them and add a label to the screen with the distance.

function onSelect() {
  if (reticle.visible) {
    measurements.push(matrixToVector(reticle.matrix));
    if (measurements.length == 2) {
      let distance = Math.round(getDistance(measurements) * 100);
      let text = initLabel(distance)
      labelContainer.appendChild(text);
      labels.push({div: text, point: getCenterPoint(measurements)});
      measurements = [];
      currentLine = null;
    } else {
      currentLine = initLine(measurements[0]);
      scene.add(currentLine);
    }
  }
}

When the first measurement point is added, we add a line to the scene, which starts at the measurement point and ends at the reticle. The second point of the line is updated every frame.

In the following image you can see the result of the measurement of a ruler with a length of 30 cm.

To be able to draw the label at the center of the line, the domElement of the label and its center point are stored in a list. The center point is projected onto the screen and the position of the domElement is updated every frame.

labels.map((label) => {
  let xrCamera = renderer.xr.getCamera(camera)
  let pos = toScreenPosition(label.point, xrCamera);
  label.div.style.transform = “translate(-50%, -50%)
      translate(“+pos.x+“px,”+pos.y+“px)”;
})

We now have a fully functional Measure web app, accessible just with a browser and a URL. To be honest, I was quite surprised, how accurate the measurements are. I expected an accuracy of ± 5 cm, but most of the times, the accuracy was more like ± 1 cm. This depends of course on the light conditions, the texture of the surfaces, the quality of the camera, you name it.