@threlte/extras

interactivity

To add click, pointer and wheel events to your Threlte app use the interactivity plugin.

<script>
  import { interactivity } from '@threlte/extras'
  interactivity()
</script>

<script lang="ts">
  import { Canvas } from '@threlte/core'
  import Scene from './Scene.svelte'
</script>

<div class="relative h-full w-full">
  <Canvas>
    <Scene />
  </Canvas>
</div>

<script lang="ts">
  import type { Vector2Tuple } from 'three'
  import { Grid, interactivity, OrbitControls } from '@threlte/extras'
  import { Spring } from 'svelte/motion'
  import { T } from '@threlte/core'
  import { useCursor } from '@threlte/extras'

  interactivity()

  const boxPosition = new Spring<Vector2Tuple>([0, 0])
  const random = () => 10 * Math.random() - 5
  const scale = new Spring(1)
  const boxSize = 1
  const positionY = $derived(0.5 * boxSize * scale.current)

  const { onPointerEnter, onPointerLeave } = useCursor()

  const notHoveringColor = '#ffffff'
  const hoveringColor = '#fe3d00'
  let color = $state(notHoveringColor)
</script>

<T.OrthographicCamera
  zoom={80}
  position={10}
  makeDefault
>
  <OrbitControls />
</T.OrthographicCamera>

<T.AmbientLight intensity={0.4} />
<T.DirectionalLight position={[1, 2, 5]} />

<T.Mesh
  onclick={() => {
    boxPosition.target = [random(), random()]
  }}
  onpointerenter={() => {
    onPointerEnter()
    scale.target = 2
    color = hoveringColor
  }}
  onpointerleave={() => {
    onPointerLeave()
    scale.target = 1
    color = notHoveringColor
  }}
  scale={scale.current}
  position.x={boxPosition.current[0]}
  position.y={positionY}
  position.z={boxPosition.current[1]}
>
  <T.BoxGeometry args={[boxSize, boxSize, boxSize]} />
  <T.MeshStandardMaterial {color} />
</T.Mesh>

<Grid
  position.y={-1 * 0.5}
  cellColor="#ffffff"
  sectionColor="#ffffff"
  sectionThickness={0}
  fadeDistance={25}
  cellSize={2}
/>

All child components can now make use of the new events.

<script>
  import { interactivity } from '@threlte/extras'
  interactivity()
</script>

<T.Mesh
  onclick={() => {
    console.log('clicked')
  }}
>
  <T.BoxGeometry />
  <T.MeshStandardMaterial color="red" />
</T.Mesh>

Available Events

The following interaction events are available:

<T.Mesh
  onclick={(e) => console.log('click')}
  oncontextmenu={(e) => console.log('context menu')}
  ondblclick={(e) => console.log('double click')}
  onwheel={(e) => console.log('wheel')}
  onpointerup={(e) => console.log('up')}
  onpointerdown={(e) => console.log('down')}
  onpointerover={(e) => console.log('over')}
  onpointerout={(e) => console.log('out')}
  onpointerenter={(e) => console.log('enter')}
  onpointerleave={(e) => console.log('leave')}
  onpointermove={(e) => console.log('move')}
  onpointermissed={() => console.log('missed')}
/>

Event Data

All interaction events contain the following data:

type Event = THREE.Intersection & {
  intersections: THREE.Intersection[] // The first intersection of each intersected object
  object: THREE.Object3D // The object that was actually hit
  eventObject: THREE.Object3D // The object that registered the event
  camera: THREE.Camera // The camera used for raycasting
  delta: THREE.Vector2 //  Distance between mouse down and mouse up event in pixels
  nativeEvent: MouseEvent | PointerEvent | WheelEvent // The native browser event
  pointer: Vector2 // The pointer position in normalized device coordinates
  ray: THREE.Ray // The ray used for raycasting
  stopPropagation: () => void // Function to stop propagation of the event
  stopped: boolean // Whether the event propagation has been stopped
}

Event Propagation

Propagation works a bit differently to the DOM because objects can occlude each other in 3D. The intersections array in the event data includes all objects intersecting the ray, not just the nearest. Only the first intersection with each object is included. The event is first delivered to the object nearest the camera, and then bubbles up through its ancestors like in the DOM. After that, it is delivered to the next nearest object, and then its ancestors, and so on. This means objects are transparent to pointer events by default, even if the object handles the event.

event.stopPropagation() doesn’t just stop this event from bubbling up, it also stops it from being delivered to farther objects (objects behind this one). All other objects, nearer or farther, no longer count as being hit while the pointer is over this object. If they were previously delivered pointerover events, they will immediately be delivered pointerout events. If you want an object to block pointer events from objects behind it, it needs to have an event handler as follows:

<T.Mesh onclick={(e) => e.stopPropagation()} />

even if you don’t want this object to respond to the pointer event. If you do want to handle the event as well as using stopPropagation(), remember that the pointerout events will happen during the stopPropagation() call. You probably want your other event handling to happen after this.

Interactivity Event Target

If no event target is specified, all event handlers listen to events on the domElement of the renderer (which is the canvas element by default). You can specify a different target by passing a target prop to the interactivity plugin.

<script>
  import { interactivity } from '@threlte/extras'

  interactivity({
    target: document
  })
</script>

It’s also possible to change the target at runtime by updating the store target returned from the interactivity plugin.

<script>
  import { interactivity } from '@threlte/extras'

  const { target } = interactivity()

  $effect(() => {
    target.set(document)
  })
</script>

Interactivity Event Compute

In the event that your event target is not the same size as the canvas, you can pass a compute function to the interactivity plugin. This function receives the DOM event and the interactivity state and should set the pointer property of the state to the pointer position in normalized device coordinates as well as set the raycaster up for raycasting.

<script>
  import { interactivity } from '@threlte/extras'
  import { useThrelte } from '@threlte/core'

  const { camera } = useThrelte()

  interactivity({
    compute: (event, state) => {
      // Update the pointer
      state.pointer.update((p) => {
        p.x = (event.clientX / window.innerWidth) * 2 - 1
        p.y = -(event.clientY / window.innerHeight) * 2 + 1
      })

      // Update the raycaster
      state.raycaster.setFromCamera(state.pointer.current, $camera)
    }
  })
</script>

Interactivity Event Filtering

You can filter and sort events by passing a filter to the interactivity plugin. The function receives all hits and the interactivity state and should return the hits that should be delivered to the event handlers in the order they should be delivered.

<script>
  import { interactivity } from '@threlte/extras'

  interactivity({
    filter: (hits, state) => {
      // Only return the first hit
      return hits.slice(0, 1)
    }
  })
</script>

Interactivity State

To access the interactivity state, you can use the useInteractivity hook in any child component of the component that implements the interactivity plugin as follows:

<script>
  import { useInteractivity } from '@threlte/extras'

  const { pointer, pointerOverTarget } = useInteractivity()

  $inspect($pointer, $pointerOverTarget)
</script>

where this is the type of the interactivity state:

export type State = {
  enabled: CurrentWritable<boolean>
  target: CurrentWritable<HTMLElement | undefined>
  pointer: CurrentWritable<Vector2>
  pointerOverTarget: CurrentWritable<boolean>
  lastEvent: MouseEvent | WheelEvent | PointerEvent | undefined
  raycaster: Raycaster
  initialClick: [x: number, y: number]
  initialHits: THREE.Object3D[]
  hovered: Map<string, IntersectionEvent<MouseEvent | WheelEvent | PointerEvent>>
  interactiveObjects: THREE.Object3D[]
  compute: ComputeFunction
  filter?: FilterFunction
}

TypeScript

Prop Types

By default, the interactivity plugin does not add any prop types to the <T> component. You can however extend the types of the <T> component by defining the Threlte.UserProps type in your ambient type definitions. In a typical SvelteKit application, you can find these in src/app.d.ts. The interactivity plugin exports the InteractivityProps type which you can use as shown below:

import type { InteractivityProps } from '@threlte/extras'

declare global {
  namespace Threlte {
    interface UserProps extends InteractivityProps {}
  }
}

export {}

Now all event handlers on <T> components will be type safe.

<script>
  import { interactivity } from '@threlte/extras'
  interactivity()
</script>

<T.Mesh
  onclick={(e) => {
    // e: IntersectionEvent<MouseEvent>
  }}
/>