Plan a route. Learn a way through.
3D Pathfinding
A 3D pathfinding and learning environment. Compare deterministic graph search with an agent learning from rewards, repeated journeys, and negative sink states.
- JavaScript
- React
- Three.js
- React Three Fiber
- Tween.js
Public source
Two approaches, one world
Dijkstra and A* search the graph using costs; breadth-first and depth-first search explore it in different orders. The Q-Learning mode instead learns from repeated interactions with rewards. All five run in the same editable 3D world, but planning a route and learning how to navigate are different processes.
From world state to a visible result
World and Settings configure the grid. Weighted and unweighted search modules return visited nodes and predecessor chains. Grid also holds the learning state: cell values, visits, rewards, and episode snapshots. Three.js vertex colours and a shared Tween.Group turn those results into visible movement.
WORLD / CELLS + WALLS + REWARDS
│
┌─────┴───────────┐
▼ ▼
GRAPH SEARCH LEARNING LOOP
distances choose → move → reward
predecessors update scalar cell value
│ │
└─────┬───────────┘
▼
THREE.JS / TWEEN REPLAYOne world. Two ways to navigate.
- 01 / interface
World + Settings
Algorithm, endpoints, epochs
Learning rate + curiosity
- configure → 2. Grid / terrainRef
- 02 / shared model
Grid / terrainRef
30 × 30 cells · walls / rewards
Distances, predecessors, visits
- graph nodes → 3. Graph search modules
- states + rewards → 4. qLearning()
- 03 / solver
Graph search modules
Weighted: Dijkstra / A*
Unweighted: BFS / DFS
- visited nodes + predecessors → 5. Records + route
- 04 / learning loop
qLearning()
chooseAction → next cell
Reward → scalar value update
- repeat until terminal / 1,000 steps → 4. qLearning()
- one value snapshot per episode → 5. Records + route
- 05 / stored results
Records + route
Episode value snapshots
Predecessors / policy trace
- animate results → 6. Three.js + Tween.Group
- 06 / renderer
Three.js + Tween.Group
Vertex colours + wall meshes
useFrame advances tweens
- 1World + Settings Grid / terrainRefconfigure
- 2Grid / terrainRef Graph search modulesgraph nodes
- 3Grid / terrainRef qLearning()states + rewards
- 4qLearning() qLearning()repeat until terminal / 1,000 steps
- 5Graph search modules Records + routevisited nodes + predecessors
- 6qLearning() Records + routeone value snapshot per episode
- 7Records + route Three.js + Tween.Groupanimate results
Read from the implementation
World.jsSettings.jsGrid.jsalgorithms/weightedSearchAlgorithm.jsalgorithms/unweightedSearchAlgorithm.jsalgorithms/helpers.js
Seven obstacles, a thousand episodes
The new film reruns the actual app with seven buildings clustered between a start and goal. A sink ends an episode with −100; reaching the goal gives +100. Selected episodes 1, 503, and 998 replay in order: 137 moves ending in a sink, then 95 and 34 moves reaching the goal. Early episodes start at random cells; the final quarter starts at the configured green cell. These are different journeys, not a controlled shortest-path comparison.
What this learning rule actually learns
The interface calls the mode Q-Learning, but the implementation stores a scalar value per cell rather than a state–action Q-table. It updates that value using the sampled next cell and its reward. Exploration remains active in later episodes. The final hundred episodes in this seeded run all reached the goal; the selected route shows useful avoidance, not proof of an optimal or fully stabilized policy.
Making the recorded history readable
The original playback schedules all value snapshots at the same delay, hiding the progression. An isolated capture copy records the chosen states and replays selected episodes at readable speeds in the original 3D renderer. The readout reports actual episode and move counts; the heatmap shows values before each episode. The reward, action selection, learning update, and termination rules are unchanged.
The engineering boundary
Dijkstra’s film remains at 2.5× speed. Its weighted search scans an array for the next closest node; the source suggests a priority queue as a future improvement. The learning experiment is likewise presented on its own terms: a way to observe changing behaviour, rather than a production routing engine or textbook Q-learning implementation.
