Foresight API Challenge

The 2026 challenge
has concluded.

Student and researcher teams from around the world built packing agents for one of Physical AI's hardest spatial reasoning problems. Explore how the competition worked and view the final public rankings.

Challenge Concluded

View Final Public Leaderboard →Read the Docs ↓

Challenge Timeline

Apr 28, 5pm PTPublic leaderboard closed.
Top five were finalized.

Apr 29, 12pm PTPrivate evaluation began.

May 1, 12pm PTPrivate evaluation ended.

CorrectionFinal eval ends May 1, 12pm PT (full 48 hours).

Prize Structure

Grand Prize

$50K

#1 on private evaluation with ≥70% IQM density

Eligibility

.edu+

Researchers and students with a valid .edu or equivalent university email address

Duration

→ May 1

Public leaderboard closes April 28 at 5 PM PDT

Private Eval

48 hrs

April 29 12pm – May 1 12pm PT

How It Worked

Qualify Through the Game

Participants played the truck loading game and scored ≥50% packing density to receive an API key.

Play the Game →

Build a Packing Agent

Teams used the REST API to send placements and receive game state, building their own understanding of the physics without a provided simulator.

Compete and Advance

Agents played random sequences. The top five on the public leaderboard advanced to a private evaluation of 50 held-out sequences.

Scoring

Your score is packing density — the fraction of usable truck volume you fill with boxes.

// Density formula
density = sum(box_volumes) / (max_x_reached × truck_width × truck_height)

Numerator

Total volume of all placed boxes

Denominator

Deepest box face × truck cross-section

Implication

Gaps behind your pack front are penalized

How the Competition Worked

The competition had two phases. The public leaderboard determined who advanced, and the private evaluation determined the final result.

Phase 1~April 6 – 28

Public Leaderboard

Participants could play up to 50 games per day in compete mode. Each game drew from the same box pool with a randomly shuffled sequence. Rankings used the interquartile mean (IQM) of each agent's most recent 50 compete games.

Min Games

To appear on the leaderboard

Rate Limit

50 / day

Compete mode, per API key

Advance

Top 5

Advance to private evaluation

Phase 2April 29 12:00 PM – May 1 12:00 PM PT

Private Evaluation

The top five from the public leaderboard played 50 held-out sequences that no one had seen before. The sequences were pre-generated and frozen, and every finalist played the same sequences in the same order with no retries.

Final scores used the interquartile mean (IQM) across all 50 private sequences. A score of ≥70% IQM density was required to qualify for the prize. This round focused on pure packing without robot reachability constraints, visibility concerns, or observation noise.

Sequences

Held-out, never seen before

Retries

None

Each sequence played exactly once

Window

48 hours

Complete all 50 within the eval window

Validation

Before any prize is awarded, top submissions are validated:

→

Visual Review: Top submissions produce a full replay that is reviewed by a human. Your agent's placements are watched step-by-step.

→

Autonomy: Submissions must be generated by automated agents, not human teleoperation. Anomalous patterns will be flagged for review.

→

Final Check: Top submission may be asked to run against additional sequences while observed by Dexterity. This will not affect the ranking.

Rules

Prize$50K to #1 on private eval (≥70% IQM required)

Eligibility.edu or equivalent university email

TeamsAllowed — prize splits among members

Rate Limit50 compete games/day per API key

RankingIQM of packing density (most recent 50 games)

Concurrent GamesMax 2 in-progress per API key

SimulatorNot provided — build your own

Agents OnlyAutomated agents required, no human teleoperation

InterviewWinner invited to interview at Dexterity

Questions? Reach us at foresight-api-challenge@dexterity.ai

Leaderboard

View the final public rankings from the concluded 2026 challenge.

View Final Public Leaderboard →

API Reference & Guide

Everything you need to build and test your truck-packing agent.

Get Started ↓

Overview

The Foresight Packing Challenge API is a REST service where your agent packs boxes into a truck, one at a time. You receive a box, choose where to place it (position + orientation), and get back the full post-placement state including physics-settled positions.

Base URL

https://dexterity.ai/challenge/api

Auth

API Key

Passed in the request body, not headers

Format

JSON

All requests and responses are JSON

Modes

Dev Mode

dev

No physics simulation — boxes land exactly where you place them. No rate limiting, not scored. Use this to build and test your agent.

Compete Mode

compete

Full physics simulation. Boxes settle realistically. Rate limited, scored, and ranked on the leaderboard. This is where it counts.

Truck Segment Dimensions

Depth (X-axis)

2.0 m

Width (Y-axis)

2.6 m

Height (Z-axis)

2.75 m

Quickstart

Get from zero to your first placement in three steps.

Start a game

POST /challenge/api/start

# curl
curl -X POST https://dexterity.ai/challenge/api/start \
  -H "Content-Type: application/json" \
  -d '{
    "api_key": "YOUR_API_KEY",
    "mode": "dev"
  }'

// Response
{
  "game_id": "g_a1b2c3d4e5f6",
  "truck": { "depth": 2.0, "width": 2.6, "height": 2.75 },
  "current_box": {
    "id": "108148",
    "dimensions": [0.406, 0.310, 0.211],
    "weight": 3.13
  },
  "boxes_remaining": 120,
  "mode": "dev"
}

Place the box

Send the box ID, a 3D position [x, y, z], and an orientation quaternion [w, x, y, z].

POST /challenge/api/place

curl -X POST https://dexterity.ai/challenge/api/place \
  -H "Content-Type: application/json" \
  -d '{
    "game_id": "g_a1b2c3d4e5f6",
    "box_id": "108148",
    "position": [0.203, 0.155, 0.106],
    "orientation_wxyz": [1.0, 0.0, 0.0, 0.0]
  }'

// Response
{
  "status": "ok",
  "placed_boxes": [
    {
      "id": "108148",
      "position": [0.203, 0.155, 0.106],
      "orientation_wxyz": [1.0, 0.0, 0.0, 0.0],
      "dimensions": [0.406, 0.310, 0.211]
    }
  ],
  "current_box": {
    "id": "209311",
    "dimensions": [0.350, 0.280, 0.190],
    "weight": 2.45
  },
  "boxes_remaining": 119,
  "density": 0.0003,
  "game_status": "in_progress"
}

Repeat until done

Keep placing boxes from current_box in each response. The game ends when game_status becomes "completed" — either too many boxes collapsed or you stopped early. Your score is the final density.

Tip: In compete mode, physics simulation means boxes settle and can shift. The placed_boxes array returns post-settle positions — use these to plan your next placement.

Game Loop

Here's a minimal Python agent that plays a complete game with naive left-to-right packing:

import requests

BASE = "https://dexterity.ai/challenge/api"

# 1. Start a game
game = requests.post(f"{BASE}/start", json={
    "api_key": "YOUR_API_KEY",
    "mode": "dev"
}).json()

game_id = game["game_id"]
box = game["current_box"]
y_cursor = 0.0

# 2. Place boxes one at a time
while box:
    dims = box["dimensions"]  # [length, width, height]

    resp = requests.post(f"{BASE}/place", json={
        "game_id": game_id,
        "box_id": box["id"],
        "position": [dims[0]/2, y_cursor + dims[1]/2, dims[2]/2],
        "orientation_wxyz": [1, 0, 0, 0]
    }).json()

    y_cursor += dims[1]
    box = resp.get("current_box")

    if resp["game_status"] == "completed":
        print(f"Game over: {resp['termination_reason']}, density={resp['density']:.4f}")
        break

Termination Conditions

Your game ends when one of these happens:

Reason	Description
unstable	3+ boxes displaced >10cm after a placement
player_stop	You called `/stop` to end the game early

Endpoints

POST/challenge/api/start

Start a new game session. Returns the truck config and first box.

Request Body

Field	Type	Description
api_key*	string	Your API key
mode	string	Defaults to `"compete"`. Use `"dev"` for testing

Response

Field	Type	Description
game_id	string	Unique game session ID
truck	object	Dimensions in meters: `{ depth, width, height }`
current_box	object	First box: `{ id, dimensions, weight }`
boxes_remaining	int	Number of boxes left in the queue
mode	string	Confirmed game mode

Important: You can have at most 2 concurrent games in progress per API key. If you already have 2 active games, /start will fail — call /stop on an existing game first. Use /my-games?status=in_progress to see what's currently active.

POST/challenge/api/place

Place the current box in the truck. Returns the updated state with physics-settled positions.

Request Body

Field	Type	Description
game_id*	string	From the `/start` response
box_id*	string	Must match `current_box.id`
position*	[x, y, z]	Center position in meters
orientation_wxyz*	[w, x, y, z]	Rotation quaternion (auto-normalized)

Response

Field	Type	Description
status	string	`"ok"` or `"terminated"`
placed_boxes	array	All placed boxes with post-settle positions and orientations
current_box	object \| null	Next box to place, or `null` if game over
boxes_remaining	int	Boxes left in the queue
density	float	Current packing density (your score)
game_status	string	`"in_progress"` or `"completed"`
termination_reason	string \| null	Why the game ended (if it did)

Important: You must place the box whose id matches current_box.id. You cannot skip or reorder boxes.

GET/challenge/api/status/{game_id}

Get the current state of a game. Useful for debugging or resuming after a crash.

Response

Field	Type	Description
game_id	string	The game session ID
game_status	string	`"in_progress"` or `"completed"`
mode	string	`"dev"` or `"compete"`
boxes_placed	int	Number of boxes placed so far
boxes_remaining	int	Boxes left in the queue
density	float	Current packing density
placed_boxes	array	All placed boxes with current positions
current_box	object \| null	Next box to place

POST/challenge/api/stop

End a game early. Your score is the density at time of stopping.

Request Body

Field	Type	Description
game_id*	string	The game session to stop
api_key*	string	Your API key

GET/challenge/api/my-games

Retrieve all your games and a summary of your stats. Requires your API key.

Query Parameters

Field	Type	Description
api_key*	string	Your API key
mode	string	Filter by `"dev"` or `"compete"`
status	string	Filter by `"in_progress"` or `"completed"`

Response

Field	Type	Description
api_key	string	Your API key
display_name	string \| null	Your display name
summary	object	Aggregate stats: total/completed/in-progress game counts, avg & best compete density, games today, daily limit
games	array	All matching games (newest first), each with game_id, mode, status, density, boxes_placed, total_boxes, termination_reason, timestamps

GET/challenge/api/health

Health check. Returns {"status": "ok"}.

Coordinate System

The truck uses a right-handed coordinate system with the origin at the front-bottom-left corner.

        Z (up, height: 2.75m)
        ^
        |
        |
        +---------> Y (width: 2.6m)
       /
      /
     v  X (depth: 2.0m, loading direction)

Axis	Direction	Range
X	Depth (loading direction, front → back)	0 → 2.0 m
Y	Width (left → right)	0 → 2.6 m
Z	Height (floor → ceiling)	0 → 2.75 m

Position

The position field is the center of the box. To place a box flush on the floor in the front-left corner:

// Box dimensions: [length, width, height] = [0.4, 0.3, 0.2]
"position": [0.2, 0.15, 0.1]  // half-extents from origin

Orientation

Quaternion in [w, x, y, z] format. Common rotations:

Rotation	Quaternion	Effect
Identity	`[1, 0, 0, 0]`	No rotation
90° around Z	`[0.707, 0, 0, 0.707]`	Swap length ↔ width
90° around Y	`[0.707, 0, 0.707, 0]`	Swap length ↔ height
90° around X	`[0.707, 0.707, 0, 0]`	Swap width ↔ height

Tip: Quaternions are auto-normalized by the API. You don't need to worry about exact unit length, but near-zero magnitude will be rejected.

Error Handling

All errors return a JSON body with error (code) and message (human-readable) fields.

Error Code	HTTP	Description
invalid_api_key	401	API key is missing, invalid, or revoked
invalid_game_id	404	Game not found or already completed
invalid_box_id	400	box_id doesn't match the current box
rate_limited	429	Daily compete-mode game limit reached
(validation)	422	Invalid position, degenerate quaternion, missing fields

// Example error response
{
  "error": "invalid_box_id",
  "message": "Expected box_id '209311', got '108148'"
}

Tips for Building Your Agent

Start with dev mode

Use "mode": "dev" while building. Boxes land exactly where you place them — no physics surprises. Switch to "compete" when you're ready to test against real physics and get on the leaderboard.

Use placed_boxes for planning

Every /place response includes the full placed_boxes array with post-physics positions. In compete mode, boxes may shift from where you placed them. Use the settled positions to compute valid placement locations for the next box.

Think about stability

Placing boxes without support underneath will cause them to fall and potentially topple the stack. If 3 or more boxes displace by more than 10cm, the game terminates with unstable.

Optimize rotations

Each box can be placed in 6 orientations (3 axes × 2 flips that matter). Trying all valid rotations for each box is a key optimization. A box that doesn't fit one way might fit perfectly rotated.

Rate limits

Dev Mode

Unlimited

No daily game limit

Compete Mode

50 / day

Per API key

Concurrent

2 max

In-progress games per key. Call /stop before starting more.

Leaderboard

25 games min

IQM of most recent 50 games

Interactive docs

Swagger UI is available at /docs for interactive API exploration.

Happy packing. We'll see you on the leaderboard.

The 2026 challengehas concluded.

Prize Structure

How It Worked

Qualify Through the Game

Build a Packing Agent

Compete and Advance

Scoring

How the Competition Worked

Public Leaderboard

Private Evaluation

Validation

Rules

Leaderboard

API Reference & Guide

Overview

Modes

Truck Segment Dimensions

Quickstart

Start a game

Place the box

Repeat until done

Game Loop

Termination Conditions

Endpoints

Request Body

Response

Request Body

Response

Response

Request Body

Query Parameters

Response

Coordinate System

Position

Orientation

Error Handling

Tips for Building Your Agent

Start with dev mode

Use placed_boxes for planning

Think about stability

Optimize rotations

Rate limits

Interactive docs

The 2026 challenge
has concluded.