Brick-Composer: Using MLLMs for Assembly with Diverse Bricks
Quick Answer
The Brick-Composer framework enhances MLLMs' assembly skills, improving brick selection accuracy by over 300% and raising assembly success rates from under 1% to 15%.
Quick Take
Utilizing BC-Bench for evaluation, the Qwen-3-8B model demonstrates the potential for effective brick assembly through targeted learning strategies.
Key Points
- Brick assembly is framed as a sequential decision-making problem with two main subtasks.
- Current MLLMs struggle with fine-grained brick selection and precise pose estimation.
- Brick-Composer integrates Human Design Sparks, World Feedback, and Synthetic Experience for training.
- After training, Qwen-3-8B can correctly compose 42% of steps for complete objects.
- BC-Bench is introduced as the first benchmark for evaluating MLLMs in brick assembly.
Paper Resources
Source Excerpt
arXiv:2606. 05445v1 Announce Type: new Abstract: We dream of AI agents that can read arbitrary designs and construct real-world objects from reusable building blocks. As a first step toward this vision, we study whether multimodal (MLLMs) possess the visual grounding and spatial reasoning capabilities required for brick assembly.
We formulate brick assembly as a sequential decision-making problem, where each step involves two subtasks: brick selection, identifying the target brick from candidate components, and brick pose estimation, predicting where and how the selected brick should be placed. …
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →RAIL Guard: Closing the Evaluation-to-Remediation Gap in Responsible AI for Agents
RAIL Guard introduces a closed-loop AI pipeline for large language models (LLMs) that evaluates outputs across eight dimensions and iteratively remediates failures, achieving 96.9% convergence compared to 49.1% for traditional block-and-retry methods. The system reduces unsafe agent executions by 33% without impacting task completion and is available as open-source SDKs.