A Hand-Drawn Language for Human-Robot Collaboration in Wood Stereotomy

This study introduces a novel, hand-drawn language designed to foster human-robot
collaboration in wood stereotomy, central to carpentry and joinery professions. Based on skilled carpenters’
line and symbol etchings on timber, this language signifies the location, geometry of woodworking joints,
and timber placement within a framework. A proof-of-concept prototype has been developed, integrating
object detectors, keypoint regression, and traditional computer vision techniques to interpret this language
and enable an extensive repertoire of actions. Empirical data attests to the language’s efficacy, with the
successful identification of a specific set of symbols on various wood species’ sawn surfaces, achieving a
mean average precision (mAP) exceeding 90%. Concurrently, the system can accurately pinpoint critical
positions that facilitate robotic comprehension of carpenter-indicated woodworking joint geometry. The
positioning error, approximately 3 pixels, meets industry standards.