Image2Lego: Customized LEGO Set Generation from Images
LEGO® bricks are a popular form of entertainment that lets to construct anything one can imagine from simple building blocks.
However, in order to simplify the construction of more complex images, new brick models would be helpful. Therefore, a recent study on arXiv.org proposes a method for producing LEGO®-type brick 3D models directly from 2D images.
The model converts a 2D image to a latent representation and decodes it to a 3D voxel model. Then, a novel algorithm transforms the voxelized model into 3D bricks. The same approach can be extended to other creative applications. For example, current models are able to generate 2D mosaics from a facial picture.
A 3D face can be generated from a single 2D image using the proposed approach. Also, a tool that directly converts an imagined drawing into a LEGO® model is proposed. The study improves the accessibility of personalized LEGO creations.
Although LEGO sets have entertained generations of children and adults, the challenge of designing customized builds matching the complexity of real-world or imagined scenes remains too great for the average enthusiast. In order to make this feat possible, we implement a system that generates a LEGO brick model from 2D images. We design a novel solution to this problem that uses an octree-structured autoencoder trained on 3D voxelized models to obtain a feasible latent representation for model reconstruction, and a separate network trained to predict this latent representation from 2D images. LEGO models are obtained by algorithmic conversion of the 3D voxelized model to bricks. We demonstrate first-of-its-kind conversion of photographs to 3D LEGO models. An octree architecture enables the flexibility to produce multiple resolutions to best fit a user’s creative vision or design needs. In order to demonstrate the broad applicability of our system, we generate step-by-step building instructions and animations for LEGO models of objects and human faces. Finally, we test these automatically generated LEGO sets by constructing physical builds using real LEGO bricks.
Research paper: Lennon, K., “Image2Lego: Customized LEGO Set Generation from Images”, 2021. Link: https://arxiv.org/abs/2108.08477