Alexander Htet Kyaw
Selected Projects
16| New Frontiers
Lunar Habitat Module Multi-objective
Optimization
15| Wind Weaves
Performative Wind Screen and
CFD Simulation
Simulation
Optimization
Robotics
Computer Vision
Material
Fabrication
Simulation
Material
Fabrication
Simulation
Optimization
Extended Reality
8| AR GluLam
High Precision Augmented Reality for
Glulam Fabrication
14|Undulating Frame
Space Frame Structural Simulation,
Analysis and Modeling
9| Convergence
Wire Arc Additive Manufactured
3D Printed Sculpture
10| Bamboo Network
Active Bending in Physics-Based Mixed
Reality Environment
2| AI Assembly
Object Recognition and Computer Vision
for Mixed Reality Assembly
AI/Machine Learning
Extended Reality
Fabrication
Extended Reality
Extended Reality
AI/Machine Learning
Computer Vision
6| UnLog Tower
Gesture Recognition for Feedback-Based
Mixed Reality and Robotic Fabrication
AI/Machine Learning
Fabrication
Robotics
Extended Reality
Fabrication
Material
Simulation
Material
Simulation
Robotics
Fabrication
Fabrication
Material
3| Automated Lego Sorter
Computer Vision and Autonomous
Robotic System
Computer Vision
Robotics
Fabrication
Data Visualization
1| Speech to Reality
Text to Mesh to Voxel to Robotic Toolpath
to Automated Assembly
11| Covid Demographics
Web Application for Interactive Data
Visualization in d3.js
Data Visualization
Data Visualization
AI/Machine Learning
7| Daily Paper Plotter
Machine Building and Design for a
Toilet Paper Plotter
Robotics
12| The Maker-space
Net Zero Plus Community Fabrication
Center
13| Solar Kiosk
Digital Fabrication, Mass Customization
and Performance
Fabrication
Simulation
Optimization
Simulation
Optimization
Data Visualization
Robotics
5| Tangible AI Mind Map
AI Enhanced Mind Map with Large
Language Models and Gesture Recognition
Data Visualization
AI/Machine Learning
Cornell University - Class of 2023
Bachelor of Architecture in Science and Technology
Minor in Computer Information Science
Massachusetts Institute of Technology - Dual Degree Candidate Class of 2025
Master of Science in Electrical Engineering and Computer Science
Master of Science in Architectural Studies in Computation
Contact Info
Email: alexkyaw@mit.edu
Personal: 6072799832
Harvard University
Cross Registered Student
Graduate School of Design
Website
Personal: www.alexanderhtetkyaw.com
Linkedin: www.linkedin.com/in/alexkyaw
Research: https://scholar.google.com/citations?user=g9y-
wRAEAAAAJ&hl=en
4| Curator.AI
Vision Language Understanding for
Contextual Product Recommendation
Extended Reality
AI/Machine Learning
We present a system that transforms speech into physical objects by combining 3D generative
Artificial Intelligence with robotic assembly. The system leverages natural language input to
make design and manufacturing more accessible, enabling individuals without expertise in 3D
modeling or robotic programming to create physical objects. We propose utilizing discrete robot-
ic assembly of to address the challenges of using generative AI outputs in physical production,
such as design variability, fabrication speed, and material waste. The system interprets speech
to generate 3D objects, discretizes them into components, computes an assembly sequence,
and generates toolpath. Various objects, ranging from chairs to shelves, can be are prompted via
speech and realized within 5 minutes using a 6-axis robotic arm.
1 |Speech To Reality
Speaking Objects into existence via 3D Generative AI and Robotic Assembly
Massachusetts Institute of Technology | Fall 2023 - (On Going Research)
Alexander Htet Kyaw, Miana Smith, Se Hwan Jeon and Prof. Neil Gershenfeld
Submited to IEEE International Conference for Robotic Automation (ICRA)
Publication Preprint: https://doi.org/10.48550/arXiv.2409.18390
Speech to Text to 3D Mesh
Assembly Sequence to Robotic Tool Path
AI/Machine Learning
Robotics
Discretization of the AI Generated
Mesh into Assembly Components
Subdivision Grid Based on Component
Size and AI Generated 3D Mesh
Cantilevering Components (Red)
Supported Components (Green)
Assembly Sequence Adjusted
for Cantilevering Components
3D Mesh to Components to Assembly Sequence
https://youtu.be/tEnVV5G-HSg
User Input "Assemble me a table with one leg"
<Processing Speech>
<Running Text-to-Mesh AI Model>
<Voxelizing Geometry>
<Optimizing Assembly Sequence>
<Generating Robotic Tool Path>
<Starting Robotic Assembly>
Assembly time: 3 minutes 36 seconds
Discretization of the AI Generated
Mesh into Assembly Components
Subdivision Grid Based on Component
Size and AI Generated 3D Mesh
Cantilevering Components (Red)
Supported Components (Green)
Assembly Sequence Adjusted
for Cantilevering Components
"A shelf with two tiers"
" I want a small chair "
"I want a tall dog"
User Speech Input: “A stool with four leg”
User Speech Input: “A shelf with two tiers”
Discretization of the AI Generated
Mesh into Assembly Components
Subdivision Grid Based on Component
Size and AI Generated 3D Mesh
Cantilevering Components (Red)
Supported Components (Green)
Assembly Sequence Adjusted
for Cantilevering Components
""A stool with four legs""
ASSEMBLY TIMELINE - User Prompt "I want a stool with three legs"
40 Seconds
1 Minute 10 Seconds
2 Minutes 30 Seconds
ROBOT END EFFECTOR
Printed Circuit Board for Servo
3D Printed Connection with Mounting Plate
End Effector connected to Mounting Plate
PRINTED CIRCUIT BOARD (PCB) for Conveyor Belt Stepper Motor
Printed Circuit Board Design in KiCAD
Milling the PCB design
Soldered Components on the Circuit Board
CONVEYOR BELT
Component Feed in Conveyor Belt
Motor, Belt and Gear
Stopper at the end to prevent slipping
Work-In-Progress Robot-Material System in Collaboration with Miana Smith (Mobile Robots)
Automated Part Selection for Using Vision-Language Models (VLM) and 3D Generative AI for Multi Component Robotic Assembly
User Input
Vision Language Model (VLM) for Automated Part Selection
3D Generative AI
Robotic Assembly
Text to Mesh
(Instant Mesh Model)
Discretized Mesh
Based on Primary Component
Baseline 1
Random Selection
Assign panels by randomly selecting
surfaces across the 3D object
Baseline 2
Predefined Algorithm
Assign panels on all horizontal, top-facing
surfaces aligned in the Z-direction
Our Method
Automated Part Selection via Vision
Language Model (VLM)
Assign panels based on context and
functionality of the object
To evaluate the VLM approach, we compared it against two baseline algorithms. We asked participants to choose options where panels are appropriately
assigned based on functionality of the object. Participants were allowed to pick one or more options for each object. Ten participants evaluated five objects.
"Build a shelf with two tiers"
"A simple chair"
"A Rabbit SitÝng"
"Make me a lamp"
"A Wide Bowl"
"I want a round coffee table"
2 | AI ASSEMBLY
Object Recognition, Computer Vision & Digital Twin for Mixed Reality Assembly
Cornell University | Fall 2022
Expanded Design Thesis Part 1 |Advisors : Jenny Sabin, Sasa Zivkovic
In collaboration with Haotian Ma
Publication Preprint: https://doi.org/10.7298/ha3k-4e73
Recent development in digital modeling tools has enabled architects to design with nonstandard
and custom geometries. While complex components can be digitally fabricated, human
assembly is still required. This thesis explores the potential for human-machine collaboration to
facilitate the co-assembly process. The proposed system utilizes object recognition, computer
vision, digital twin technology, and mixed reality to recognize self-similar components, display
3D assembly instructions, and adapt to real-time changes in the construction environment. All
aspects of the project is done collaboratively with my partner.
Input from the Physical World
Computer Processing
Augmented Reality Display
Step 1: Object Recognition to Identify Components
Step 2: Pick Identified Component
Step 3: Place the Component based on instruction
FEATURE 1: COMPONENT RECOGNITION
AI POWERED AUGMENTED REALITY VIA OBJECT RECOGNITION
Extended Reality
AI/Machine Learning
Computer Vision
https://youtu.be/y3HkzA4hQ4w
16
15
Instruction Number: 146
Instruction Number: 148
Instruction Number: 147
FEATURE 2: END-TO-END ASSEMBLY INSTRUCTIONS
COMPONENTS AND AGGREGATION
Mesh Volume Input
Voxelize/Pixelate
Assign Lego Blocks
CONVENTIONAL
AI ASSEMBLY
2D Drawing and Documentation
3D Blueprints via Augmented Reality
Assembly & Instruction is Separate
Instruction is Seamlessly Superimposed
Search and Sort Before Assembly
Identification of Desired Component
CONVENTIONAL
AI ASSEMBLY
Search and Sort Before Assembly
No Need for Search and Sort
3D Model as Reference for Assembly
Instruction is Seamlessly Superimposed
Text Labels to Differentiate Parts
Labels are displayed via AR
LEGO BLOCKS
CASE STUDY: 3D PRINTED NODES
17
18
WORKFLOW AND DATA PIPELINE
Sorted Assembly 25 Mins 54 Secs
AI Assembly 19 Mins 39 Secs
USER TESTING - ASSEMBLY EXPERIENCE AND TIME (Participant: 2nd year student without prior experience with AR/MR)
Unsorted Assembly 32 Mins 11 Secs
Pascal-Voc: Pattern Analysis, Statistical-Modeling, and Computational Learning - Visual Object Classes
Open CV: Open-source software library of programming functions for real-time computer vision by Intel
TensorFlow: Open-source software library for machine learning and artificial intelligence by Google
REAL TIME CUSTOM OBJECT RECOGNITION AND DIGITAL TWIN
Custom Object Recognition
Digital Twin Location of all Recognized Objects
3D Printed Nodes
with labels in Mixed Reality
AI Powered
Digital Twin
Custom Object
Recognition
19
20