Alexander's Portfolio

Welcome to interactive presentation, created with Publuu. Enjoy the reading!

Alexander Htet Kyaw

Selected Projects

16| New Frontiers

Lunar Habitat Module Multi-objective

Optimization

15| Wind Weaves

Performative Wind Screen and

CFD Simulation

Simulation

Optimization

Robotics

Computer Vision

Material

Fabrication

Simulation

Material

Fabrication

Simulation

Optimization

Extended Reality

8| AR GluLam

High Precision Augmented Reality for

Glulam Fabrication

14|Undulating Frame

Space Frame Structural Simulation,

Analysis and Modeling

9| Convergence

Wire Arc Additive Manufactured

3D Printed Sculpture

10| Bamboo Network

Active Bending in Physics-Based Mixed

Reality Environment

2| AI Assembly

Object Recognition and Computer Vision

for Mixed Reality Assembly

AI/Machine Learning

Extended Reality

Fabrication

Extended Reality

Extended Reality

AI/Machine Learning

Computer Vision

6| UnLog Tower

Gesture Recognition for Feedback-Based

Mixed Reality and Robotic Fabrication

AI/Machine Learning

Fabrication

Robotics

Extended Reality

Fabrication

Material

Simulation

Material

Simulation

Robotics

Fabrication

Fabrication

Material

3| Automated Lego Sorter

Computer Vision and Autonomous

Robotic System

Computer Vision

Robotics

Fabrication

Data Visualization

1| Speech to Reality

Text to Mesh to Voxel to Robotic Toolpath

to Automated Assembly

11| Covid Demographics

Web Application for Interactive Data

Visualization in d3.js

Data Visualization

Data Visualization

AI/Machine Learning

7| Daily Paper Plotter

Machine Building and Design for a

Toilet Paper Plotter

Robotics

12| The Maker-space

Net Zero Plus Community Fabrication

Center

13| Solar Kiosk

Digital Fabrication, Mass Customization

and Performance

Fabrication

Simulation

Optimization

Simulation

Optimization

Data Visualization

Robotics

5| Tangible AI Mind Map

AI Enhanced Mind Map with Large

Language Models and Gesture Recognition

Data Visualization

AI/Machine Learning

Cornell University - Class of 2023

Bachelor of Architecture in Science and Technology

Minor in Computer Information Science

Massachusetts Institute of Technology - Dual Degree Candidate Class of 2025

Master of Science in Electrical Engineering and Computer Science

Master of Science in Architectural Studies in Computation

Contact Info

Email: alexkyaw@mit.edu

Personal: 6072799832

Harvard University

Cross Registered Student

Graduate School of Design

Website

Personal: www.alexanderhtetkyaw.com

Linkedin: www.linkedin.com/in/alexkyaw

Research: https://scholar.google.com/citations?user=g9y-

wRAEAAAAJ&hl=en

4| Curator.AI

Vision Language Understanding for

Contextual Product Recommendation

Extended Reality

AI/Machine Learning

We present a system that transforms speech into physical objects by combining 3D generative

Artificial Intelligence with robotic assembly. The system leverages natural language input to

make design and manufacturing more accessible, enabling individuals without expertise in 3D

modeling or robotic programming to create physical objects. We propose utilizing discrete robot-

ic assembly of to address the challenges of using generative AI outputs in physical production,

such as design variability, fabrication speed, and material waste. The system interprets speech

to generate 3D objects, discretizes them into components, computes an assembly sequence,

and generates toolpath. Various objects, ranging from chairs to shelves, can be are prompted via

speech and realized within 5 minutes using a 6-axis robotic arm.

1 |Speech To Reality

Speaking Objects into existence via 3D Generative AI and Robotic Assembly

Massachusetts Institute of Technology | Fall 2023 - (On Going Research)

Alexander Htet Kyaw, Miana Smith, Se Hwan Jeon and Prof. Neil Gershenfeld

Submited to IEEE International Conference for Robotic Automation (ICRA)

Publication Preprint: https://doi.org/10.48550/arXiv.2409.18390

Speech to Text to 3D Mesh

Assembly Sequence to Robotic Tool Path

AI/Machine Learning

Robotics

Discretization of the AI Generated

Mesh into Assembly Components

Subdivision Grid Based on Component

Size and AI Generated 3D Mesh

Cantilevering Components (Red)

Supported Components (Green)

Assembly Sequence Adjusted

for Cantilevering Components

3D Mesh to Components to Assembly Sequence

https://youtu.be/tEnVV5G-HSg

User Input "Assemble me a table with one leg"

<Processing Speech>

<Running Text-to-Mesh AI Model>

<Voxelizing Geometry>

<Optimizing Assembly Sequence>

<Generating Robotic Tool Path>

<Starting Robotic Assembly>

Assembly time: 3 minutes 36 seconds

Discretization of the AI Generated

Mesh into Assembly Components

Subdivision Grid Based on Component

Size and AI Generated 3D Mesh

Cantilevering Components (Red)

Supported Components (Green)

Assembly Sequence Adjusted

for Cantilevering Components

"A shelf with two tiers"

" I want a small chair "

"I want a tall dog"

User Speech Input: “A stool with four leg”

User Speech Input: “A shelf with two tiers”

Discretization of the AI Generated

Mesh into Assembly Components

Subdivision Grid Based on Component

Size and AI Generated 3D Mesh

Cantilevering Components (Red)

Supported Components (Green)

Assembly Sequence Adjusted

for Cantilevering Components

""A stool with four legs""

ASSEMBLY TIMELINE - User Prompt "I want a stool with three legs"

40 Seconds

1 Minute 10 Seconds

2 Minutes 30 Seconds

ROBOT END EFFECTOR

Printed Circuit Board for Servo

3D Printed Connection with Mounting Plate

End Effector connected to Mounting Plate

PRINTED CIRCUIT BOARD (PCB) for Conveyor Belt Stepper Motor

Printed Circuit Board Design in KiCAD

Milling the PCB design

Soldered Components on the Circuit Board

CONVEYOR BELT

Component Feed in Conveyor Belt

Motor, Belt and Gear

Stopper at the end to prevent slipping

Work-In-Progress Robot-Material System in Collaboration with Miana Smith (Mobile Robots)

Automated Part Selection for Using Vision-Language Models (VLM) and 3D Generative AI for Multi Component Robotic Assembly

User Input

Vision Language Model (VLM) for Automated Part Selection

3D Generative AI

Robotic Assembly

Text to Mesh

(Instant Mesh Model)

Discretized Mesh

Based on Primary Component

Baseline 1

Random Selection

Assign panels by randomly selecting

surfaces across the 3D object

Baseline 2

Predefined Algorithm

Assign panels on all horizontal, top-facing

surfaces aligned in the Z-direction

Our Method

Automated Part Selection via Vision

Language Model (VLM)

Assign panels based on context and

functionality of the object

To evaluate the VLM approach, we compared it against two baseline algorithms. We asked participants to choose options where panels are appropriately

assigned based on functionality of the object. Participants were allowed to pick one or more options for each object. Ten participants evaluated five objects.

"Build a shelf with two tiers"

"A simple chair"

"A Rabbit SitÝng"

"Make me a lamp"

"A Wide Bowl"

"I want a round coffee table"

2 | AI ASSEMBLY

Object Recognition, Computer Vision & Digital Twin for Mixed Reality Assembly

Cornell University | Fall 2022

Expanded Design Thesis Part 1 |Advisors : Jenny Sabin, Sasa Zivkovic

In collaboration with Haotian Ma

Publication Preprint: https://doi.org/10.7298/ha3k-4e73

Recent development in digital modeling tools has enabled architects to design with nonstandard

and custom geometries. While complex components can be digitally fabricated, human

assembly is still required. This thesis explores the potential for human-machine collaboration to

facilitate the co-assembly process. The proposed system utilizes object recognition, computer

vision, digital twin technology, and mixed reality to recognize self-similar components, display

3D assembly instructions, and adapt to real-time changes in the construction environment. All

aspects of the project is done collaboratively with my partner.

Input from the Physical World

Computer Processing

Augmented Reality Display

Step 1: Object Recognition to Identify Components

Step 2: Pick Identified Component

Step 3: Place the Component based on instruction

FEATURE 1: COMPONENT RECOGNITION

AI POWERED AUGMENTED REALITY VIA OBJECT RECOGNITION

Extended Reality

AI/Machine Learning

Computer Vision

https://youtu.be/y3HkzA4hQ4w

16

15

Instruction Number: 146

Instruction Number: 148

Instruction Number: 147

FEATURE 2: END-TO-END ASSEMBLY INSTRUCTIONS

COMPONENTS AND AGGREGATION

Mesh Volume Input

Voxelize/Pixelate

Assign Lego Blocks

CONVENTIONAL

AI ASSEMBLY

2D Drawing and Documentation

3D Blueprints via Augmented Reality

Assembly & Instruction is Separate

Instruction is Seamlessly Superimposed

Search and Sort Before Assembly

Identification of Desired Component

CONVENTIONAL

AI ASSEMBLY

Search and Sort Before Assembly

No Need for Search and Sort

3D Model as Reference for Assembly

Instruction is Seamlessly Superimposed

Text Labels to Differentiate Parts

Labels are displayed via AR

LEGO BLOCKS

CASE STUDY: 3D PRINTED NODES

17

18

WORKFLOW AND DATA PIPELINE

Sorted Assembly 25 Mins 54 Secs

AI Assembly 19 Mins 39 Secs

USER TESTING - ASSEMBLY EXPERIENCE AND TIME (Participant: 2nd year student without prior experience with AR/MR)

Unsorted Assembly 32 Mins 11 Secs

Pascal-Voc: Pattern Analysis, Statistical-Modeling, and Computational Learning - Visual Object Classes

Open CV: Open-source software library of programming functions for real-time computer vision by Intel

TensorFlow: Open-source software library for machine learning and artificial intelligence by Google

REAL TIME CUSTOM OBJECT RECOGNITION AND DIGITAL TWIN

Custom Object Recognition

Digital Twin Location of all Recognized Objects

3D Printed Nodes

with labels in Mixed Reality

AI Powered

Digital Twin

Custom Object

Recognition

19

20

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46

Made with Publuu - flipbook maker