AI floor plan segmentation and automatic furniture layout platform
Challenge
A New York interior design and proptech company (confidential, under NDA) wanted homeowners and partner systems to upload a residential floor plan and get back a fully furnished layout. General-purpose segmentation models could not reliably pick out walls, doors and windows on real floor plans, and furniture placement has hard physical rules: clearances around doors, sight-lines to windows and what each room type needs. The same output had to work in an editable app and as a data feed for other systems.
A New York interior design and proptech company (confidential, under NDA) wanted homeowners and partner systems to upload a residential floor plan and get back a fully furnished layout. General-purpose segmentation models could not reliably pick out walls, doors and windows on real floor plans, and furniture placement has hard physical rules: clearances around doors, sight-lines to windows and what each room type needs. The same output had to work in an editable app and as a data feed for other systems.
Solution
We fine-tuned a SAM2 segmentation model on a custom dataset of real-estate floor plans so it detects walls, doors and windows. A rule-based placement engine then furnishes each room against the detected geometry, respecting clearances, window sight-lines and room-type rules, and generates several valid layouts per home. We chose rules over ML-generated layouts so results stay predictable and auditable. The same engine drives a React interface, where users rearrange, swap or regenerate furniture, and a public API that returns the furnished plan as structured data. Python inference runs inside a Node.js and Express backend with MongoDB and Redis on AWS.
We fine-tuned a SAM2 segmentation model on a custom dataset of real-estate floor plans so it detects walls, doors and windows. A rule-based placement engine then furnishes each room against the detected geometry, respecting clearances, window sight-lines and room-type rules, and generates several valid layouts per home. We chose rules over ML-generated layouts so results stay predictable and auditable. The same engine drives a React interface, where users rearrange, swap or regenerate furniture, and a public API that returns the furnished plan as structured data. Python inference runs inside a Node.js and Express backend with MongoDB and Redis on AWS.
Results
The platform segments a floor plan in around 10 seconds, with the fine-tuned model reaching about 90% accuracy on walls, doors and windows in held-out testing. Every home gets one or more furnished layouts that users can edit in the app, and the engine re-checks each change against the placement rules. Partner systems fetch the same furnished plans through the API without any UI work.
The platform segments a floor plan in around 10 seconds, with the fine-tuned model reaching about 90% accuracy on walls, doors and windows in held-out testing. Every home gets one or more furnished layouts that users can edit in the app, and the engine re-checks each change against the placement rules. Partner systems fetch the same furnished plans through the API without any UI work.