This paper introduces new research that seeks to develop low-cost, high quality, mass customizable building assembly systems that provide visually rich design variations for housing or other small structures. The building systems are intended to be tailored for particular cultures and communities by incorporating vernacular decorative design into the assembly design. Two complementary research areas are brought together in this work: shape grammars and digital fabrication. The visual, aesthetic aspects of the research are explored through shape grammars. The physical design and manufacturing aspects are explored through advanced digital design and fabrication technologies, and in particular, build on recent work on monomaterial assemblies with interlocking components that can be fabricated with computer numerical control machines and assembled easily by hand. The long-term objective of this research is the development of formal, visual–physical grammars with rules that generate complete computer-assisted design/computer-assisted manufacturing data for fabrication of full-scale components for assembly design variations. This paper reports on the first phase of this research: pilot studies for prototype assembly systems that incorporate vernacular languages from different parts of the world. The results of these studies are very promising, and demonstrate a spectrum of strategies for embedding visual properties in structural systems. Important next steps in this research are outlined. If successful, this work will lead to new solutions for low-cost, easily manufactured housing, which is especially critical in developing countries and for postdisaster environments. These new housing solutions will not only provide shelter but also support important cultural values through the integration of familiar visual design features. Beyond the specific context of housing and building assemblies, the research has the potential to impact the design and manufacture of designed artifacts on many scales and in many domains, especially in domains where visual aesthetics need to be considered jointly with physical, structural, or material requirements, and where design customization and variation is important.