City Decoder: an Interpretable Multimodal Framework for Urban Pattern Recognition

Contemporary urban environments are increasingly shaped by intertwined layers of infrastructure, data, and regulation that produce spatial patterns difficult to interpret through single-modality analyses or purely predictive models. While recent AI-driven urban studies have advanced large-scale measurement and forecasting, they often struggle to (i) integrate heterogeneous modalities such as imagery, geospatial networks, archives, and planning documents, (ii) relate spatial change to regulatory and institutional timelines, and (iii) generate traceable outputs that support architectural and planning reasoning. This study addresses these limitations by proposing an interpretable, multimodal framework for urban pattern interpretation. The paper introduces City Decoder , a spatiotemporal pattern-recognition framework designed to decode urban environments through evidence-linked interpretation rather than prediction or optimisation. Methodologically, the research conducts a structured comparative review of representative trajectories from major global city laboratories—including Beijing City Lab, MIT Senseable City Lab, ETH Future Cities Laboratory , and Stanford Urban Informatics Lab —coding their dominant data modalities, analytical tasks, and output types. Based on this synthesis, City Decoder is specified as an operational pipeline that aligns time-series urban imagery, vector geospatial data, and regulatory texts within a shared spatial–temporal reference. The framework performs recurrence detection, cross-layer coupling analysis, and discontinuity identification between policy intent and material outcomes. The framework is designed to produce a Pattern Atlas , relational mappings, and diagnostic reports intended to support architectural and urban analysis, policy evaluation, and scenario-based design inquiry. The primary contribution is a transferable methodological specification for interpretive, traceable pattern recognition in urban research—one that repositions multimodal AI from a predictive tool toward an instrument of spatial understanding.