{"id":"arxiv/2605.08318","url":"https://arxiv.org/abs/2605.08318","tags":["cs.LG","cs.AI","math.NA","physics.comp-ph","stat.ML"],"score":0,"title":"When Attention Beats Fourier: Multi-Scale Transformers for PDE Solving on Irregular Domains","domain":"cs.LG","source":"arxiv","authors":["Brandon Yee","Pairie Koh","Jack Rodriguez","Mihir Tekal"],"reviews":0,"subject":"cs.LG","summary":"We study the problem of \\emph{architecture selection} for deep learning models trained to solve partial differential equations (PDEs), asking when transformer-based architectures with learned attention outperform Fourier-domain neural operators. We introduce the \\textbf{Multi-Scale Attention Transformer} (\\msat{}), a deep learning architecture that encodes spatiotemporal solution histories as token sequences and trains end-to-end via a composite supervised objective with optional physics-informed regularization terms. We conduct a comprehensive empirical evaluation against nine baselines --…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-05-08","reactions":{}}