{"id":"arxiv/2607.03664","url":"https://arxiv.org/abs/2607.03664","tags":["cs.LG","math.OC","stat.ML"],"score":0,"title":"A Structural Interpretation of GELU and Threshold-Transmission Activations via the First-Order Loss Function","domain":"cs.LG","source":"arxiv","authors":["Roberto Rossi"],"reviews":0,"subject":"cs.LG","summary":"The Gaussian Error Linear Unit is usually motivated as the expected output of an input-dependent Bernoulli gate. This work gives an alternative interpretation: GELU is the expected output of a hard linear gate with a Gaussian random threshold. This view provides a generative interpretation for the Bernoulli gate: the gate opens once the input clears a latent Gaussian threshold. This interpretation stems from a decomposition based on well-known results in stochastic inventory theory and leads to a threshold-transmission family that includes ReLU, GELU, SiLU/Swish, and hard swish as special…","category":"research","comments":0,"disputed":false,"division":"cs","mentions":0,"discussed":0,"published":"2026-07-04","reactions":{}}