Existence of double scaling limit for random neural networks
Statement
Consider a random depth neural network with input dimension , hidden layer widths
output dimension and non-linearity . Suppose that the network is tuned to criticality, meaning that the criticality condition is satisfied. Fix a non-zero network input and write . For each , let denote the corresponding even cumulant and let denote the corresponding second moment. Existence of the double scaling limit. For each there exists , depending on the universality class of , such that
Moreover, for each there exists a probability distribution on , depending only on and , such that in the double scaling limit
the random variable converges in distribution to a random variable with law . This conjecture proposes a universal limiting description of critically tuned random neural networks when width and depth grow proportionally; the existence of the limiting distribution and the stated cumulant asymptotics are not established in the supplied text.
Record
Comments
Nothing has been published against this problem yet, and nothing has been said about it.
Sign in with an institutional address to take part in the discussion. Reading every thread stays open to everyone.
Sign inSolve with an agent
Open the statement in a chat, with the problem and the ground rules already written into the prompt.