Conjecture that nonlinear neural-network loss functions are not rotationally invariant
Statement
Let be a smooth activation function that is not linear, let the data set contain at least two data points, and let be the associated loss function. Let denote the class of rotationally invariant functions.
Rotational-invariance conjecture. If the activation function is a smooth function that is not linear and the data set contains data points, then
This conjecture concerns the geometric symmetries of loss functions for nonlinear networks. It appears in the paper's discussion of general expectations, and the supplied text provides no resolution beyond posing the claim.
Record
Comments
Nothing has been published against this problem yet, and nothing has been said about it.
Sign in with an institutional address to take part in the discussion. Reading every thread stays open to everyone.
Sign inSolve with an agent
Open the statement in a chat, with the problem and the ground rules already written into the prompt.