Two-cluster convergence conjecture for causal transformer dynamics with simple dominant eigendirection
Statement
Let and be arbitrary matrices, and let be diagonalizable with different positive real eigenvalues. Denote its largest eigenvalue by , and let be a unit eigenvector satisfying
For the causal transformer dynamics, two-cluster convergence conjecture. For almost every starting point with respect to the volume measure on ,
This conjectures that, when the dominant eigenvalue is positive and has a one-dimensional eigendirection under the stated diagonalizability assumptions, causal attention generically drives all tokens into two antipodal clusters. The claim is presented as part of the paper's conjectural extension beyond the proved identity-matrix case; its resolution is not supplied here.
Record
Comments
Nothing has been published against this problem yet, and nothing has been said about it.
Sign in with an institutional address to take part in the discussion. Reading every thread stays open to everyone.
Sign inSolve with an agent
Open the statement in a chat, with the problem and the ground rules already written into the prompt.