ProbXiv
Explorersign in
Problem archiveProblem record

Statement

Let QQ and KK be arbitrary matrices, and let VV be diagonalizable with dd different positive real eigenvalues. Denote its largest eigenvalue by λmax⁡\lambda_{\max}, and let ξ\xi be a unit eigenvector satisfying

Vξ=λmax⁡ξ.V\xi=\lambda_{\max}\xi.

For the causal transformer dynamics, two-cluster convergence conjecture. For almost every starting point (x1(0),…,xn(0))(x_1(0),\ldots,x_n(0)) with respect to the volume measure on (Sd−1)n(\mathbb{S}^{d-1})^n,

∀k∈[n],lim⁡t→∞xk(t)∈ξ,−ξ.\forall k\in[n],\qquad \lim_{t\to\infty}x_k(t)\in\\{\xi,-\xi\\}.

This conjectures that, when the dominant eigenvalue is positive and has a one-dimensional eigendirection under the stated diagonalizability assumptions, causal attention generically drives all tokens into two antipodal clusters. The claim is presented as part of the paper's conjectural extension beyond the proved identity-matrix case; its resolution is not supplied here.

Record

Comments

Nothing has been published against this problem yet, and nothing has been said about it.

Sign in with an institutional address to take part in the discussion. Reading every thread stays open to everyone.

Sign in

Solve with an agent

Open the statement in a chat, with the problem and the ground rules already written into the prompt.

This opens a third-party site. Nothing is posted back to ProbXiv and nothing you write there is recorded here — what a model gives you is an attempt, which a person still has to check.