index462 – Attention Driven by Your Text next

Attention Explorer (Text-Aware) index462

This page ingests the shared text from index461. The sequence length slider is capped by your current word count. Adjust d_k & temperature τ to see how attention sharpens or smooths. The generated random embeddings are stable unless you re-randomize.

Editable Text & Parameters

EDIT TEXT (shared up to 500 words)

Words: {{meta.total}}

Unique: {{meta.vocab}}

H(p): {{meta.entropy | number:3}}

PPL: {{meta.perplexity | number:1}}

Parameters

\(\mathrm{Attention}(Q,K,V) = \operatorname{softmax}\left( \frac{QK^{T}}{\sqrt{d_k}\,\tau} \right) V\)

WORDS IN SHARED TEXT {{meta.total}} MAX ALLOWABLE L {{meta.maxL}} SEQUENCE LENGTH L {{params.seqLen}} KEY / QUERY DIMENSION d_k {{params.d_k}} TEMPERATURE τ {{params.temperature | number:2}}

Entropy (text): {{meta.entropy | number:3}}

Perplexity: {{meta.perplexity | number:1}}

|V|: {{meta.vocab}}

Higher text entropy generally implies more diverse keys—making sharp peaks less likely unless d_k is large or τ small.

Attention Matrix

Row softmax distributions (L × L). Hover for weight.

q {{hoverInfo.q}} → k {{hoverInfo.k}} : w = {{hoverInfo.w | number:4}}

Value Output (Aggregated)

{{valueVectors | json}}