Unverified50% confidenceFactExact time
Hidden states refer to the dense vector representations produced by the internal layers of a large language model when generating each token, carrying the model's compressed encoding of the current context semantics.
1
Sources
50%
Confidence
Long-term
Relevance
11/19/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedTaking Llama-3.1-8B as an example, each token has a hidden state vector of dimension 4096 at each layer, encoding the semantic, syntactic, and even pragmatic information of that token in the current context.76% similarUnverifiedA sparse autoencoder is an unsupervised learning architecture that compresses high-dimensional model activation vectors into an overcomplete sparse latent space and then reconstructs the original activations, forcing each latent variable to be sensitive to only a few input patterns.70% similarUnverifiedResearchers have found that hidden character techniques can be used to launch prompt injection attacks against large language models, embedding hidden instructions within normal text that the model will parse and execute.70% similarUnverifiedLogit-lens projects the hidden states of each layer into the vocabulary logit space to observe what the model would output if forced to decode at that layer.70% similarUnverifiedAn embedding vector is a dense vector representation that maps text into a high-dimensional space, typically generated by a pretrained language model.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1186429API
curl https://kongchang.com/api/v1/knowledge/claims/1186429MCP
get_claim(id=1186429)