Forward Pass
The fundamental computation process of neural network inference, in which a model receives an input token sequence and produces a probability distribution through successive layer computations. Each API call to an LLM constitutes an independent forward pass and retains no state across requests.
Timeline (last 90 days)
Improper weight initialization causes signals to decay or explode layer by layer during forward propagation, and gradients to vanish or diverge during backpropagation.
A forward pass is the process of a neural network computing once completely from the input layer to the output layer; an autoregressive language model needs N forward passes to generate N tokens, with latency growing linearly with output length; a classifier-style model needs only one forward pass.
Drex 1.5 can return probabilities for all candidate options in a single forward pass
When using a generative model to evaluate K candidate options, the total cost grows linearly with the number of options, whereas a single forward pass can compress the inference cost to a single computation
All Facts (4)
Improper weight initialization causes signals to decay or explode layer by layer during forward propagation, and gradients to vanish or diverge during backpropagation.
50%UnverifiedA forward pass is the process of a neural network computing once completely from the input layer to the output layer; an autoregressive language model needs N forward passes to generate N tokens, with latency growing linearly with output length; a classifier-style model needs only one forward pass.
50%UnverifiedDrex 1.5 can return probabilities for all candidate options in a single forward pass
50%UnverifiedWhen using a generative model to evaluate K candidate options, the total cost grows linearly with the number of options, whereas a single forward pass can compress the inference cost to a single computation
50%