PCA

SVD is the mathematical core of PCA dimensionality reduction, recommender system matrix factorization, and the LoRA low-rank adaptation fine-tuning method.

PCA 降维通过对数据协方差矩阵做特征分解，找到数据方差最大的投影轴，从而在最大保留信息的前提下压缩维度

PCA (Principal Component Analysis) is a classic unsupervised dimensionality reduction technique that finds the directions of greatest variance in data (principal components) by computing the eigenvalues and eigenvectors of the data's covariance matrix.

PCA was first proposed by Karl Pearson in 1901 and later developed and refined by Harold Hotelling in 1933.

PCA is equivalent to performing eigenvalue decomposition on the data covariance matrix; if SVD is applied to the centered data matrix X to obtain X=UΣVᵀ, then the right singular vectors V are the principal component directions of PCA, and the squared singular values divided by the number of samples are the variances corresponding to each principal component.

Principal Component Analysis (PCA) involves both eigenvalue decomposition from linear algebra and variance maximization from statistics.

The abliteration technique constructs a contrastive dataset of harmful requests versus normal requests, uses PCA or mean-difference methods to locate the principal direction vector of refusal behavior, then orthogonally removes it from the weight matrix via projection—requiring no gradient updates or retraining, and can be completed in a few minutes on a consumer-grade GPU.

PRSR的低秩子空间估计通过对平行句对残差向量组成的矩阵做SVD或PCA，取前k个奇异向量张成子空间，并将最后一层token隐藏状态投影到该子空间的正交补上。

Common dimensionality reduction techniques such as t-SNE, UMAP, and PCA can project high-dimensional embedding vectors onto two-dimensional or three-dimensional planes to observe clustering structures.

Traditional methods such as PCA and TRPCA tend to flatten the data or ignore its inherent multimodal tensor structure, thereby failing when confronted with complex faults involving sensor-time correlations

In the same case study, TRPCA and PCA-based anomaly detectors achieved only 0.69 or lower macro F1 scores

PCA is a linear dimensionality reduction method that retains the directions of greatest global variance; it is computationally fast but has limited expressiveness for nonlinear structures.

The author tried three dimensionality reduction methods—PCA, UMAP, and tSNE—and ultimately concluded that tSNE produced the best structure.

The project includes PCA and t-SNE dimensionality reduction analysis implemented in NumPy, allowing layer-by-layer viewing of the test set's distribution in feature space.

The paper proposes a training-free, prompt-conditioned method for LLM style axis discovery that extracts style structure from model decoding variance by repeatedly sampling at high temperature on the same prompt.

The method proceeds in three steps: repeatedly sampling a single prompt at high temperature to generate multiple completions, applying PCA dimensionality reduction to the hidden activations of the completions, and automatically assigning semantic labels to the positive and negative poles of each principal axis—all without updating model parameters.

PCA reduces the dimensionality of hidden activations from multiple samples of the same prompt; the first principal component captures the direction of greatest variance among the sampled results, corresponding to the style dimension along which the model varies most dramatically.

The low recall indicates that relying solely on the top two principal axes with the greatest variance cannot cover all style dimensions that humans care about; extending to more principal components can improve recall but introduces noisy axes that cause precision to decline.

The research employs SMOTE (Synthetic Minority Over-sampling Technique) and PCA (Principal Component Analysis) for data preprocessing.

The research designs four hybrid preprocessing configurations for comparison: normalization only, SMOTE balancing, PCA transformation, and SMOTE combined with PCA.