SEED-MAX
A feature selection algorithm based on a sequential nested model selection framework. It recursively computes empirical excess risk and progressively selects the candidate feature that most reduces risk, and is used for pruning in random feature extraction.
Timeline (last 90 days)
The paper proposes the SEED-MAX (Successive Empirical Excess Risk Difference Maximization) algorithm, which reformulates feature selection as a variant of the sequential nested model selection problem and identifies the most parsimonious yet most informative feature subset by recursively computing the empirical risk.
The core logic of SEED-MAX is to recursively compute the empirical risk, selecting at each step the candidate feature that can reduce the excess risk to the greatest extent to add to the set, maintaining a set of selected features and obtaining theoretical guarantees through a greedy recursive strategy.
The paper derives a lower bound on the probability that the SEED-MAX algorithm correctly selects features, showing that as the size of the training dataset increases, the probability that the algorithm correctly identifies the optimal feature subset can approach 1 indefinitely.
Building on SEED-MAX, the paper introduces the Fisher test (F-Test) to further evaluate the retained features, used to quickly eliminate features whose contribution is not statistically significant.
On the UCR time series classification datasets, the sequential combination of SEED-MAX and F-Test can achieve a pruning rate of about 92% (only 8% of the original random features are retained), while the drop in classification accuracy is almost negligible.
Comparative experiments show that using SEED-MAX alone can already prune effectively, but adding F-Test further improves the pruning rate while maintaining classification performance.
When SEED-MAX is combined with F-Test, SEED-MAX is responsible for screening the conditionally dependent feature subset from a global risk perspective, while F-Test performs a second round of trimming from the perspective of statistical significance; the two are complementary, raising the pruning rate from the level achieved by SEED-MAX alone to about 92%.
The authors have already open-sourced the implementations of SEED-MAX and SEED-MAX-F-Test on GitHub, making it convenient for researchers and engineers to reproduce and extend.
The applicability of the SEED-MAX framework may not be limited to W-RKS; any learning task that relies on a large number of candidate features and performs screening may benefit from this nested model selection perspective.
Future research can explore the application potential of the SEED-MAX framework in scenarios such as deep learning feature-layer pruning and base learner selection in ensemble learning.
All Facts (10)
When SEED-MAX is combined with F-Test, SEED-MAX is responsible for screening the conditionally dependent feature subset from a global risk perspective, while F-Test performs a second round of trimming from the perspective of statistical significance; the two are complementary, raising the pruning rate from the level achieved by SEED-MAX alone to about 92%.
50%UnverifiedThe core logic of SEED-MAX is to recursively compute the empirical risk, selecting at each step the candidate feature that can reduce the excess risk to the greatest extent to add to the set, maintaining a set of selected features and obtaining theoretical guarantees through a greedy recursive strategy.
50%UnverifiedThe paper derives a lower bound on the probability that the SEED-MAX algorithm correctly selects features, showing that as the size of the training dataset increases, the probability that the algorithm correctly identifies the optimal feature subset can approach 1 indefinitely.
50%UnverifiedBuilding on SEED-MAX, the paper introduces the Fisher test (F-Test) to further evaluate the retained features, used to quickly eliminate features whose contribution is not statistically significant.
50%UnverifiedOn the UCR time series classification datasets, the sequential combination of SEED-MAX and F-Test can achieve a pruning rate of about 92% (only 8% of the original random features are retained), while the drop in classification accuracy is almost negligible.
50%UnverifiedComparative experiments show that using SEED-MAX alone can already prune effectively, but adding F-Test further improves the pruning rate while maintaining classification performance.
50%UnverifiedFuture research can explore the application potential of the SEED-MAX framework in scenarios such as deep learning feature-layer pruning and base learner selection in ensemble learning.
50%UnverifiedThe authors have already open-sourced the implementations of SEED-MAX and SEED-MAX-F-Test on GitHub, making it convenient for researchers and engineers to reproduce and extend.
50%UnverifiedThe applicability of the SEED-MAX framework may not be limited to W-RKS; any learning task that relies on a large number of candidate features and performs screening may benefit from this nested model selection perspective.
50%UnverifiedThe paper proposes the SEED-MAX (Successive Empirical Excess Risk Difference Maximization) algorithm, which reformulates feature selection as a variant of the sequential nested model selection problem and identifies the most parsimonious yet most informative feature subset by recursively computing the empirical risk.
50%