[KongchangAI]
ConceptBlock-Diagonal MLP

Block-Diagonal MLP

A fully connected layer whose weight matrix has a block-diagonal structure, with the parameter matrix nonzero only within several disjoint diagonal blocks. It is equivalent to partitioning the input features into subgroups, applying an independent linear transformation to each subgroup, and concatenating the results, reducing the parameter count from O(n²) to O(n²/b) with a corresponding reduction in computation.

Timeline (last 90 days)

Nov 9

The spectral branch of SSF-Net uses the Fourier transform to map features into the frequency domain, and then processes global spectral features through a block-diagonal MLP.

Unverified50%
Nov 9

The weight matrix of the block-diagonal MLP has a block-diagonal structure, which is equivalent to partitioning the input features into several sub-groups and applying linear transformations to each separately, reducing the parameter count from O(n²) to O(n²/b) (b is the number of blocks).

Unverified50%
Nov 9

SSF-Net's differentiation lies in replacing the fully connected spectral processing commonly used in combinations of Mamba-type structures with Fourier methods by a block-diagonal MLP, thereby compressing the parameter count.

Unverified50%

All Facts (3)

Source Articles