Locality
An inductive bias in neural networks whereby each neuron connects only to a local region of the input; it is one of the core design principles of convolutional networks.
Timeline (last 90 days)
In the teacher-student model, when there is no correlation between input positions (c=0), the convolutional student and the locally connected student follow the same learning trajectory, but the convolutional student learns exactly P times as fast as the locally connected student (P being the number of input positions).
When the student network's convolution kernel size matches the teacher's, only O(D) samples are needed to recover the teacher parameters; if the kernel size is larger, O(D log D) samples are needed (D being the feature dimension).
Locality and weight sharing do not always synergize: when inputs are highly correlated, weight sharing may harm performance, and the benefit of locality depends on how well the kernel size matches the task structure.
All Facts (3)
In the teacher-student model, when there is no correlation between input positions (c=0), the convolutional student and the locally connected student follow the same learning trajectory, but the convolutional student learns exactly P times as fast as the locally connected student (P being the number of input positions).
50%UnverifiedWhen the student network's convolution kernel size matches the teacher's, only O(D) samples are needed to recover the teacher parameters; if the kernel size is larger, O(D log D) samples are needed (D being the feature dimension).
50%UnverifiedLocality and weight sharing do not always synergize: when inputs are highly correlated, weight sharing may harm performance, and the benefit of locality depends on how well the kernel size matches the task structure.
50%