The advantages of frozen embeddings are extremely low computational cost, no risk of overfitting, and the ability to be pre-computed and cached for repeated use; the disadvantage is that the representations may not perfectly match the distribution of the target task.