New Breakthrough in Multi-Task Learning: Implicit Syntactic Augmentation Boosts Sentiment Classification Efficiency

Sentiment Analysis (SA) is a fundamental task in natural language processing, aimed at identifying and classifying subjective information in text. Traditional dictionary-based methods, while transparent, struggle with linguistic phenomena such as negation and intensification. Syntactic parsing has been introduced into SA pipelines to address these issues, but conventional parsers incur significant computational overhead. Recent research shows that reframing dependency parsing as a Sequence Labeling (SEL) task can improve efficiency. Against this backdrop, this article explores two strategies for integrating syntactic knowledge into neural sentiment analysis models within a unified Multi-Task Learning (MTL) framework, and analyzes their performance-efficiency trade-offs.

Two Multi-Task Learning Strategies

The study proposes two strategies: Standard-MTL and Contrastive-MTL. Standard-MTL uses a shared encoder and two task-specific heads to jointly learn dependency parsing (via sequence labeling) and sentiment polarity classification. Contrastive-MTL provides implicit syntactic guidance through a contrastive auxiliary objective and a syntax-biased attention pooling mechanism, without requiring explicit label prediction. Both rely on syntactic annotations during training, but neither requires handcrafted composition rules or external parsers at inference time.

Standard-MTL: Explicit Joint Learning

Standard-MTL adopts a conventional multi-task learning architecture, with a shared encoder handling both parsing and sentiment classification tasks. The parsing task is implemented via sequence labeling, while sentiment classification uses another task head. This design allows the model to optimize both objectives simultaneously during training, thereby leveraging syntactic information to enhance sentiment representations. However, the study finds that Standard-MTL's performance is relatively sensitive to the dataset and encoding method, and is not sufficiently stable.

Contrastive-MTL: Implicit Syntactic Guidance

Contrastive-MTL takes a different approach: instead of directly predicting syntactic labels, it lets the model learn syntactic structure through a contrastive learning objective, combined with a syntax-biased attention pooling mechanism. This method also uses syntactic annotations during training, but requires no parser at inference time, thereby avoiding computational overhead. Experiments show that this implicit supervision approach can match or even surpass the effects of explicit parsing.

Multi-Task Learning (MTL) is a machine learning paradigm in which a single model learns multiple related tasks simultaneously, improving generalization and data efficiency by sharing representations and parameters. Its core assumption is that tasks share underlying features, producing positive transfer. In NLP, MTL is commonly used to jointly learn tasks such as parsing, semantic role labeling, and sentiment classification. Common architectures include hard parameter sharing (shared encoder + task-specific heads) and soft parameter sharing (independent parameters per task constrained by regularization). The Standard-MTL in this article adopts a hard-sharing architecture, while Contrastive-MTL achieves more implicit knowledge transfer through a contrastive objective.

Dependency Parsing is a type of syntactic analysis that identifies dependency relations between words in a sentence (e.g., subject-verb, verb-object) and represents them as a directed tree. Traditionally, dependency parsing is treated as a structured prediction problem, commonly using transition-based or graph-based parsers, which incur considerable computational overhead. Reframing dependency parsing as a Sequence Labeling (SEL) task means predicting a dependency label for each word (e.g., the direction and type of the dependency arc), thereby simplifying structured prediction into per-word classification and significantly improving efficiency. This reframing allows the use of standard sequence labeling models such as BiLSTM-CRF, facilitating integration with neural sentiment analysis models.

Contrastive Learning is a self-supervised learning paradigm that learns representations by pulling positive sample pairs closer and pushing negative pairs apart. In NLP, contrastive objectives often construct positive and negative examples through data augmentation or auxiliary tasks. The Contrastive-MTL in this article uses syntactic annotations to construct contrastive objectives: for example, representations of the same sentence under different syntactic perturbations serve as positives, and different sentences as negatives, enabling the model to implicitly capture syntactic structure. Combined with syntax-biased attention pooling, the model assigns higher weights to syntactically relevant words when aggregating word representations, further injecting syntactic knowledge. This implicit supervision avoids explicit parsing at inference time, reducing deployment costs.

Experimental Evaluation and Results

The study evaluates on eleven datasets covering seven languages: the MARC corpus (English, Spanish, French, German) and seven datasets from SemEval-2022 Task 10 (English, Spanish, Norwegian, Basque, Catalan). Experiments use four existing SEL encoding schemes.

Performance Comparison

Under relative encoding, Contrastive-MTL outperforms the syntax-naive baseline on 10 of 11 datasets, and surpasses the external rule pipeline that parses at inference time (SELSP) on all 6 applicable datasets, while evaluating 1.5 to 9 times faster. Standard-MTL's performance is more dependent on the dataset and encoding method.

Relative Encoding is a way of encoding relative positions or syntactic relations between words in sequence labeling. In dependency parsing sequence labeling, relative encoding typically uses the positional offset of each word's dependency head relative to the current word (e.g., +1, -2) as a feature, or encodes the type of dependency label. Compared with absolute position encoding, relative encoding better captures local syntactic structure and is more robust to variations in sentence length. The experiments in this article find that under relative encoding, Contrastive-MTL's performance advantage is most pronounced, indicating that the encoding method significantly affects multi-task learning outcomes. Other encoding methods may include absolute position encoding or POS-based encoding, but relative encoding performs better in syntax-aware tasks.

Efficiency Advantages

Contrastive-MTL's main advantage lies in efficiency: since no parser is needed at inference time, it significantly reduces computational overhead while matching or even exceeding the performance of explicit parsing methods. This is quite important for real-world deployment, especially in resource-constrained environments or those requiring real-time processing.

Multi-Source Cross-Validation

According to analyses from Bilibili creators and YouTube channels, the application of multi-task learning in NLP is gaining increasing attention. By comparing the two strategies, this study demonstrates the effectiveness of implicit syntactic supervision during training. The consensus is that Contrastive-MTL's balance of performance and efficiency makes it a promising direction. However, some viewpoints note that Standard-MTL may still have advantages on specific datasets, and more comprehensive evaluation is needed in the future.

Conclusion and Outlook

The study shows that implicit syntactic supervision during training can match or surpass explicit parsing at inference time, without computational overhead. Contrastive-MTL performs excellently on most datasets, while Standard-MTL is more dependent. This finding offers new ideas for syntactic integration in sentiment analysis, and future work can further explore generalization across different languages and tasks.