MoPrO-SD
A modular prompt optimization framework for stance detection tasks. It decomposes prompts into functional semantic modules and refines them sequentially, combined with category-conditioned error analysis, to improve large language model performance on stance detection tasks.
Timeline (last 90 days)
MoPrO-SD is a modular prompt optimization framework specifically designed for stance detection.
The core approach of MoPrO-SD is to decompose prompts into functional semantic modules and then sequentially refine these modules using the LLM's own feedback.
MoPrO-SD investigates three refinement strategies: instruction-only optimization (I), instruction plus reasoning (I+R), and instruction plus per-example reasoning (I+R+E).
MoPrO-SD uses class-conditioned errors to refine stance labels by separately analyzing errors for each stance category.
Both the VaccineStance dataset and the code for MoPrO-SD have been open-sourced on GitHub.
The research conducted experiments on four datasets and five open-weight LLMs.
MoPrO-SD outperforms manually written baseline prompts (Base prompt) in many settings.
On the SemEval-2016 dataset, class-conditioned feedback achieved higher average Macro-F1 scores than aggregated feedback on four out of five backbone models.
The two reasoning-based strategies I+R and I+R+E both perform strongly, but their relative advantage varies across different models and datasets, with neither strategy absolutely outperforming the other in all scenarios.
All Facts (9)
MoPrO-SD is a modular prompt optimization framework specifically designed for stance detection.
50%UnverifiedThe core approach of MoPrO-SD is to decompose prompts into functional semantic modules and then sequentially refine these modules using the LLM's own feedback.
50%UnverifiedMoPrO-SD investigates three refinement strategies: instruction-only optimization (I), instruction plus reasoning (I+R), and instruction plus per-example reasoning (I+R+E).
50%UnverifiedMoPrO-SD uses class-conditioned errors to refine stance labels by separately analyzing errors for each stance category.
50%UnverifiedThe two reasoning-based strategies I+R and I+R+E both perform strongly, but their relative advantage varies across different models and datasets, with neither strategy absolutely outperforming the other in all scenarios.
50%UnverifiedThe research conducted experiments on four datasets and five open-weight LLMs.
50%UnverifiedMoPrO-SD outperforms manually written baseline prompts (Base prompt) in many settings.
50%UnverifiedOn the SemEval-2016 dataset, class-conditioned feedback achieved higher average Macro-F1 scores than aggregated feedback on four out of five backbone models.
50%UnverifiedBoth the VaccineStance dataset and the code for MoPrO-SD have been open-sourced on GitHub.
50%