TurnTurk-GPT: A Turkish Turn Prediction Model Where Word-Final Morphology Is Key

Introduction

Turn-taking in conversation is at the heart of natural interaction. Accurately predicting when a speaker will finish their turn is essential for building fluent voice assistants, real-time translation, and customer service bots. However, this task poses unique challenges in morphologically rich languages such as Turkish. A recent paper introduces TurnTurk-GPT, a text-based Turkish turn predictor that fine-tunes a Turkish GPT-2 model with the TurnGPT objective and is systematically evaluated on real conversational data.

Model and Evaluation Methodology

The core of TurnTurk-GPT is to transfer the TurnGPT training objective to a Turkish GPT-2, fine-tuning it on both synthetic and real conversations. Evaluation is conducted on real Turkish dialogues and compared against a rule-based baseline, a zero-shot Turkish language model, and a widely used multilingual end-of-turn detector.

The evaluation design closely mirrors realistic streaming ASR scenarios: it uses unpunctuated, lowercased text, makes decisions at the word level, and adopts record-disjoint and speaker-disjoint data splits, nested threshold selection, and paired cluster bootstrap tests. This rigorous setup ensures the reliability of the results.

TurnGPT objective: TurnGPT is a Transformer-based dialogue turn prediction model whose training objective typically employs contrastive learning or binary classification to determine whether the current word is an end-of-turn point. Specifically, the model receives dialogue context, outputs the probability that each word position is an end-of-turn, and is optimized via cross-entropy loss. This objective enables the model to capture prosodic and syntactic cues in dialogue rather than relying on explicitly annotated turn boundaries. TurnTurk-GPT transfers this objective to a Turkish GPT-2, leveraging its pretrained linguistic knowledge for fine-tuning to adapt to Turkish morphological properties.

Average Precision (AP): Average Precision (AP) is a commonly used evaluation metric in information retrieval and classification tasks, especially suited to class-imbalanced scenarios. It computes the area under the precision-recall curve, comprehensively accounting for model performance across different thresholds. In turn prediction, a higher AP indicates that the model can identify end-of-turn points while maintaining a low false positive rate. Compared with accuracy, AP is more sensitive to the recognition of the minority class (end-of-turn points), making it better suited to evaluating turn prediction, a task with imbalanced positive and negative samples.

Key Findings: Morphological Cues and Generalization Ability

The experiments reveal several important phenomena. TurnTurk-GPT outperforms the multilingual detector in both record-disjoint and speaker-disjoint evaluations. However, it outperforms the zero-shot language model only when the test speaker appears in the training set; for unseen speakers, the two are statistically indistinguishable. This indicates that the model's generalization ability with respect to speaker identity is limited.

More strikingly, removing suffixes from turn-final words causes Average Precision (AP) to drop by as much as 44%, whereas removing suffixes from intra-sentential words causes only a 2% drop. This clearly localizes Turkish end-of-turn cues to verb-final morphology. Turkish is a verb-final language, and suffixes carry tense, person, and mood information; the model evidently learns to exploit these morphological features.

Furthermore, the model trained only on synthetic conversations performs worst, showing that real conversational data is indispensable for capturing natural turn-taking patterns. When speakers pause, the zero-shot and multilingual baselines approach random-chance levels, whereas TurnTurk-GPT still maintains some performance.

Morphologically rich languages and suffixes: Morphologically rich languages (such as Turkish, Finnish, and Hungarian) express grammatical relations through numerous suffixes; a single root can carry multiple suffixes indicating tense, person, number, case, and so on. In Turkish, verb-final suffixes are particularly critical, often carrying tense and person information and potentially signaling sentence endings. For example, the past tense first-person singular suffix "-dim" marks both tense and person, and this information density enables the model to learn end-of-turn cues from word-final morphology. Removing these suffixes destroys the key features the model relies on, leading to significant performance degradation.

Paired cluster bootstrap test: Paired cluster bootstrap is a statistical resampling method used to assess the significance of differences in model performance. It resamples conversation records or speakers (cluster units) with replacement to generate multiple bootstrap samples, computes the distribution of performance metrics, and thereby obtains confidence intervals and p-values. This method can handle the non-independence in dialogue data (samples within the same speaker or same record being correlated) and is more robust than traditional bootstrap. In the evaluation of TurnTurk-GPT, this test is used to compare performance differences among different models under record-disjoint and speaker-disjoint settings, ensuring the statistical reliability of the conclusions.

Significance and Outlook

The contribution of TurnTurk-GPT lies not only in the model itself but also in the public release of the evaluation protocol and model, providing a benchmark for turn prediction research in Turkish and other morphologically rich languages. The study emphasizes the importance of morphology in turn prediction and points out the challenge of cross-speaker generalization. Future work can explore how to incorporate speaker adaptation techniques and how to fuse text models with acoustic features to further improve robustness.

For dialogue system developers, this research suggests that in morphologically rich languages, ignoring word-final inflection can lead to significant performance degradation. When building multilingual dialogue systems, turn prediction modules need to be customized to the characteristics of each language.

One-line Summary

TurnTurk-GPT fine-tunes Turkish GPT-2 for turn prediction, revealing the key role of verb-final morphology and the challenge of cross-speaker generalization.

Paragraph Summary

TurnTurk-GPT is a text-based turn predictor built on Turkish GPT-2, fine-tuned on real conversations with the TurnGPT objective. Evaluation is conducted in streaming ASR scenarios using unpunctuated lowercased text and word-level decision points, with record-disjoint and speaker-disjoint splits, nested threshold selection, and paired cluster bootstrap tests. Results show that the model outperforms the multilingual detector under both splits, but outperforms the zero-shot language model only when the test speaker is seen in training. Removing suffixes from turn-final words causes a 44% drop in Average Precision, whereas removing suffixes from intra-sentential words causes only a 2% drop, indicating that end-of-turn cues reside mainly in verb-final morphology. The model trained only on synthetic conversations performs worst, highlighting the importance of real data. The model and evaluation protocol are publicly released, providing a benchmark for turn prediction in morphologically rich languages.