Building an AI Issue Triage Assistant with Python, Spring Boot, and React

Starting from 1,000 Real Issues

GitHub maintainers spend a lot of time every day reading new issues, deciding which label to apply, and checking whether a similar report already exists. One developer used machine learning to explore how to speed up this "first-pass screening" while leaving the final decision to humans.

The project's core data source is the GitHub Search API — 1,000 recent issues were collected from pandas-dev/pandas, saving titles, descriptions, labels, URLs, and creation times. Training on real public issues is far more convincing than using a few dozen fabricated examples.

But the raw labels need careful handling. An issue may have many labels, or none at all. To do three-way classification (Bug / Docs / Enhancement), the author kept only issues that carried exactly one of these three labels. This way, every training sample has a single definite answer, and pandas' other labels aren't forced into a three-class problem.

Another easy trap is data leakage. Issue titles commonly contain prefixes like "BUG:", "DOC:", or "ENH:". If not cleaned up, the model would simply learn to "copy the answer" from the title prefix rather than learn from the report content. Stripping these prefixes before training is a key step to ensuring the model is actually useful.

Data leakage is a subtle but serious pitfall in machine learning: when training data contains information that would not be available during real prediction, the model performs unusually well on the validation set but fails miserably once deployed. The title prefixes mentioned in the article are a typical example — GitHub issue titles often carry BUG:, DOC:, ENH: prefixes, and these prefixes are almost equivalent to the label itself. If not removed, the model only needs to learn to recognize the prefix to achieve high accuracy, without learning the ability to "judge the type from the report content" at all.

There are many other forms of leakage: using future data to predict the past, mixing the target variable into features in some form, performing global statistics before splitting training and test sets, etc. The core principle of prevention is to simulate the real inference scenario — ask yourself "would this information actually be available at prediction time?" Stripping titles before training in the article is a concrete application of this principle.

How Two Types of Text Features Are Combined

The classifier uses two text representations simultaneously:

• TF-IDF word features: captures keywords and bigram phrases, such as read_csv, error, documentation — these are strong signals
• Sentence embeddings: uses the all-MiniLM-L6-v2 model to generate sentence vectors, capturing broader semantics so that two reports can match even if they use completely different wording

After concatenating the two feature sets, a balanced logistic regression classifier is trained. TF-IDF uses the title plus up to 1,500 characters of description, while sentence embeddings use the title plus up to 1,000 characters of description.

How do the evaluation results look? Among the 1,000 collected issues, 713 carried exactly one of the target labels: 459 Bug, 144 Docs, 110 Enhancement. On a 143-sample holdout set, the classifier reached 0.94 accuracy, with F1 scores of Bug 0.96, Docs 0.88, and Enhancement 0.91.

These numbers measure label classification performance on this dataset, not whether the search can find true duplicate issues. As the issue collection is refreshed, the number of classes and the scores will change.

TF-IDF (Term Frequency-Inverse Document Frequency) is a classic text vectorization method that represents each document as a sparse vector, where each dimension corresponds to a word or phrase in the vocabulary. The weight is the product of two parts: term frequency (how many times the term appears in the current document) and inverse document frequency (the less the term appears in the whole corpus, the higher the weight). This way, words like read_csv or error that concentrate in a few issues get high weights, while pervasive words like the and is approach zero. The advantage of TF-IDF is that both training and inference are extremely fast and highly interpretable — you can directly see which words contribute most to classification; the drawback is that it only matches literal vocabulary and cannot understand synonyms, e.g., "崩溃" and "crash" appear completely unrelated to it.

Sentence embeddings take a different route: they use a pretrained language model (such as the all-MiniLM-L6-v2 mentioned in the article) to encode an entire sentence into a dense vector, typically around 384 dimensions. Such models are trained on massive text and learn to map semantically similar sentences to nearby positions in vector space, so even if two reports use completely different words, as long as their meaning is close, the cosine similarity will be high. all-MiniLM-L6-v2 is a distilled small model, small in size and fast in inference, suitable for local deployment.

Concatenating the two features is a complementary strategy: TF-IDF provides precise lexical signals, sentence embeddings provide fuzzy semantic signals, and the classifier uses both at once, often more robust than using either alone.

Boundaries of Similar Issue Search

The search functionality is also vector-based: a sentence embedding is generated and normalized for each stored issue, the same is done for a new report, and then results are ranked by the dot product of normalized vectors (i.e., cosine similarity), returning the five most similar with their original GitHub links.

This is similarity search, not a verified duplicate detector. High similarity only means the two texts are close in the model's vector space; it does not prove they describe the same bug. Even if both reports are about read_csv, they may involve completely different failure scenarios. Maintainers need to open the link and judge for themselves.

There is also a natural limitation: the search can only return issues from the 1,000 collected reports. If a relevant issue is not in the collection, the app cannot find it.

Cosine similarity is a metric for how close two vectors' directions are, calculated as the dot product of the two vectors divided by the product of their magnitudes, ranging from -1 to 1. In text retrieval, we usually normalize the vectors first (scale the magnitude to 1); then cosine similarity simplifies to a direct dot product, which is computationally more efficient. Cosine similarity only cares about direction, not length, making it especially suitable for comparing text vectors — a long report and a short report will have close vector directions as long as the topic is the same.

Note that high cosine similarity does not mean "the same problem". In sentence embedding space, "read_csv crashes when reading an empty header" and "read_csv errors but continues when reading an empty header" may be very close, yet they are actually different behaviors. That is why the article repeatedly emphasizes that this is similarity search rather than duplicate detection: it ranks by vector distance, returns the five most similar, but leaves the judgment of "is it really the same bug" to the maintainer. This is responsible design — separating what machines are good at (quickly filtering candidates) from what humans are good at (making judgments with context).

How Python, Java, and React Are Wired Together

The request path for the local full-stack version is very clear:

React + Vite → Spring Boot /api/analyze → FastAPI /analyze → Python model

The React page sends a title and optional description, Spring Boot forwards to FastAPI, Python loads the saved classifier, predicts the label, runs similarity search, and returns two parts of the result, which React then displays as a suggested label and linked reports.

The purpose of adding a Java layer is to practice cross-language service connectivity and maintain clear API boundaries. This also makes request and response contracts important — the browser, Java service, and Python service must agree on the same set of JSON fields.

For the public demo, the author built a separate interface with Streamlit that directly loads the same saved Python model and search index. This way users can try it through a single link without starting three local services. The full React/Spring Boot/FastAPI flow is kept in the repository for anyone who wants to study the full-stack architecture to run and inspect.

Cross-language service splitting is common in real systems, but it brings a hidden cost: contract consistency. When a request travels through three layers (browser → Java → Python), any misspelled field name, type mismatch, or missing default value at any layer can cause silent failure. The "request and response contracts" mentioned in the article refer to exactly this — all three parties must agree on the same set of JSON fields (e.g., title, description, predicted_label, similar_issues). In practice, OpenAPI/Swagger or shared interface documentation is typically used to formalize contracts, along with integration tests covering the end-to-end path.

Why add a Java layer may puzzle readers. Functionally, FastAPI could serve the React frontend directly. The author's motivation for adding Spring Boot is to practice cross-language service connectivity and understand API boundaries — a learning-oriented architectural choice, not a performance or deployment necessity. This also explains why the public demo falls back to a single-process Streamlit solution — its goal is one-click user experience, not demonstrating distributed architecture. Both solutions share the same saved model and index, ensuring consistent prediction results.

Trying a Real Report

Enter a title like this:

read_csv crashes when the file has an empty header

The demo suggests the Bug label and returns five related pandas issues. These linked reports are a starting point for investigation, not an automatic duplicate verdict. You can also try documentation and enhancement examples and observe how the predicted label changes with wording and context.

Lessons and Next Steps

The most valuable takeaway from this project is that building an ML feature goes far beyond training a model. You need to collect and clean data, prevent simple label leakage, evaluate on a holdout sample, ensure inference matches training, design APIs, and clearly explain the output's limitations in the interface.

There is still room for improvement. The current dataset is limited, and no benchmark of confirmed duplicate/non-duplicate issue pairs has been built. A next step could be to create such an evaluation set to measure whether related issue ranking actually helps find true duplicates. The issue index could also be expanded and refreshed, and classification performance tracked over time separately for Bug, Docs, and Enhancement.

One-line Summary

Trains a three-way classifier on 1,000 real pandas issues and does similarity search, leaving the final judgment to maintainers.

Paragraph Summary

This project collects 1,000 real GitHub issues from the pandas repository, keeping only the 713 that carry exactly one of the Bug/Docs/Enhancement labels for three-way classification. The classifier concatenates TF-IDF word features with all-MiniLM-L6-v2 sentence embeddings and trains a balanced logistic regression, reaching 0.94 accuracy on 143 holdout samples. Similar issue search is based on cosine similarity of normalized sentence vectors, returning the five most similar with original links, explicitly a starting point for investigation rather than an automatic duplicate verdict, and limited to the 1,000 collected items. The full-stack path is React → Spring Boot → FastAPI → Python model, while the public demo uses Streamlit to directly load the same model. The author emphasizes preventing label leakage from title prefixes, distinguishing label classification from true duplicate detection, and explaining output limitations in the interface.