Local Gemma Model Helps You Figure Out Why Customers Go Silent

Why Chat-Based Selling Often Loses Deals in Silence

Most small sales don't die from a clear "no" — they die from "let me check with her and get back to you," followed by nothing. For a candle maker taking orders in Instagram DMs, a bike mechanic booking repairs on WhatsApp, or a private tutor chatting with parents on WeChat, this happens almost every day.

Developer dongwei-polyred built a tool called FollowUp Local specifically to solve this problem. You just paste in a conversation, and a Gemma model running on your own computer tells you: which stage the customer is stuck at, what their real concern is behind their words, which sentence is the evidence, and how and when to send the next message.

It Can Read Concerns Customers Never Say Out Loud

FollowUp Local categorizes stalled customer states into eight stages, including waiting_on_someone_else, price_hesitation, comparing_options, gone_quiet, and more. Beyond the stage label, the model also provides the real concern, a one-line quote from the customer as evidence, and a suggested follow-up message.

The author demoed a fictional wedding favor order. The customer said "let me confirm with her and get back to you," and the model not only correctly tagged it as waiting_on_someone_else, but also identified that the real concern was the customer being unsure about label quality and wanting to see a physical sample before placing a large order. The evidence was exactly that line — "let me confirm with her." The author stresses that the truly valuable part isn't generic advice like "follow up in two days," but discovering that the root of hesitation was actually that sticker.

Technical Implementation: Small Models Can Do Fine-Grained Work

The tool calls Gemma 3 through Ollama, running entirely on-device. The author tested two sizes on a Linux machine with no GPU (8 vCPUs). gemma3:1b takes about 6–9 seconds per analysis, but often classifies every conversation as asking_questions and gives generic replies. gemma3:4b takes about 15–24 seconds per analysis, picks reasonable stages, reuses facts the seller themselves mentioned (like the $8 sample fee, refunded after ordering), and can reply in Chinese to Chinese conversations. 4B is the default; older laptops can switch to 1B via an environment variable.

Four things make a small model usable. First, use Ollama's format field to pass in a JSON schema that constrains six fields and an enum of eight stages — the model can't invent stages or return half-sentences. Second, provide a three-line example conversation with the ideal answer; the author says this improves quality more than any additional instructions. Third, use keyword signals as hints rather than rules — for example, when phrases like "check with," "我问问," or "考虑一下" are detected, pass them to the model as a HINT. With hints, the 1B model goes from always answering asking_questions to usually picking the right stage, while 4B sometimes disagrees with the hint, which the author considers a good thing — it means the model is making its own judgment. Fourth, validate the model's returned quote; if the quote isn't in the source text, replace it with the last real customer message.

Ollama is an open-source tool for running large language models locally. Users can pull and run open-weight models like Llama, Gemma, and Mistral on their personal computers with a single command, without writing inference code or configuring a CUDA environment. It has built-in model quantization and memory management, supports calls via REST API or Python library, and provides a format parameter to pass in a JSON schema that constrains output structure. For developers who want to embed AI capabilities into their own tools without handing data to the cloud, Ollama is essentially a ready-to-use local inference server. FollowUp Local in this article calls Gemma 3 through Ollama, enabling analysis even on a CPU machine with no GPU.

Gemma 3 is a series of open-weight lightweight language models released by Google, with parameter sizes covering 1B, 4B, 12B, 27B, and more, supporting multiple languages and longer contexts. Unlike closed-source APIs, anyone can download the weights and run or fine-tune them on their own hardware. The 1B and 4B versions are designed for edge devices and CPU inference — after quantization they have low memory footprint and fast response, but their reasoning ability and instruction-following are noticeably weaker than larger models. The author chose Gemma 3's 1B and 4B versions precisely because they can run on ordinary laptops, while compensating for capability gaps through structured prompt engineering — validating the feasibility of "small model + good engineering" in vertical tasks.

It Knows Clearly That It Will Make Mistakes

The author doesn't shy away from the weaknesses. In testing, the 4B model sometimes claimed to have performed an action ("I just sent the sticker photo"), sometimes added comparisons the seller never mentioned ("something other stores don't have"), and in Chinese got family relationships wrong (writing "your dad" instead of "the child's dad").

So FollowUp Local only generates drafts for people to read — it never sends any messages automatically. For messages sent in the name of a small business owner, the author believes human oversight is actually the correct design.

JSON Schema is a standard format for describing JSON data structures. It can specify which fields an object contains, the type of each field, required items, and enumeration value ranges. In LLM calls, passing a schema to an inference framework that supports structured output (like Ollama's format) causes the framework to restrict the model during decoding so it can only generate tokens conforming to the schema — preventing the model from going off-script and producing missing fields, type errors, or invented stage labels. This article uses exactly this mechanism to lock output to six fields and an enum of eight stages, which is the key means of getting 1B/4B small models to produce programmatically parseable results.

Why Open Weights Are Critical for This Scenario

The author explains from four angles why open weights are a prerequisite for this tool. Chat logs contain customer names, budgets, family situations, and even addresses — the most private data of small sellers. Pasting a customer's WhatsApp into the cloud is hard to accept, and may not even be something the seller has the right to share. Local operation means conversations never leave the computer — users can even verify this by unplugging the network cable.

The cost structure is equally critical. The value of each analysis is real but small — a seller might run dozens per day. With a hosted API, every analysis incurs a cost, forcing per-message pricing or a subscription model. Local inference has zero marginal cost, making a one-time purchase possible. Open model weights also let the author freely choose between 1B and 4B. Future "vertical industry versions" would essentially mean fine-tuning Gemma on licensed conversations from an industry — something a closed-source interface can't do. Finally, if the author stops maintaining it, the tool and model can still keep running on the user's machine.

Open Weights refers to the parameter files of a trained model being publicly released, so anyone can download, run locally, modify, or even fine-tune them — in contrast to closed-source models that only provide API access. Open weights let data stay entirely local, make inference marginal cost zero, let users freely choose model sizes, and allow developers to do vertical fine-tuning based on their own industry data. The author emphasizes that chat logs involve customer privacy; with cloud APIs, it's hard to convince small sellers to paste conversations. Open weights let FollowUp Local run on the user's own computer, and even if the author stops maintaining it, the tool and model remain usable.

Commercialization Ideas

The author envisions two sales approaches. One is a packaged desktop app with the model built in, sold as a one-time download, aimed at users who will never open a terminal. The other is vertical versions for a single industry — tutoring agencies or wedding vendors, for example — customizing stage names, signals, and example replies based on how customers in that industry actually hesitate. Since inference runs on the customer's own hardware, the serving cost of each additional user is zero.

Editorial View

This project demonstrates a path worth watching: using schema constraints plus a bit of prompt engineering to bring 4B-level local models to a usable level in vertical scenarios. Its value isn't in how strong the model is, but in breaking a fuzzy interpersonal judgment problem into enumerable, verifiable structured output — and in using the "generate drafts only, never send automatically" design to hold the line on small-model errors. For any developer wanting to run AI applications locally, this combination of structured output, single-shot examples, and keyword hints may be more effective than switching to a larger model.

One-line Summary

FollowUp Local uses a local Gemma model to help small sellers analyze why customers go silent and draft follow-up messages.

Paragraph Summary

FollowUp Local is a locally-run open-source tool designed for small sellers taking orders in DMs, solving the problem of customers disappearing after "let me check." It calls Gemma 3 through Ollama, classifying conversations into eight stall stages and outputting real concerns, source-text evidence, and follow-up message drafts. The 4B model takes about 15–24 seconds per analysis on a CPU machine with no GPU and handles Chinese and English; 1B is faster but less accurate. The author uses four engineering techniques — schema constraints, single-shot examples, keyword hints, and quote validation — to bring small models to a usable level, and explicitly lists three types of errors, hence generating drafts only without auto-sending. Open weights ensure privacy, zero marginal cost, and fine-tunability.