[KongchangAI]
· 2 min read· 1,497 words

Gemini 4 Neon Opens Paid API: Million-Token Long Tasks and Local LLM Benchmarks

Gemini 4 Neon Opens Paid API: Million-Token Long Tasks and Local LLM Benchmarks

Gemini 4 Neon's paid API opens, OpenAI expands live dictation, plus local LLM benchmarks and GitHub Skills rankings.

Today's LLM roundup covers four developments. Gemini 4 Neon is fully available on the paid Gemini API, handling up to 1M tokens per request at $3/$15 per million input/output tokens, targeting long complex tasks—with trade-offs in cost and information density. OpenAI's live dictation now spans iOS, Android, web, and desktop with the 10-minute limit removed. Local LLM tests show the same model on a 32GB machine ranging from 60.8 to 9.2 tokens/s depending on inference software, with peak memory near total capacity. GitHub Skills rankings by stars show coding agents splitting into reusable capability units, though rankings are volatile and only measure attention.

Google's Gemini 4 Neon has officially opened its paid API to developers, focusing on long context and complex task handling. Meanwhile, OpenAI upgraded voice input, local LLM benchmark results surfaced, and GitHub Skills rankings were updated—together forming today's main developments in the LLM space.

Gemini 4 Neon: A Large-Context Model for Long Tasks

According to reporting from the YouTube channel Today's LLM News, Google Gemini 4 Neon is now fully available on the paid Gemini API. The model is positioned for handling "long and complex tasks," and its most notable feature is the ability to process up to 1 million tokens (or wordpieces) in a single request—equivalent to handing a lengthy report to the model for a full read-through at once, without needing to split it into chunks.

On pricing, effective November 18, input costs $3 per million tokens and output costs $15 per million tokens, with costs rising as input scale grows. This pricing structure follows the industry convention of billing input and output separately, but million-token context means a single call may incur a higher absolute cost, so developers need to estimate costs in advance when designing long-document processing pipelines.

What Long Context Actually Means

The value of a million-token context isn't that "the number is bigger"—it's that it changes how tasks are organized. In the past, processing long reports, multiple contracts, or large codebases typically required retrieval augmentation or chunk-and-stitch approaches; now you can try feeding everything in at once. The trade-off is higher cost and greater demands on effective information density—the longer the context, the more worth watching is the risk of the model losing details in middle sections.

Token and wordpiece: A token is the basic unit of text a language model processes; the model first splits text into tokens and then encodes them into vectors. For English, one token roughly corresponds to 0.75 words, while for Chinese it's often one to two characters. Wordpiece is the subword tokenization algorithm used by the BERT family, which splits rare words into more common fragments based on frequency, so "1 million tokens or wordpieces" refers to an input budget of the same order of magnitude. Understanding the tokenization method helps with estimation: a 100,000-character report corresponds to roughly 100,000-plus tokens, so a million-token context can hold several such documents—but raw character count and token count are not equivalent.

Retrieval-augmented generation (RAG) and chunking: RAG first retrieves relevant fragments from an external knowledge base, then feeds them to the model along with the question; chunking means splitting a long document into small segments for separate processing and then stitching the results together. Both exist to work around context window limits. Million-token context makes "feed it all in" possible, but RAG still has advantages in cost, traceability, and ultra-long documents, and in practice the two are often combined.

The "lost in the middle" phenomenon: Research shows models make better use of information at the beginning and end of context, with lower recall for key details in middle sections—long context does not equal uniform attention. So even after stuffing an entire document in, you should still reduce the risk of omissions through explicit instructions, structured questions, or placing key information up front.

OpenAI Voice Input: Live Dictation Across Multiple Platforms

OpenAI is rolling out live dictation, which converts speech to text. According to the same source, the feature will cover iOS, Android, web, and desktop, while also removing the previous 10-minute recording limit.

The practical impact of removing the time limit is worth noting: long meetings, dictated notes, and interview transcription no longer require segmented recording, and voice input is moving from a "short-phrase tool" toward a "continuous recording tool." Multi-platform coverage also means users can carry the same input habits across different devices.

Local LLM Benchmarks: Software and Format Determine Speed

Running large models locally is another parallel track. According to public tests, QEN3-8-27B running on a 32GB machine was measured for output speed during short generation: it reached 60.8 tokens per second with Mirai Uzu, but only 9.2 tokens per second with OMLX. The same model's 4-bit MLX quantized checkpoint occupies 16.1GB of disk.

The same model's 4-bit MLX checkpoint, a compressed copy of the model,

The test also measured wired memory (the portion that must reside in physical memory), with peak values ranging from about 19.6GB to 24.9GB, varying with software and model format. The key takeaway from this data: with the same model and the same hardware, simply switching inference software can produce order-of-magnitude speed differences; and peak memory approaching or even exceeding the 32GB total capacity means you must leave headroom for the system and other applications in real use.

Real-World Constraints of Local Deployment

For developers hoping to run models locally, selection isn't just about model parameters—it's also about the maturity of the inference framework, compatibility of quantization formats, and whether peak memory leaves enough buffer. The gap between the two software options in the test shows that toolchain optimization may weigh no less than the model itself in local scenarios.

Quantization and MLX: Quantization compresses model weights from 16-bit floating point to lower precision such as 4-bit or 8-bit to reduce memory and disk usage, at the cost of slight precision loss. The 4-bit MLX checkpoint mentioned here is a model copy quantized to 4 bits. MLX is Apple's array computing and inference framework designed for its own chips (M-series unified memory), able to fully leverage the unified memory architecture; OMLX and Mirai Uzu are different inference frontends built on top of it, and differences in implementation details directly affect throughput.

Wired memory (resident memory): This refers to the portion that cannot be swapped out to disk and must occupy physical memory long-term, including model weights, KV cache, and runtime overhead. KV cache grows linearly with context length, and peaks rise noticeably during long conversations. A peak of 19.6–24.9GB on a 32GB machine means limited space remains for the system and other applications; when exceeded, the system will compress memory or swap pages, causing a sharp drop in speed.

GitHub Skills: Capability Rankings for AI Coding Agents

On GitHub, a starred project equals a bookmark. According to a November 18 count, three Skills aimed at AI coding agents were ranked by the number of stars their projects received; the count only includes stars still on the projects at that time, serving as a measure of attention—quality still needs separate evaluation.

We rank these three skills by how many stars their projects got on November 18th.

Ranked first is brainstorming in Obra/superpowers with 1,903 stars, up from 787 the previous day; second is TDD (test-driven development) in Matt Pocock's/skills with 6,401 stars, up from 871; third is OpenAI's gh-address-toggins, which uses the GitHub CLI with authenticated access to locate and categorize review comments on the current branch's PR and ask which ones to fix, with 28 stars, up from 27.

It stands for test-driven development.

The Workflow Thinking Behind Skills

The three skills point to three different modes of agent collaboration: brainstorming has the agent verify goals and compare approaches before coding; TDD has the agent write tests first and then make small code changes; and review-comment handling automates the triage step of PR feedback. The single-day jumps in star counts also show that these lightweight skills are in a phase of rapid discovery and spread—the rankings themselves fluctuate significantly, and their reference value should be judged alongside the actual quality of the specific projects.

Overall Observations

Today's developments span four layers—model capabilities, interaction methods, local deployment, and development workflows: Gemini 4 Neon bets on long-task scenarios with million-token context, OpenAI pushes voice input toward continuous recording, local LLM benchmark data reveals toolchain differences, and GitHub Skills reflect how coding agents are being subdivided into reusable capability units. For developers, what's worth tracking isn't a single version number, but how these changes alter how tasks are organized and how costs are structured.

One-line Summary

Gemini 4 Neon's paid API opens, OpenAI expands live dictation across platforms, and local LLM benchmarks plus GitHub Skills rankings round out today's LLM developments.

Paragraph Summary

The article rounds up four developments in the LLM space today. Google Gemini 4 Neon is fully available on the paid Gemini API, processing up to 1 million tokens per request at $3 per million input tokens and $15 per million output tokens, targeting long and complex tasks—though long context also brings trade-offs in cost and information density. OpenAI is rolling out live dictation covering iOS, Android, web, and desktop, and removing the 10-minute recording limit, moving voice input toward a continuous recording tool. Local LLM benchmarks show that on a 32GB machine, the same model can range from 60.8 tokens/s to 9.2 tokens/s depending on inference software, with peak resident memory approaching total capacity. GitHub Skills rankings by star count reflect how coding agents are being subdivided into reusable capability units, but rankings fluctuate widely and only represent attention.

#Gemini 4 Neon#paid API#long context#1 million tokens#OpenAI live dictation#local LLM#MLX quantization#GitHub Skills#AI coding agents#LLM pricing
Share:

Related articles