anypick: An Open-Source Library That Automatically Selects the Best LLM Model

The Real Challenge of LLM Model Selection

When building AI systems, developers face a persistent dilemma: the model marketplace evolves at breakneck speed. The most cost-effective model today may be surpassed by a newer, cheaper alternative within three months. Manually tracking prices, latency, and benchmark scores across hundreds of models on platforms like OpenRouter and Vercel AI SDK is practically impossible.

The deeper challenge is that model selection is fundamentally a multi-objective optimization problem — sometimes you need the lowest price, sometimes the fastest response, and sometimes specific capabilities (like function calling, long context, or multimodal support). Without a unified tool to handle these dimensions, developers are left relying on intuition and manual comparisons.

anypick was built to solve exactly this problem.

Breaking Down anypick's Core Features

anypick is an open-source library available in both Python and TypeScript, with a consistent API design across both languages. Its core functionality is organized into three layers:

Automatic Model Catalog Sync

The library supports pulling model catalogs from two primary data sources:

• OpenRouter: Aggregates hundreds of models from major providers including Anthropic, OpenAI, Google, and Meta, with real-time pricing and performance data
• Vercel AI SDK: A model catalog aimed at frontend and full-stack developers

Developers no longer need to manually maintain model lists — anypick keeps the data fresh automatically.

Multi-Dimensional Filter Pipeline

The heart of anypick's design is a pipeline pattern — you chain multiple filter conditions together to progressively narrow down candidate models. Supported filter dimensions include:

• Price: Filter by input/output token cost per unit, with budget caps
• Latency: Filter by response speed, ideal for latency-sensitive scenarios
• Benchmarks: Filter by scores on standard evaluations like MMLU and HumanEval
• Capabilities: Filter by model features such as function calling, vision input, and long-context support

The strength of this pipeline design lies in its composability — each filter step is independent, can be added or removed as needed, and can be reused across different projects.

Intelligent Best-Model Selection

After filtering, anypick can select the optimal model from the candidate list based on a specified criterion. For example:

• best_price: Selects the cheapest model that meets the conditions
• best_throughput: Selects the model with the highest throughput
• Custom scoring functions are also supported

Technical Design Highlights

Isomorphic API Across Python and TypeScript

The same API design is implemented in both languages, which is highly valuable for full-stack teams: backend Python services and frontend/Node.js services can use identical model selection logic, eliminating behavioral inconsistencies caused by language differences. This design decision is particularly practical in real-world LLM engineering.

Designed for Long-Term Stability

The author has explicitly stated the library's design goal: to let LLM systems go without manual model replacements every three months. This is a pragmatic engineering objective. By decoupling model selection logic from business logic, when a model's price spikes or it gets deprecated, you simply adjust the filter conditions — the system automatically adapts to the next best option without touching core business code.

This approach mirrors the role of Dependency Injection in traditional software engineering: separating the volatile parts (specific models) from the stable parts (business logic).

Real-World Use Cases

Cost-Sensitive Batch Processing

Tasks like bulk document classification and data extraction are latency-tolerant but extremely cost-sensitive. anypick can automatically select the cheapest model on the market that meets capability requirements, updating automatically as market conditions change.

Real-Time Conversational Applications

User-facing interactions demand low latency. You can set a latency threshold as a hard constraint, then rank remaining models by price or quality.

Multi-Tenant SaaS Platforms

Different customers have different budgets and requirements. You can configure distinct filter pipelines per tenant to dynamically assign the most appropriate model, rather than serving all users with a single model.

A/B Testing and Model Evaluation

Before rolling out a new model, use anypick to quickly generate a filtered list of qualifying candidates for comparative experiments.

Current Limitations and Caveats

As an early-stage open-source project, anypick has a few points worth noting:

• Data freshness dependency: The update frequency of the catalog data directly impacts its value. If the model catalog can't sync in real time, filter results may be based on stale data
• Benchmark limitations: Standard benchmarks (like MMLU) don't always correlate strongly with specific business tasks — high benchmark scores don't guarantee good real-world task performance
• Dual-language maintenance overhead: Maintaining both Python and TypeScript implementations means every feature update requires parallel work in two codebases, which is a non-trivial burden for maintainers long-term
• Limited data source coverage: Currently only covers OpenRouter and Vercel, with no support for enterprise-common platforms like AWS Bedrock or Azure OpenAI

Conclusion

anypick fills a neglected gap in the LLM engineering toolchain — the automation and systematization of model selection. Its value doesn't lie in complex algorithms, but in abstracting a repeatedly occurring manual workflow into a programmable pipeline.

For developers building LLM applications that require long-term maintenance, this kind of tooling is worth paying attention to. Competition in the model marketplace will only intensify — today's optimal choice may be obsolete tomorrow. A model selection layer that automatically adapts to change is practical infrastructure for building sustainable AI systems.