Orca Security Builds Natural Language Security Queries on Amazon Bedrock, Cutting Latency by 75%

Cloud security platform Orca Security is working to let security analysts ask questions in everyday language instead of spending time learning complex query syntax. The company built a natural language query agent called Discovery AI on top of Amazon Web Services' Amazon Bedrock, automatically converting plain English questions into Sonar query statements. According to officially disclosed figures, compared with the initial baseline approach, this system reduced query costs by 90%, cut input token counts by 93%, and reduced latency by 75%.

The Translation Challenge: From Natural Language to Security Queries

Orca Security's core capabilities are built on the Sonar platform. Sonar is the company's agentless cloud security scanning engine, capable of discovering assets, configurations, and potential risks across cloud environments. Analysts use the Sonar query language to filter and locate specific issues — for example, finding exposed S3 buckets in a given region, or tracking down over-privileged identities with administrator permissions.

The problem is that while the Sonar query language is powerful, it has a steep learning curve. New members of a security team often need weeks to become proficient at writing queries, and senior analysts are reluctant to manually compose complex statements during urgent incident response. Discovery AI's goal is to eliminate this syntax barrier, letting users directly drive queries with questions like "which EC2 instances are exposed to the public internet and unpatched."

Amazon Bedrock serves as the underlying model service, providing access to a variety of foundation models. Orca Security chose Bedrock because it can invoke models from different vendors on a single platform while meeting enterprise requirements for data security and compliance. According to an AWS official blog post, Discovery AI's architecture went through multiple rounds of iteration before finally striking a balance between accuracy, cost, and response speed.

Agentless cloud security scanning is Orca Security's core technical approach. Traditional cloud security tools typically require installing a lightweight agent on every host, container, or virtual machine to collect runtime data, which carries high deployment and maintenance costs and is prone to coverage blind spots. The agentless approach instead calls the native APIs provided by cloud vendors — for example, AWS's resource configuration snapshots, EBS volume snapshots, and IAM policy read interfaces — to "bypass" the cloud platform side and obtain asset inventories, configuration states, and vulnerability information. The advantages of this approach are fast deployment, full coverage, and zero intrusion into business systems, making it especially suitable for multi-cloud and rapidly scaling environments. The trade-off is that there is some latency in data acquisition and a strong dependence on cloud vendor APIs. The Sonar platform is built precisely on this mechanism, which is why it can obtain large amounts of structured asset metadata — and this also explains why context compression becomes a key engineering step for Discovery AI.

How a Multi-Stage Architecture Compresses Input Tokens by 93%

Discovery AI does not throw the user's raw question directly at a large language model. According to descriptions from Orca Security's engineering team, the system uses a staged processing pipeline in which each stage passes only the necessary information to the next.

The first stage is intent recognition and query classification. The system first determines which type of security query the user's question belongs to — asset discovery, configuration checks, compliance auditing, or threat hunting. This step is handled by a smaller model or a rule engine, avoiding the use of a large model for every request.

The second stage is context compression. Security queries often need to reference large amounts of asset metadata, policy definitions, and historical records. If all this context is stuffed into the model's input window, token consumption balloons rapidly. Orca Security's approach is to extract only the fields and a small number of examples most relevant to the current query type, shrinking the input scale. This is also the main source of the 93% reduction in input tokens.

The third stage is query generation. The streamlined context filtered through the first two steps is sent to a model on Bedrock to generate the Sonar query statement. The generated result then undergoes syntax validation and permission checks to ensure it does not produce invalid or over-privileged queries.

The core logic of this multi-stage design is: not every step needs the most expensive model and the most complete context. After breaking the task down, the large model is only responsible for what it does best — translating structured intent into query syntax.

Token is the basic unit of measurement for how large language models process text, roughly understandable as a word or a fragment of a word. Model service billing, context window limits, and inference time are all typically measured in token counts. Input tokens refer to the tokens occupied by the user's prompt and any attached context, while output tokens are the portion occupied by the model's generated content. In security query scenarios, if the complete asset inventory, policy definitions, and historical records are stuffed into the prompt every time, input tokens quickly reach tens of thousands or more, driving up costs and slowing responses. Therefore, streamlining "the context fed to the model" to retain only fields strongly relevant to the current task is the first lever of cost optimization and the main source of the officially cited 93% reduction in input tokens.

The Engineering Trade-Offs Behind a 90% Cost Reduction

Orca Security's announced 90% query cost reduction needs to be understood in its specific context. The "initial baseline" mentioned officially likely refers to an early version, which very probably used an end-to-end single-model approach: the user's question plus all context sent to a large model in one go, directly outputting the query. This approach is simple to implement but suffers from persistently high token consumption and latency.

From an engineering perspective, the cost reduction mainly comes from three directions. First is the aforementioned input token compression, which directly reduces billable units. Second is model tiering, where intent recognition and validation use lightweight models or non-model solutions, with only the query generation step calling a large model. Third is caching and reuse, where similar security query requests can share intermediate results to avoid redundant computation.

The logic behind the 75% latency reduction is similar. Fewer input tokens inherently shortens inference time, and the fast responses of lightweight stages in the multi-stage architecture further improve the end-to-end experience. For security analysts, moving query responses from "waiting over ten seconds" to "returning within seconds" directly affects how often they use natural language queries.

However, these figures are all results from Orca Security's tests in its own scenarios. Different enterprises' cloud environment scale, query complexity, and concurrency will all affect real-world performance. Treating these metrics as a reference benchmark is more appropriate than treating them as a universal promise.

Model tiering, also called model routing or large-small model collaboration, refers to invoking models of different specifications within the same system according to task difficulty and importance. Lightweight models have fewer parameters, are fast at inference, and have low per-call cost, making them suitable for relatively deterministic judgments such as classification, intent recognition, and format validation; large models are powerful but expensive, suitable for tasks requiring complex reasoning and generation. Discovery AI assigns only the final translation from natural language to query statements to the large model on Bedrock, while the preceding intent recognition and subsequent syntax and permission validation are handled by lighter solutions. This division of labor concentrates expensive compute on the step that truly creates value, and is an important structural reason for the 90% cost reduction and 75% latency decrease.

How Far Can Natural Language Security Queries Go

Discovery AI represents a broader trend: security tools are shifting from "written by experts for experts" to "natural language driven." Platforms like Splunk and Datadog are also exploring similar capabilities, letting operations and security staff complete queries and analysis through conversation.

The challenge on this path is not whether the model can generate syntactically correct queries, but whether it can understand the implicit intent in security contexts. For example, "find all risky storage buckets" — what counts as risky? Is it public access, lack of encryption, or missing version control? Different teams may have completely different definitions. Discovery AI needs to map such vague expressions onto the enterprise's own security policies, which often requires additional configuration and tuning.

Another key issue is explainability. When the system generates a query and returns results, analysts need to know why the query was constructed the way it was. If the natural language layer completely obscures the underlying Sonar logic, security teams may distrust the results, especially when false positives or false negatives occur.

Orca Security's practice shows that Amazon Bedrock, as a model access layer, can support this kind of application, but the real value comes from the engineering architecture built around the model. Translating natural language into security queries is not a prompt engineering problem — it is a system design problem.

Explainability is a repeatedly emphasized requirement in security operations. When analysts see a list of assets or risks returned by a query, they need to judge whether the results are trustworthy: which conditions did this query actually filter on, why were certain assets included, and why were others excluded. If the natural language layer completely hides the underlying Sonar query logic, users can only passively accept the results, and once false positives or false negatives occur, it is hard to determine whether the cause is a semantic understanding deviation or a policy configuration issue. Feasible approaches include echoing back the generated query statement, annotating the adopted intent and context sources, and providing a one-click correction entry point that writes back to policy, making the natural language layer a transparent translator rather than a black box.

One-line Summary

Orca Security built Discovery AI on Bedrock, using a multi-stage architecture to cut natural language query costs by 90%.

Paragraph Summary

Orca Security launched Discovery AI based on Amazon Web Services' Amazon Bedrock, letting security analysts ask questions in natural language and having the system automatically convert them into Sonar query statements. To address the high token consumption and latency of the original end-to-end approach, the system uses a multi-stage pipeline: first intent recognition and query classification, then context compression retaining only the most relevant fields and examples, and finally invoking a large model to generate the Sonar query along with syntax and permission validation. Compared with the initial baseline, query costs dropped 90%, input tokens fell 93%, and latency decreased 75%. These optimizations mainly come from input token compression, model tiering, and caching and reuse. The article emphasizes that the real difficulty of natural language security queries lies not in syntax generation, but in understanding the fuzziness of security semantics and maintaining result explainability — essentially a system design problem rather than prompt engineering.