Deckhand
An AI IT help desk agent built by ShipStation Global on Amazon Bedrock AgentCore. It resides in Slack and supports knowledge-base question answering, recognition of error screenshots and automatic creation of Jira tickets.
Timeline (last 90 days)
The ShipStation Global engineering team built an AI agent named Deckhand based on Amazon Bedrock AgentCore, residing in Slack and used as an IT help desk.
In the first three months after launch, Deckhand resolved 97.5% of questions in the Slack channel.
Deckhand's positive feedback rate was 96%.
Deckhand has three core capabilities: answering routine IT questions based on a curated knowledge base, reading screenshots of error messages, and automatically creating Jira tickets.
Deckhand adopts a RAG (retrieval-augmented generation) architecture: it first vectorizes the user's question, retrieves the most semantically relevant passages from the knowledge base, and then hands them to the large model to generate an answer.
When reading agent deployment data, one needs to pay attention to the criteria for judging the "resolution rate" (employee confirmation, the question no longer being followed up, or the agent marking it on its own); different criteria will significantly affect the figures.
Deckhand's 97.5% and 96% metrics may suffer from survivorship bias, and the first three months are often the honeymoon period when the knowledge base is maintained most diligently and the question types are most concentrated; these two numbers are better used as evidence that "the direction is right" rather than as promised values that can be extrapolated to a long-term stable state.
The time window for ShipStation Global to push Deckhand from project initiation to launch was significantly shortened thanks to the maturity of infrastructure such as AgentCore.
All Facts (8)
The ShipStation Global engineering team built an AI agent named Deckhand based on Amazon Bedrock AgentCore, residing in Slack and used as an IT help desk.
50%UnverifiedIn the first three months after launch, Deckhand resolved 97.5% of questions in the Slack channel.
50%UnverifiedDeckhand's positive feedback rate was 96%.
50%UnverifiedDeckhand has three core capabilities: answering routine IT questions based on a curated knowledge base, reading screenshots of error messages, and automatically creating Jira tickets.
50%UnverifiedDeckhand adopts a RAG (retrieval-augmented generation) architecture: it first vectorizes the user's question, retrieves the most semantically relevant passages from the knowledge base, and then hands them to the large model to generate an answer.
50%UnverifiedWhen reading agent deployment data, one needs to pay attention to the criteria for judging the "resolution rate" (employee confirmation, the question no longer being followed up, or the agent marking it on its own); different criteria will significantly affect the figures.
50%UnverifiedDeckhand's 97.5% and 96% metrics may suffer from survivorship bias, and the first three months are often the honeymoon period when the knowledge base is maintained most diligently and the question types are most concentrated; these two numbers are better used as evidence that "the direction is right" rather than as promised values that can be extrapolated to a long-term stable state.
50%UnverifiedThe time window for ShipStation Global to push Deckhand from project initiation to launch was significantly shortened thanks to the maturity of infrastructure such as AgentCore.
50%