This is a composite case study drawn from our implementations. The company described is a plausible synthetic composite, not a specific named customer. The workflow patterns, data volumes, and outcome metrics reflect the range we observe across actual deployments. We are describing this as a composite case study rather than a named client case because that is what it is — and because Japanese enterprise customers universally prefer not to be identified in public vendor materials.
With that said: the HR knowledge agent problem we describe here is real, common, and consistently underestimated by DX teams who have not run into it before.
The Situation Before the Agent
Imagine a manufacturing company with approximately 400 employees across three sites — main plant in Aichi, regional office in Osaka, and a Tokyo headquarters. The HR department is a team of four people managing the full range of HR functions: payroll coordination, onboarding, benefits administration, leave management, and policy communications.
A recurring pattern in their workweek: employees contact the HR team with questions about company policies. Not complaints or escalations — simple lookup questions. "How many paid leave days can I carry over into next year?" "What is the procedure for requesting bereavement leave?" "Does the commuter allowance policy cover express train charges?" "I am working from home more than 60% of the time — do I need to notify HR?" Questions that have clear, documented answers in the company's employee handbook and policy manuals.
The HR team estimated that policy lookup questions consumed between 30% and 40% of their responding time each week. The questions themselves were not complex — the answers existed in documented form — but finding the right passage in a multi-document policy library, confirming it was the current version, and pasting a helpful response took time. Multiply that across dozens of questions per week and you have a significant portion of a four-person team's capacity being used to do something that, in principle, should not require human judgment at all.
The HR team had two related frustrations. First, employees often did not check the policy documents themselves because they found the PDF library hard to navigate — multiple versions of some documents, inconsistent naming conventions, search that returned entire documents rather than the relevant paragraph. Second, when the HR team answered by email, the same question would often come from a different employee a few weeks later, because there was no central searchable record of previous answers.
What Was Built and How
The deployment started with a scoping conversation that surfaced something important: the HR team did not want the agent to replace their judgment on ambiguous cases. They were entirely comfortable with the agent handling the clear-answer policy lookup tier. They were not comfortable with the agent attempting to adjudicate borderline situations — for example, a leave request that might qualify under two different policy provisions — where HR professional judgment was actually needed.
This boundary shaped the agent design significantly. We built a hard escalation rule into the agent flow: any query that matches a list of escalation trigger keywords (disputes, exceptions, disciplinary matters, salary corrections) is immediately routed to HR with the original query text, rather than the agent attempting an answer. This is not a limitation of the agent — it is a deliberate design choice to keep the agent operating only in the space where confident, accurate answers are possible.
The knowledge base was assembled from four sources: the employee handbook (most recent version, PDF, approximately 80 pages), the separate leave policy document (updated 14 months prior), the commuter and expense allowance policy (two active versions because the policy had changed mid-year and some employees were still covered under the old version based on their employment date), and a company-authored FAQ document that the HR team had built informally over several years of answering common questions.
The two-version leave policy situation was the most technically interesting part of the setup. Rather than attempting to have the agent reason about which version applied to which employee, we built a pre-query disambiguation step: the agent asks for the employee's hire date before answering any policy question that could be version-dependent, then routes the query to the correct document version based on that input. It is a simple pattern but it prevents the category of confident-wrong-answer that occurs when a model synthesizes from both versions without being aware of the applicability distinction.
Deployment was to the company's existing Microsoft Teams environment, using the Askhub Teams connector. No new employee-facing application was introduced — the agent appears as a Teams bot that employees can message directly or add to department channels.
The First Month in Production
The first week had a pattern we have seen in other deployments: a surge of exploratory queries as employees tested the agent's capabilities, including some queries that were clearly tests rather than genuine requests ("what is the meaning of life?", "can you write me a poem?"). The agent's response to out-of-scope queries — "I am here to help with questions about company policies and HR procedures; for anything else, please contact HR directly" — was well-received. Employees appreciated that the agent was honest about its scope rather than attempting to be helpful in ways it was not designed for.
By the second week, query patterns had settled into genuine policy lookup use. Common query types: leave carryover rules (consistently the highest volume), commuter allowance calculations for mixed-mode commuters (trains plus bicycle for the last mile), work-from-home notification requirements, and procedural questions about submitting specific HR forms.
The HR team began tracking how many queries reached them via escalation versus how many were handled by the agent. In the first month, the agent handled approximately 70% of inbound policy queries without escalation. The HR team's assessment was that this matched their expectations — the remaining 30% escalated were the cases that genuinely needed human judgment, not cases where the agent failed on answerable questions.
One unexpected benefit: the agent's query log became a useful source of information for the HR team about what employees were actually confused about. High-volume query types that the HR team had not previously tracked were now visible. In one case, the log surfaced that employees at the Osaka site were querying a specific expense policy significantly more often than employees at other sites — which turned out to reflect a difference in how the site manager had been communicating the policy locally. The HR team used this to trigger a policy clarification communication to that site.
What Did Not Work Perfectly
Three things required adjustment after launch. First, the chunking of the employee handbook produced some responses that quoted a single paragraph accurately but omitted adjacent context that was necessary to understand the answer fully. For example, a question about the annual leave accrual rate returned the correct rate, but the correct rate had an exception for employees in their first year of employment that was in the following paragraph — not the retrieved chunk. This was fixed by widening the context window in retrieval and adding a post-retrieval check for "exception" and "ただし" (a Japanese conjunction typically preceding exceptions or qualifications) in neighboring chunks.
Second, some employees initially phrased questions in casual Japanese rather than the semi-formal register the agent was tuned for. The agent's responses stayed formal, which felt slightly mismatched to some users. We adjusted the system prompt to detect casual register in the query and slightly soften the response formality while remaining appropriate for a business context.
Third: the FAQ document the HR team had built was internally inconsistent in a few places — the same question had slightly different answers in different sections, reflecting different update cycles. Rather than letting the agent synthesize across these inconsistencies (which would produce confident-sounding wrong answers), we worked with the HR team to spend two hours resolving the inconsistencies in the source document. This is an important principle: the agent's accuracy ceiling is the consistency of its source material, and no retrieval or reasoning sophistication can substitute for coherent source documents.
Six Months Later
At the six-month mark, the HR team's self-reported estimate was that policy lookup queries occupying uninterrupted HR staff time had dropped by approximately 60% from pre-deployment levels. The remaining HR time spent on policy questions went to the genuinely complex cases — situations requiring judgment, exception decisions, and sensitive matters that should always be handled by a human.
The agent's knowledge base has been updated three times since launch: once when the annual leave policy was revised, once when a new work-from-home notification requirement was added, and once when the commuter allowance policy was simplified to a single version. Each update took approximately 30 minutes of configuration time inside the Askhub platform — the HR team's designated administrator handles these updates without external support.
The question we get asked most often about this case: "what about employees who prefer to ask HR directly rather than using the agent?" They continue to do so. The agent did not replace the HR team's availability for questions — it reduced the volume of lookup queries that consumed the team's time, freeing more of their attention for the people who needed genuine HR engagement. That is the scope it was designed for, and it is the scope where it performed well.
