Internal search tuning means adjusting how your intranet, wiki or knowledge base finds and ranks documents so employees reach the right answer on the first or second try. Most of the gain does not come from ranking settings. It comes from fixing vocabulary, content structure and the queries you test against. This guide walks through a tuning process in the order we would run it, and explains why the documents themselves are usually the real problem.
TL;DR
- Diagnose before you tune. Most "search problems" are naming, metadata and content-structure problems that no boost setting will solve.
- Treat failed and zero-result searches as content requests. They tell you what to write next.
- Build a golden set of diverse queries and measure every change against it, so you do not tune for the loudest executive's pet query.
- Map how people search to how documents are filed. Synonyms and curated results beat clever algorithm tweaks.
- Some knowledge never reaches a document. If it lives in people's heads, no tuning can surface it.
Table of Contents
- What internal search tuning actually is
- Why internal search is not web search
- Step 1: Read your search logs like a demand forecast
- Step 2: Close the vocabulary gap
- Step 3: Check structure before touching weights
- Step 4: Build a golden set and measure
- Step 5: Unify the experience
- Perspective: the answer is not always in a document
- Where Kept fits
- A 30-day tuning plan
- Common mistakes
- FAQ
- Sources
What internal search tuning actually is
Tuning is an iterative loop: observe how people search, change something, measure whether the right documents moved closer to the top, and repeat. Coveo's overview of search tuning describes it as adjusting parameter settings to give each user the best experience. That is accurate as far as it goes, but for internal search the parameters are rarely the bottleneck.
The Nielsen Norman Group has long observed that organizations tend to treat intranet search as an IT configuration task. In practice, a search complaint is often a content naming and metadata complaint in disguise. If the policy is called "Remuneration Framework v3 FINAL" and everyone types "pay bands," the engine is working and the content is failing.
That framing changes the work. Instead of asking "which field should we boost?", you ask:
- What are people typing?
- What do they expect to find?
- Does a document exist that answers it, and is it named the way they think?
Why internal search is not web search
Public web search is built around precision. There are millions of plausible pages, so the engine's job is to put a few excellent ones on top. Internal search is different. NN/g notes that on a single website or intranet, recall can be quite important: if you search for every past project of a certain type, missing one is a real failure.
That has practical consequences for how you judge success:
| Web search | Internal search | |
|---|---|---|
| Main risk | Too many results | Missing a known, critical document |
| Typical goal | Best single answer | Best answer plus complete coverage when needed |
| Content | Billions of pages, competing for rank | A few thousand documents, often poorly titled |
| Query volume | Huge, so patterns are statistically clean | Small, so a handful of queries can skew your view |
| Vocabulary | Public, SEO-shaped | Internal jargon, acronyms, team slang |
The small query volume matters most. With a few hundred searches a day, one vocal manager's complaint can feel like a trend. It is not, and we will come back to that.
Step 1: Read your search logs like a demand forecast
Your logs are the cheapest research you will ever do. Earley Information Science makes the point that search logs reveal what content your organization is missing, and that zero-result queries are best read as explicit requests rather than system errors.
Pull three lists for the last 60 to 90 days:
- Top queries. The head of the distribution. Check the first page of results for each by hand.
- Zero-result queries. Cluster them by meaning, not exact text, so "expense claim," "expenses form" and "reimbursement" land in one bucket.
- Reformulated queries. Sessions where someone searched, changed the words and searched again. These show where the first attempt failed and which words finally worked.
The third list is the most useful and the least used. The second query is often the label the document should have carried.
No team can close every gap at once. Earley suggests weighing gaps by volume, by how many people are affected, and by the operational impact of not finding the answer. We would add a fourth lens: how hard the answer is to write down. A missing holiday policy is an afternoon. A missing explanation of how a senior engineer decides when to roll back a release is a project.
If you want a practical starting point on measurement, our piece on the first 30 failed searches to review lays out a way to begin with a small, honest sample rather than a dashboard.
One caution on privacy. Search logs reveal what individuals are worried about, such as layoffs, medical leave or a grievance procedure. Aggregate before you review. Some tools only report a gap once at least two different users have hit it; Gfacility's knowledge gaps documentation describes that kind of privacy threshold, and it is a sensible default to copy in your own reporting.
Step 2: Close the vocabulary gap
The quiet killer of enterprise search is vocabulary mismatch. People rarely search with the formal taxonomy used to file documents. They search with the words they use in meetings. NN/g's guidance on intranet search puts the fix in the same place: map the terms people type to the titles documents carry, rather than just tuning ranking.
Concrete moves, roughly in order of effort:
- Add synonyms and alternative terms. "PTO," "vacation," "time off" and "annual leave" should return the same policy. Include acronyms and the old names of renamed teams and tools.
- Rewrite titles for the searcher. Replace "Procedure 4.2.1" with a title that states the task: "How to request a purchase order over $5,000."
- Curate results for common queries. NN/g recommends manually improving results for frequent queries. For your top 50 queries, pin the best document. It is unglamorous and it works.
- Offer suggestions as people type. Curated suggestions steer people toward terms that actually exist in your content.
- Fix the search box itself. Use a visible box rather than only a magnifying-glass icon, as NN/g's intranet search research advises.
None of this touches a ranking formula. All of it changes what people find.
Step 3: Check structure before touching weights
If you use semantic or vector search, the same principle holds. Teams see mediocre similarity scores and start tweaking weights. Often the cause is that documents are long, mixed-topic pages that were never written to be retrieved in pieces.
Before adjusting any setting, ask:
- Does each page answer one question, or six?
- Is the answer near the top, or buried under background?
- Are there three near-duplicate versions with different dates and no clear owner?
- Does the title say what the page is for?
Our five-layer knowledge base architecture goes deeper on how to organize content so that both people and AI retrieval can use it. If your content sits in a loose wiki, the comparison of a wiki and a knowledge base explains why notes alone rarely become trusted answers.
Step 4: Build a golden set and measure
Tuning without measurement is mood. The antidote is a fixed list of test queries with known good answers, often called a golden set. Each time you change a synonym list, a title or a setting, you rerun the set and compare.
How to build one in an afternoon:
- Sample 50 to 100 real queries from your logs. Include head queries, mid-frequency queries and long-tail oddities.
- Add a handful of known-item searches, such as the document everyone must be able to find.
- For each, record which document or documents should come back, and mark them as essential or acceptable.
- Choose a small set of metrics and keep them stable.
For metrics, a common combination is Precision@1 and Precision@3 for the top of the list, NDCG@10 for position-weighted quality, and Recall@50 to confirm the engine is not blind to certain topics. The research on enterprise search above explains why recall belongs in the set. A team describing a continuous evaluation loop for retrieval on DEV Community uses a fixed test set plus a regression gate, so that a change which helps one area cannot quietly break another.
The pet query trap
The most common way tuning goes wrong is optimizing for pet queries: the handful of searches run by executives or the loudest teams. You fix their complaint, and ten long-tail queries from everyone else get slightly worse. Search relevance engineers warn about exactly this. The defense is the golden set. When someone senior sends you a screenshot of a bad result, add that query to the set, fix it, and verify that nothing else moved. Their query gets treated fairly, and it does not become the whole program.
Step 5: Unify the experience
Employees do not think in systems. They want one place to look for a document, a person or a tool. NN/g's research found that good intranet search bridges content, people and tools in one interface. That is a hard goal if answers sit in a shared drive, a ticketing system, a chat tool and a wiki.
If you are replacing an older search layer, note that the old off-the-shelf options are gone: Google Search Appliance and Google Site Search were discontinued, so organizations now build on their own indexing infrastructure. Whatever you choose, plan connectors to the systems people already use, and decide which source wins when two disagree. Our write-up on the systems you already use describes how we approach that.
Perspective: the answer is not always in a document
We spend our days on documentation, so here is a view some search specialists find uncomfortable. You can tune a search engine until your golden set scores beautifully and people will still fail to find the thing that matters. The reason is that it was never written down.
A documented procedure holds the rule. The person who runs it holds the exceptions, the thresholds they actually use, and the reasons behind both. "Approve under $10,000 unless the vendor is new" is in the SOP. "I also check whether they've changed banking details this quarter, because that's how the last fraud attempt looked" is in someone's head. Search logs will show people asking for it, zero results, and your team will conclude the engine is weak. The engine is fine. The knowledge is tribal.
This is why we think zero-result clusters should feed two queues. One is content that exists in people's heads and needs to be captured. The other is content that exists but is badly named. Only the second is a tuning job.
Capturing the first kind is closer to an oral-history interview than to a writing assignment. People are poor at producing procedures from a blank page and good at explaining their work when someone asks the right follow-up question. That is the principle behind how we built Kept: guided conversations that record the exceptions and reasoning in the expert's own words, then structure them into something searchable and citable. Our guide to process discovery methods covers what logs and observation miss, and why interviews fill the gap.
Tuning tells you where the holes are. It cannot fill them.
— Anthony
Where Kept fits
If your failed-search list keeps pointing to expertise that lives with a few people, Kept for business is built for that case. Teams document processes through conversation, keep context and reasoning together with the steps, and search with answers that cite their sources. Workspaces are opt-in, with controls over access and deletion. You can see the available plans, kept for you and kept for business, on the pricing page.
For teams whose gap is onboarding, where new hires search for answers nobody wrote down, see how conversations become searchable onboarding docs. For the knowledge a departing specialist takes with them, the expert handoffs use case is the closest match.
A 30-day tuning plan
Week 1: observe. Export 60 to 90 days of queries. Cluster zero-result and reformulated searches. Note which clusters point to missing content and which point to naming.
Week 2: baseline. Build the golden set of 50 to 100 queries. Score it. Do not change anything yet. You need a number to beat.
Week 3: cheap fixes. Add synonyms, rewrite the worst titles, pin results for the top 50 queries, and fix duplicates. Rescore after each batch so you know which change paid off.
Week 4: fill gaps. Rank the missing-content clusters by volume, impact and difficulty. Assign owners. For the ones that depend on a person's judgment, schedule an interview rather than asking for a document.
Then repeat monthly. If you run a support team, our walkthrough on building an AI knowledge base shows how this loop looks with a defined queue of tickets, and our pilot-first view of knowledge management explains how to tie the work to outcomes leadership cares about.
Common mistakes
- Tuning on anecdotes. One screenshot is not a trend.
- Boosting recency or popularity blindly. A heavily clicked old policy can bury the current one.
- Ignoring the long tail. Most queries are rare. Together they are most of the demand.
- Treating the project as finished. Content, teams and vocabulary change. Rerun the golden set on a schedule.
- Hiding the box. If people cannot see search, they will not use it, and your logs will understate demand.
FAQ
What is the difference between search tuning and search relevance tuning?
In practice they describe the same work. Relevance tuning stresses the goal, which is getting the most useful documents ranked highest. Search tuning is the broader habit of adjusting settings, synonyms, content and result curation to reach that goal.
How do I improve intranet search without replacing the engine?
Start with the logs. Add synonyms for the terms people actually type, rewrite unclear titles, pin the best result for your most common queries, and remove duplicate documents. These changes usually deliver more than swapping engines, because most failures come from vocabulary and content rather than ranking.
What are knowledge base search best practices?
Write one page per question, put the answer first, and title pages the way people ask. Review failed and zero-result searches on a schedule, and measure changes against a fixed set of test queries. Give every page an owner so outdated versions do not outrank current ones.
How many test queries do I need in a golden set?
Fifty to a hundred is enough to begin. Draw them from real logs and include common, mid-frequency and rare searches. Grow the set whenever someone reports a bad result, so each complaint becomes a permanent test.
What if the answer people search for was never written down?
Then the fix is capture, not tuning. Interview the person who knows, record the exceptions and reasoning as well as the steps, and publish the result under a title people would actually search for.
Sources
- Intranet-Search Essentials, Nielsen Norman Group
- 7 Ways to Improve Your Website's or Intranet's Built-In Search Engine, Nielsen Norman Group
- Your Search Logs Already Know What Your GenAI Is Missing, Earley Information Science
- Knowledge gaps, Gfacility
- Search Tuning 101, Coveo
- Site search after Google, Econsultancy
- Building a continuous evaluation loop for production RAG, DEV Community