1<!doctype html><html lang=en-us><head><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1"><title>Improving German Search: Splitting The Compound Word - Daksh Pareek</title> 2<meta property="og:title" content="Improving German Search: Splitting The Compound Word - Daksh Pareek"><meta name=twitter:card content="summary"><meta property="description" content="Our full-text search failed on German compound words like HeiÃluftballonfahrt. Here’s how splitting compounds fixed it and lifted top-5 accuracy from 87% to 98% for compound queries."><meta property="og:description" content="Our full-text search failed on German compound words like HeiÃluftballonfahrt. Here’s how splitting compounds fixed it and lifted top-5 accuracy from 87% to 98% for compound queries."><link rel=stylesheet href=//cdn.jsdelivr.net/gh/highlightjs/[email protected]/build/styles/github.min.css><link rel=stylesheet href=../../../../../css/style.css><link rel=stylesheet href=../../../../../css/fonts.css><link rel=canonical href=https://daksh.be/blog/2026/09/25/improving-german-search-splitting-the-compound-word/><meta name=description content="Our full-text search failed on German compound words like HeiÃluftballonfahrt. Here's how splitting compounds fixed it and lifted top-5 accuracy from 87% to 98% for compound queries.">
2<script type=application/ld+json>{"@context":"https://schema.org","@type":"Article","headline":"\"Improving German Search: Splitting The Compound Word\"","description":"\"Our full-text search failed on German compound words like HeiÃluftballonfahrt. Here's how splitting compounds fixed it and lifted top-5 accuracy from 87% to 98% for compound queries.\"","datePublished":"2026-09-25T00:00:00Z","author":{"@type":"Person","name":"Daksh Pareek"},"mainEntityOfPage":"\"https://daksh.be/blog/2026/09/25/improving-german-search-splitting-the-compound-word/\""}</script>
2<link rel=stylesheet href=../../../../../css/custom.css></head><body class=blog><header class=masthead><h1><a href=../../../../../>Daksh Pareek</a></h1><p class=tagline>Welcome to my personal portfolio website, showcasing my projects and blogs.</p><nav class=menu><input id=menu-check type=checkbox hidden> 3<label id=menu-label for=menu-check class=unselectable hidden><span class="icon close-icon">â</span> 4<span class="icon open-icon">â°</span> 5<span class=text>Menu</span></label><ul><li><a href=../../../../../>Home</a></li><li><a href=../../../../../projects/>Projects</a></li><li><a href=../../../../../blog/>Blog</a></li><li><a href=../../../../../blog/2024/12/18/articles-i-liked/>Recommendations</a></li></ul></nav></header><article class=main><header class=title><h1>Improving German Search: Splitting The Compound Word</h1><h3>2026-09-25</h3><hr></header><h1 id=improving-german-search---part-1-splitting-the-compound-word>Improving German Search - Part 1: Splitting The Compound Word</h1><p>We have multi locale data with us and we use full text search(FTS) to query data and return results to user.</p><h2 id=problem>Problem</h2><p>English user who comes to our gifting platform searches for <code>Hot air balloon ride</code> and FTS queries as <code>hot + air + balloon + ride</code>. Products title stored in database also uses same seperate words like <code>Hot Air Balloon Ride</code>. So, FTS works well for English users.</p><p>But German joins the same concept into one word and it can join it in different ways. For example, the German translation of <code>Hot air balloon ride</code> is <code>HeiÃluftballonfahrt</code>. If a German user searches for <code>HeiÃluftballonfahrt</code>, the FTS query will be <code>heiÃluftballonfahrt</code>, which may not match the product title stored in the database as <code>HeiÃluftballon Fahrt</code> or <code>HeiÃluft Ballon Fahrt</code>.</p><h2 id=solution>Solution</h2><p>As we have seen <code>HeiÃluftballonfahrt</code> is actually a compound word made up of three words: <code>HeiÃluft</code>, <code>Ballon</code>, and <code>Fahrt</code>. So, we can split the compound word into its constituent words and then perform FTS query on those words. 6Using this we would able to match the product title stored in the database as <code>HeiÃluftballon Fahrt</code> or <code>HeiÃluft Ballon Fahrt</code> which means same as <code>HeiÃluftballonfahrt</code>.</p><h2 id=some-german-language-points-to-know>Some German language points to know</h2><ol><li><p>German joins words into compounds (Komposita)</p><ul><li>Example: <code>HeiÃluftballonfahrt</code> is “hot air balloon ride” in one word.</li><li>Why it matters: FTS compares whole words and it cannot find a word inside a longer word.</li></ul></li><li><p>The last part carries the main meaning (the head) 7In a compound, the last part names the thing and the parts before it describes it.</p><ul><li>Example: <code>Paarmassage</code> is a type of massage and <code>Ballonfahrt</code> is a type of ride (“Fahrt”).</li><li>Why it matters: The important word is at the end. A prefix match (ballon:*) finds “Ballonfahrt” but it cannot find “HeiÃluftballonfahrt”.</li></ul></li><li><p>Compounds can contain compounds</p><ul><li>Example: “HeiÃluftballonfahrt” is made up of “HeiÃluft” and “Ballonfahrt”, and “Ballonfahrt” is made up of “Ballon” and “Fahrt”.</li><li>Why it matters: A compound can be made up of other compounds, so we need to split recursively.</li></ul></li><li><p>Joining letters (Fugenelemente) 8German sometimes puts a few letters between the parts of a compound: s, es, er, en, e, n, and so on.</p><ul><li>Example: Eule + n + Motiv = Eulenmotiv. Tag + es + Zeit = Tageszeit.</li><li>Why it matters: The splitter must skip these letters otherwise “eule | nmotiv” fails.</li></ul></li><li><p>One concept, many spellings 9The same product can use a compound word, a hyphen or a phrase.</p><ul><li>Example: “Ballonfahrt”, “HeiÃluftballonfahrt”, “Fahrt im HeiÃluftballon” or “Dinnershow” and “Dinner-Show”.</li><li>Why it matters: A visitor and a product title often use different forms. Splitting brings them to the same parts.</li></ul></li><li><p>Some words only look like compounds 10A word can have parts that are real words but it is not a compound of them.</p><ul><li>City names: “Regensburg” is not “Regen” (rain) + “Burg” (castle). “Düsseldorf” is not “Düssel” + “Dorf”. “Frankfurt” is not “Frank” + “Furt”.</li><li>Other words: “Wahrzeichen” is not “wahr” (true) + “Zeichen” (sign).</li><li>English words: “Workshop” is not “work” + “shop”.</li><li>Why it matters: A computer cannot tell these words from real compounds and that’s why we have to build compound-stopwords.json</li></ul></li></ol><h2 id=implementation-flow>Implementation flow</h2><h3 id=step-0-fn-tokenizetext>Step 0: Fn tokenize(text)</h3><ul><li>toLocaleLowerCase(‘de’): first we will lowercase with the German language rules.</li><li>then any character that is not a letter ends a word like space, hyphens, digits, brackets, &, - all separate words and we would not include those characters.</li><li>return the list of words.</li></ul><h3 id=step-1-iterating-over-text>Step 1: Iterating over text</h3><ul><li>calling tokenize(text) to get the list of words.</li><li>we will do count of each word and store it in a map.</li><li>after getting that map, we will iterate over that word and if its short or its frequency is less than 2, we will skip it. We will only consider words which are 4 or more letters and have frequency 2 or more times.</li></ul><h3 id=step-2-creating-stopwords>Step 2: Creating STOPWORDS</h3><ul><li>we will iterate over all the cities present in database.</li><li>for each city we will be calling tokenize(city) to get the list of words.</li><li>if word is 8 or more letters characters, we will add it to the STOPWORDS list. This is because we don’t want to split city names like “Regensburg” or “Düsseldorf” into parts.</li><li>this will give us stopwords json file which we will use in the next step to split the compound words.<
10/li></ul><h3 id=step-3-building-compound-splitter-function>Step 3: Building Compound Splitter function</h3><pre><code class=language-javascript>splitter = new CompoundSplitter(lexicon, stopwords) 11splitter.split("HeiÃluftballonfahrt") // returns ["HeiÃluft", "Ballon", "Fahrt"] 12</code></pre><ul><li>CompoundSplitter(lexicon, stopwords): The splitter loads the word list and the stop list one time when service starts.</li><li>split(word) returns the parts of one word, or nothing. It uses these rules:<ul><li>If the word is in the stop list, it returns nothing. “Regensburg” stays whole.</li><li>If the word has fewer than 8 letters, it returns nothing. A shorter word is not long enough to have two parts of 4 letters.</li><li>It tries each cut position. The left part must have 4 or more letters, and it must be in the word list.</li><li>After the left part, it skips a joining letter if there is one: s, es, n, en or e. “eule | n | motiv” becomes “eule” + “motiv”.</li><li>The right part must have 4 or more letters. If the right part is in the word list, the cut is valid.</li><li>If the right part is not in the word list, the splitter calls split on the right part again (recursion). “sonne | n | untergang” becomes “sonne” + “unter” + “gang”.</li><li>If more than one cut is valid, it picks the best cut: 13i. It picks the cut with the fewest parts. 14ii. If two cuts have the same number of parts, it picks the cut whose least common part occurs more often.</li><li>It keeps each result in a cache, so the same word is split only one time.</li></ul></li></ul><h4 id=three-examples>Three examples</h4><pre><code>heiÃluftballonfahrt 15 heiÃluft | ballonfahrt 2 parts, least common part 3 16 heiÃluftballon | fahrt 2 parts, least common part 26 â 17 â heiÃluftballon + fahrt 18 19eulenmotiv 20 eule | nmotiv "nmotiv" not in list â 21 eule | n | motiv joining letter "n" â 22 â eule + motiv 23 24weinprobe 25 wein | probe "probe" not in list â 26 â no parts (a synonym handles this word) 27</code></pre><h3 id=step-4-where-the-splitter-runs>Step 4: Where the splitter runs</h3><p>The splitter runs at two different times:</p><ul><li>When a product is saved: SearchVectorService uses allParts(word) on products.</li><li>When a visitor searches (the query side): The query builder uses only split(word), which is the first level.</li></ul><h3 id=step-5-building-the-query>Step 5: Building the query</h3><ul><li>Compound words: Each German query word with parts become <code>(word | (part1 & part2))</code>.</li><li>Why AND and OR: All query words stay required, which is the same as today. The OR is only between two spellings of the same word, so it never makes the query wider.</li></ul><h2 id=results-that-we-see>Results that we see</h2><ul><li>Our top 5 result accuracy went from 84% to 89%. This is the percentage of correct products among the first 5 results</li><li><strong>Top-5 accuracy for queries with compound words:</strong> 87% â 98%.</li><li>No query got worse.</li><li>3 of the 30 real queries now show the correct product first. The other 27 did not change.</li></ul><h2 id=what-splitting-does-not-solve>What splitting does not solve</h2><ul><li>Words with no shared part such as “Weinprobe” and “Weinverkostung” need synonyms.</li><li>Spelling mistakes still needs to go with fuzzy match.</li><li>The splitter can take a real letter as a joining letter. The fix is if we find that we add that to the stop list.</li></ul><footer><nav class=post-nav><span class=nav-prev>← <a href=../../../../../blog/2026/05/03/why-my-github-actions-cache-was-always-cold/>Why My GitHub Actions Cache Was Always Cold</a></span> 28<span class=nav-next></span></nav>
28<script src=//cdn.jsdelivr.net/npm/@xiee/utils/js/math-code.js defer></script>
28<script src="//mathjax.rstudio.com/latest/MathJax.js?config=TeX-MML-AM_CHTML" defer></script>
28<script src=//cdn.jsdelivr.net/npm/@xiee/utils/js/center-img.js defer></script>
28<script src=//cdn.jsdelivr.net/combine/gh/highlightjs/[email protected]/build/highlight.min.js,npm/@xiee/utils/js/load-highlight.js defer></script>
28<hr><div class=copyright>© <a href=https://daksh.be>Daksh Pareek</a> 2018-2026 | <a href=https://github.com/dakshpareek>Github</a></div></footer></article>
vendor: 63 bytes, line 28
28<script async src="https://www.googletagmanager.com/gtag/js?id=
28G-LJV7YZGBBB
vendor: 11 bytes, line 28
28"></script>
28<script>var dnt,doNotTrack=!1;if(!1&&(dnt=navigator.doNotTrack||window.doNotTrack||navigator.msDoNotTrack,doNotTrack=dnt=="1"||dnt=="yes"),!doNotTrack){window.dataLayer=window.dataLayer||[];function gtag(){dataLayer.push(arguments)}gtag("js",new Date),gtag("config","G-LJV7YZGBBB")}</script>
28<script type="module" src="https://static.cloudflareinsights.com/beacon.min.js/v31edd6df95cf4e85bb4c19e7a9bdbcba1788362987495" integrity="sha512-iIg7k2xntmwu6/uSb5tpc/hySgZc4eoL31yB29W6tJFo2akwjPWcEqnCEdJvGexCL0KEQwVYv5BlowfhVz26hg==" data-cf-beacon='{"version":"2024.11.0","token":"5da309a120174b71a37175cbdf9514de","r":1,"spa":2}' crossorigin="anonymous"></script>
28 29</body></html>
Line numbers count LF bytes from the start of the resource, as the search results do. Vendor segments are library code the classifier recognised; they are stored but not indexed. Bytes are shown as Latin1 characters, one per byte.