Stop Words
In computing, stop words are words which are filtered out before or after processing of natural language data (text). Though “stop words” usually refers to the most common words in a language, there is no single universal list of stop words used by all natural language processing tools, and indeed not all tools even use such a list. Some tools specifically avoid removing these stop words to support phrase search.
Any group of words can be chosen as the stop words for a given purpose. For some search engines, these are some of the most common, short function words, such as the, is, at, which, and on. In this case, stop words can cause problems when searching for phrases that include them, particularly in names such as “The Who”, “The The”, or “Take That”. Other search engines remove some of the most common words—including lexical words, such as “want”—from a query in order to improve performance.
I was hitting Algolia’s search limits had to remove words I didn’t care about searching like “and”, “only”, “there”, or “I’ve” in an attempt to shrink the size of the posts on this site in the search index. There are quite a few lists on the internet and I ended up using a few of them for significant (> 65% average) size reductions in the search corpora.