warning - I'll be linking to some "big" sites with trackers galore, take appropriate precautions.

search engines don't search.

At least not how we might want them to or how we may have grown up expecting them to. Just type "anyone else notice search results are useless?" into your favorite mainstream search engine (Google, Bing, Yahoo, etc...) and you'll get a bunch of hits ranging from Reddit posts to serious articles outlining the failings of modern search engines.

The issues are generally the same - SEO spam, intrusive ads, companies filtering or redirecting results to their own services and affiliates. And it's all being accelerated by AI - AI generated images, AI generated site content, AI filtered and interpreted results.

And it isn't your imagination, consider this TechSpot article by Alfonso Maruccia titled "New study confirms the obvious, search results are only getting worse". It outlines the results of a German study that spent a year tracking product review search results from three popular search tools - Google, Bing, and DuckDuckGo. Suffice to say, the results are not encouraging:

Popular search engines are losing the cat-and-mouse game against SEO-oriented spam, the German researchers say, while the line between "benign" content and worthless link farms has become increasingly blurry. The situation will only worsen with generative AI algorithms, which can be (ab)used to instantly eject a lot of low-quality, fact-free content that would be perfect for SEO spam and SERP manipulation.

The research team focused on product review searches, and per the article Google responded to the study saying that it doesn't reflect the overall "quality and helpfulness" of search results.

I disagree.

Why? Because I suspect Google's definition of "quality and helpfulness" doesn't match mine, or likely, the German research team's. Alphabet Inc., Google's holding company, is currently the third largest tech company by revenue. And the main source of that revenue? Ads¹ - to the tune of $65.52 billion in Q4 2023. But not just ads, but behaviorally targeted ads facilitated by the sale of your data. I suspect they would define "quality and helpfulness" as whatever glad-handing needs to be done to their users in order to serve as much advertising and sponsored content as possible. Meaning as long as AI generated "content" and SEO spam are fooling users into clicking on results (behavior) and driving targeted ads (revenue), Google is incentivized to let it slide as much as they can get away with while working under the pretense of fighting it.

what's the solution to bad search results?

Well I don't really care, not as far as Google or other 'big search' is concerned. They say they are taking the issue seriously and are taking steps to fight against SEO Spam, content mills etc... Sure, maybe they are. But maybe there is no solution if your business model depends on selling data, targeted ads, AI, or a combination of the above.

If we start from the assumption that there is no solution for worsening search results from popular engines then a logical course of action is to implement workarounds. In my case, and I think increasingly the case for many search users, that means finding alternatives.

This blog post 'A look at search engines with their own indexes' by Rohan “Seirdy” Kumar is nice jumping off point that lists a number of search engines but also categorizes them based on how they source and rank their results.

My favorite alternate (especially for finding old stuff), Marginalia Search, reminds me strongly of web search in the early days. It's up to me to provide a well-structured search string², scan the results, comprehend them for relevance, click through, and read. Yet despite the extra work on my part to interpret the results, it's often faster than scrolling through several pages of ads and spam to pick out a few relevant links from a "traditional" search index. Marginalia's focus is on "small" websites, so don't expect links from big media and social platforms to dominate the results. Which is, of course, the point.

I also use wiby.me for some old fashioned web surfing. It's not an index assembled automatically by a crawler, but an old-school directory with sites submitted by users. So if you're looking for weird, old, and/or obscure, it's a good choice.

In most cases I find human-curated directories, web rings, and linkrolls the best way to casually explore and surf the "small" web, and alternate search indexes the best way to find specific info from the small web.

My current general search tool is Duck-Duck-Go. It serves better results than Google lately in my experience³. Per the DuckDuckGo Wikipedia entry they do take steps to exclude and downrank results from content mills and sites with "low journalistic standards." And as they are privacy-focused (they don't track and sell behavioral data) and their ad revenue is tied to general search results (not personally targeted ads) I tend to trust them more. But they use external indexes for their results, so they could be subject to AI poisoning of the well.

Rohan's list includes a number of other general search engines to try including Stract. Marked as his favorite general search engine, it's currently ad-free and open source. I've tried running a few searches on it and it seems like it works well. I may try it as my default search on one of my devices to see how it works as a daily driver.

I've been adding links to some of my favorite search tools and directories to the linkroll and I'll be adding more as I find them.

Happy surfing!