Building a Search Engine Part 2: Beating The Incumbents
You don't have to be Goliath. David can win this.
Last week, I covered how search engines work and what advantages they have. This week, I want to show you how you can beat the major players by building your own.
And it’s not that hard.
If you missed part 1 last week, you can find it here.
Advantage 1: They Are The Front Door To The Internet
This is probably the hardest advantage to overcome, because you are having to fight against a lot of conditioned behavior. I’m looking at different ways to make it easier, but your mission is simple: you have to convince people that, instead of typing something into their address bar where their default search engine lives, they need to start at a different site.
This means you have to have a lot of trust built up and you need to tell them a compelling story about why.
This is the main reason I’m foregoing a traditional landing page on Choose Your Algorithm. This is the current landing page:
Yep, that’s it. I don’t try to sell anyone anything or tell them why they should use it. Instead, I’m going to rely on the users to explain why visitors should use the site. For example, if you use the page for the Spark Podcast Network I mentioned last week, you’ll see this:
Here’s the default search page if you enter any other URL:
The whole point I’m focusing on in these earliest days: it’s not about me or what I offer. It’s about what the people I’m serving are offering.
I can’t compete on the mindshare or the level of thoroughness that a Google can provide. I don’t have the advantages of being built in to browsers.
So I’ve got to play a different game. People like rooting for David, so you’ve got to prove you’re different than Goliath.
Give people a reason to root for you.
Advantage 2: They Signal Intent
In this case, it’s all about the data that isn’t there. For Google or other large search engines, they are able to take a single query and combine it will millions of others in order to make determinations about what a given query is signaling.
In this case, when you are running a search engine for a given audience, you know more about who they are. Google has some ideas based on what their activity is, but you know them much better. You know what type of people are in your audience, what their problems are, the contexts they work in.
When you combine that with some search queries, you can make a ton of inferences about what they are looking for. In fact, you can probably help them figure out what they are actually looking for, instead of what they asked for.
As a creator, there have been so many times that I’ve searched for one thing when I actually wanted something completely different. But I didn’t know enough to look for the thing I actually wanted to search for.
How helpful would it be to know what your audience was searching for so you would know what content to create?
I can’t wait to have that info. I have all sorts of content that I want to create, but if I knew my audience wanted to know something, I could create a bunch of short-form content that answers their questions and expand into longer pieces.
Advantage 3: They Signal Trends
This is where you can get first-mover advantage pretty easily. For people to spot trends from Google queries and act on them, they need a high number of people to search for certain things over a relatively long time period. If you notice a spike in a smaller, more-targeted population, you can identify a trend much faster and react accordingly.
This is something I realized early on in my creative career, and I figured it out accidentally. Basically, I discovered Tik Tok due to the trend with sea shanties, and I wrote a blog on the fact that it showed me that the future is going to revolve around collaboration.
You can check out that blog here
But writing the blog wasn’t the insight. Suddenly getting a huge influx of traffic and having to investigate where it came from was. I was early to the trend of sea shanties and wrote that blog, which led to a bunch of traffic from Google picking it up a few days later. I was on the front page of search results for “tiktok sea shanties” and that ended up driving traffic.
It pays to be early.
You should have an idea of how you can use this to give you an advantage now. In the coming weeks, I’ll be open-sourcing a package that I’m building that will help you set this up for yourself, or you’ll be able to use Choose Your Algorithm to have a managed system that will help you do this.
The next step is to set up a dashboard for creators to see their search queries that are coming through. Starting out, I’ll just show raw queries, but as volume increases, I’ll work on building out visualizations and trends.
If you want to help support the Spark Podcast Network, you can use our search engine here.
Love this. Have some ideas and may run them by you... DM you soon. Thanks, keep up the great work