Top Voices in Search Tech: Brian Pederson

Brian Pederson

The "Top Voices in Search-Tech" initiative is a carefully curated showcase of the most impactful and influential search-tech professionals from around the world that you can connect with and learn from.


About Brian

Brian is the founder of The Search Bar, the only recruitment firm focused on placing Search & AI professionals. With extensive experience in search implementation, Brian helps companies compose high-impact teams and navigate the evolving landscape of search technology, from traditional systems to LLM-enhanced search experiences. Before launching The Search Bar in 2024, Brian spent seven years at a managed-search provider, where he established the sales, marketing, customer success, and solutions engineering functions from scratch.

Brian has developed a proprietary assessment methodology, purpose-built for job placement in the Search domain. The hand-curated talent pool now includes approximately 1,000 vetted search professionals. As an industry insider, Brian works with organizations ranging from startups to Fortune 500 companies, recently securing partnerships with major retailers like Urban Outfitters. He also serves as an advisor at 7CTOs.com, the largest community of technology executives in North America.

Where to find Brian on the web:

Let’s start from the beginning — how did you get involved in the search tech industry?

I joined a boutique managed search provider back in 2016. They were managing Solr, Elasticsearch, and then over time, OpenSearch. I think I was employee number four or five, and I joined as the Head of Sales. I stayed there for seven and a half years.

Over time, that role evolved into an executive position, and I helped build out their sales, marketing, customer success, and solutions engineering functions from scratch. We went from working with companies on $50/month plans to serving large enterprises.

I spent most of my time with those larger enterprise teams—probably two to three dozen of them—and worked very closely with their search teams and leadership. I’d help them make changes to search, add new features and functionality, and solve challenges at scale.

Then in the last four or five years, we hit the AI tailwind. A lot of those conversations started shifting toward questions like: How do we incorporate semantic search or vectors into our search capabilities? How do we integrate LLMs into different stages of the pipeline? How do we approach RAG systems?

So that was my long introduction to search—seven and a half years deep with that company.

Tell us about your current role and what you’re working on these days.

A little over a year ago, I started a new company called The Search Bar. One of the recurring questions I’d get from clients over the years was, “Hey, we need to grow our search team—we’re having trouble finding good search engineers. Can you help us out?”

Internal HR and recruiting teams often weren’t having success, and they’d reach out to me for advice or referrals. The first chapter of my career—about eight years—was actually in technical recruiting, so I already understood that landscape well. Eventually, I saw enough of a pattern that it just made sense to try this as a new venture.

No one else seemed to be doing this in a focused way. Most recruiting and staffing companies were pretty generalized; they didn’t understand the niche, and they weren’t finding success. So this felt like a bit of a blue ocean opportunity—specializing in hiring specifically for search.

Could you describe a ‘favorite failure’—a setback that ultimately led to an important lesson or breakthrough in your work?

Yeah. One of the big challenges in my previous role—when you’re competing with a company that I won’t name, but is doing billions of dollars a year with a competing service—is: how do you compete with them as a bootstrapped company providing something very similar?

Probably one of the most valuable things I ever did in my career was to interview our customers—not all of them, but the key ones we wanted more of and where we were regularly competing with the big dogs. I used a “Jobs to Be Done” framework, which is more of a product development–style interview, and really uncovered the core motivations behind why they were switching—from in-house systems or from that large competitor.

We explored the different forces at play: what was pushing them to make a change, what pain they were experiencing, and what they were hoping for—the "pull" forces of what they wanted their future to look like. I became very attuned to what was motivating people to build or change their systems and ended up completely reworking all of our messaging from the ground up. It made me far more effective at selling those services.

An interesting outcome from that process was that I developed a very holistic view of how search is built. Through other micro market research initiatives, I eventually created what I’d say is now the core asset of my current business, The Search Bar.

That asset is a breakdown of every task required to work on search, mapped to the architectural components of a search system. I also realized that in the hiring space, the roles or job titles people hold aren’t one-to-one matches with the actual work—something I had assumed from my earlier experience in recruiting. So, I created a map of interdisciplinary roles—a kind of blend of capabilities that companies are really hiring for when they say “search engineer.”

I turned this into an assessment. I’ll give you a quick visual: (imagine I’m sharing my screen). What you’d see is a database of people. Say a product manager is overseeing their search system and an AI chatbot—it’s an enterprise search use case. They answer specific questions about their needs, and based on their answers, we map their situation to both the technical components and the roles needed. You can see where different roles are represented—maybe high on insights, product, and data science. From there, I can instantly generate a shortlist of people in our network with that exact blend of experience. We’ve done this with about a thousand search professionals now. It’s been a huge differentiator in how we help teams build better search.

What are some of the biggest misconceptions about search that you often encounter?

One of the biggest misconceptions I see—especially in the current AI climate—is that people think they can throw together a RAG system or AI-powered search prototype without really understanding the importance of traditional information retrieval. A lot of startups are popping up, building quick POCs, and they don’t realize that without a strong IR foundation, the result is something they can’t actually use in production or that won’t be useful to their users.

So the misconception is: search is easy now—but the truth is, search is still hard, and you still need people with deep domain expertise to build systems that actually work.

Another interesting shift I’ve noticed is that what I’d call “traditional search” roles—people setting up Elasticsearch clusters, building keyword or term-based search systems—are becoming quite rare. A few years ago, those jobs were everywhere. Now, companies are mostly hiring for the machine learning side of search: relevance ranking, recommendations, LLM integration, and what I’d call “modern search.”

To give you an example: a year ago, I’d go on LinkedIn and see maybe 20 open jobs for “search.” Today? There are over 200, and nearly all of them involve AI—whether it’s integrating machine learning models or building more advanced, personalized, and intelligent search experiences.

What’s also interesting is that I’m seeing people leave large enterprises to go work at startups—not for the money, but because they want hands-on experience with these cutting-edge tools. Even managers and team leads are looking for roles where they can stay close to the tech and build real systems with LLMs, vector search, and AI pipelines. This is definitely a high-growth area, and it's evolving quickly.

How do you envision AI and machine learning impacting search relevance and data insights over the next 2-3 years?

It’s massive. Absolutely massive.

It ties directly into what I mentioned earlier about job volume. One of the big trends I’m seeing is that a lot of traditional SaaS companies are going through what I’d call a replatforming—or retooling—phase. They’re restructuring their systems to handle AI workloads.

Often, that starts with a large round of layoffs. But then what happens is they rehire maybe 50% of those roles, but with entirely different skill sets. These are engineers focused on building agents, re-architecting data pipelines, and setting up infrastructure for machine learning workflows. And what they quickly discover is that information retrieval is always part of the equation—whether you’re building RAG systems, chatbots, or autonomous agents, you need to retrieve relevant information and pass it through models to get something useful.

That’s where search comes in. It’s the backbone. And yet, most people don’t realize just how deeply integrated search is to all these AI initiatives.

I also serve as an advisor to an organization called Seven CTOs, which is a large community of 300–400 CTOs across the U.S. And across that network, the questions are consistent: What are we doing with AI? How are we using it to stay competitive? And the common thread in many of those conversations is the need for search expertise—because when it comes to delivering relevant answers or actionable insights, search is still doing the heavy lifting.

So yeah, the impact of AI on search relevance and data strategy is huge. And it’s only accelerating.

Are there any open-source tools or projects—beyond Elasticsearch and OpenSearch—that have significantly influenced your work?

Not for me personally, since I'm not an individual contributor—I don't use these tools directly in my day-to-day work.

That said, I’m seeing more companies attempting to build their own embedding models and fine-tune them in-house. There are a variety of open-source tools they’re using to support that effort.

Anecdotally, I hear from teams that many off-the-shelf models aren’t moving the needle in a meaningful way for their specific use cases. So the return on investment just isn’t there. That’s leading more organizations to explore building and training custom models internally, which is, of course, a major undertaking.

What is a golden tip that you’ve picked up in your years of experience?

For engineers, I’d say: having strong fundamentals in software development is still absolutely vital.

There’s a younger wave of engineers who’ve gone deep into AI tools, but they often lack core software engineering foundations—and I’m seeing that reflected in hiring outcomes. These candidates aren’t always landing the roles they want, because hiring teams are still prioritizing solid engineering skills.

Yes, tools like Cursor and others are great, and companies do want engineers who are fluent in them—but not at the expense of sound software development practices. Being a great developer still matters just as much, if not more.

What is the most unexpected or unconventional way you’ve seen search technologies applied?

Honestly, I’m so steeped in every single use case for search that I don’t often come across one that feels like a totally unexpected outlier. But I am working with a client right now—this is super recent, like within the last month—and they're in this hot new space that had a brief buzz cycle: GEO.

It’s kind of like the new SEO. Think about how traditional SEO worked, but now apply that same thinking to generative AI—basically, how companies are getting their brands and content mentioned in ChatGPT or other AI tools. It’s a whole new area that’s emerging right now, with a lot of buzz and hype around it.

So while it’s not "unconventional" in the sense of being totally out there, it is an unexpected evolution of how search is being reimagined in the generative AI era.

Here’s my spicy hot take—one that goes beyond just search for jobs or matching people to roles: relationships are still king.

I see all these AI sourcing tools and companies raising tens of millions—one just raised $20 million in a Series A a week or two ago—and honestly, they don’t move the needle for my business. I wouldn’t use them.

When people are searching for a job, that’s a major decision in their lives. They want to know who they’re talking to, who they’ll be working with. That’s not something you can automate. This applies to sales, too. It applies across the board.

The AI tools being built to "automate" the recruiter’s job? I’m just not seeing it. My entire business model is a counterpoint to the direction the industry is heading. I have a very specific niche—search and AI search—and I build real relationships with every candidate. I spend 45 minutes to an hour getting to know each one. Every single one.

It’s the same with my clients. Obviously, that’s a smaller group, but it’s the same level of depth. That’s how good matches happen. We use tools like the one I showed you, which are incredibly helpful—definitely more helpful than resumes or job descriptions—but at the end of the day, nothing beats real human connection.

And not to talk numbers publicly, but I’ve gotten to know other companies in this space who’ve gone the generalized, AI-first route—and let’s just say the results I’m able to deliver mostly by myself are pretty wild in comparison. Relationships work.

Can you suggest a lesser-known book, blog, or resource that would be valuable to others in the community?

One I’d definitely recommend—whether you’re an engineer, operator, or business leader—is Prompt Engineering for LLMs: The Art and Science of Building Large Language Model-based Applications, by John Berryman and Albert Ziegler.

It breaks down how LLMs work, how prompt engineering works, how agents work—into really digestible concepts. As a non-technical person, I was able to build agents for my business after reading it. It’s just that clear and actionable.

I even had John speak at an AI roundtable for 7CTOs, and the feedback was incredibly positive. Super useful resource for anyone navigating this space.

Anything else you want to share? Feel free to share a product or some advice that you think the search community will find valuable

My lens is always: How do I help people get jobs? That’s really the driving force behind everything I do, especially when I think about the kinds of people using products like Pulse, the engineers and practitioners in the trenches.

So, my advice to them is: if you’ve mostly worked in traditional search, start branching into the machine learning side of things. Get some exposure to ranking, relevance, and the kinds of challenges that come with AI-powered search systems. If you’re already working with AI, you’re probably getting that experience by default—but if not, now’s the time to start. That shift is where I see the biggest career growth opportunities

Pulse - Elasticsearch Operations Done Right

The world's top search experts love Pulse

Learn More

Know a search-tech guru that we should feature?

Let us know

Subscribe to the Pulse Newsletter

Get early access to new Pulse features, insightful blogs & exclusive events , webinars, and workshops.

We use cookies to provide an optimized user experience and understand our traffic. To learn more, read our use of cookies; otherwise, please choose 'Accept Cookies' to continue using our website.