Google’s BERT Technology and WordWorks

Have you heard of BERT? Not Bert Lahr, Bert Convy, Bert Blyleven, or Ernie’s friend on Sesame Street: we’re talking about “Bidirectional Encoder Representations from Transformers” (BERT), an advancement Google just rolled out to its search algorithm, which is intended to improve the results of natural language searches.

Before you click away to another page because that seems too technical, don’t: that’s about as technical as this post will get.

We were a little bit geeky with the previous post and didn’t intend to focus on something like this again so soon, but this BERT stuff is actually pretty interesting, even if you’re not into the technical things that happen under the hood of Google’s search engine. It’s also potentially important in how your site’s pages—including the informational posts a service like WordWorks provides—show up in search results. Google itself has said that its BERT update could affect as much as 10% of all search results. Since Google responds to more than 5.5 billion searches every day, 10% of all results is a big number—more than half a billion results each day, more than 6,300 each second.

Put another way: if your site currently sees 10,000 hits per day via Google, now you might see only 9,000. That probably concerns you.

If you want the “tldr” version on BERT, though, here it is:

If you’re already using high-quality written content, designed to inform human readers rather than to game an algorithm, then you’re unlikely to see a significant change due to BERT (maybe none at all). If you’ve been trying to artificially leverage some aspect of a search algorithm to pad your hits from search results, then it could be a problem for you.

Let’s get a little more into the nitty gritty, without getting too technical.

BERT in 100 words or less

Yesterday, Google used context-based natural language searches in one direction and ignored small words in most queries, looking only at keywords. Today, Google searches determine the context by moving through a search string in both directions, and the small words in search queries are now taken into account to fine tune context.

The good, the bad, and some ugly (or: enough details to understand the what and the why).

BERT isn’t totally new. Google’s been working on it for some time and released it as an open source technique in late 2018. This month (October) it became part of all Google searches. According to Google and other sources who have looked deeply at BERT, it makes improvements to 11 tasks related to natural language searches. Two of them are the most important, and the most likely to affect existing sites and searches.

One thing BERT does is add to the capabilities of search engines by allowing them to work forward and backward (the Bidirectional part). Natural language search systems determine the context of each keyword in a search string by using earlier words in the string…but not the words that come after it. There are technical reasons for this, but BERT seems to get around them, which improves search accuracy. It might not be the difference between a lightning bolt and a lightning bug, but it could be the difference between country fried chicken and country fried steak. According to Google’s researchers this bidirectional technique improved search result success by about 1.6% in the lab, and pushed the algorithm significantly beyond human success at detecting context (the algorithm had previously been just slightly better than humans). That 1.6% might not seem like a big deal, but when we’re talking about billions of searches it adds up fast.

The other main improvement touted for BERT is an ability to use the small words that search engines have traditionally tossed out: a, the, is, for, and so on. According to one source, the word “no” is usually dropped from searches, so users might have been getting the same search results for both “restaurants with vegan burrito” and “restaurants with no vegan burrito.” It’s not that simple (nothing ever is), but you get the idea.

BERT was originally trained and tested using the full text of Wikipedia. We could make jokes about that, but on the whole if someone wants a massive, readily available body of pretty good text on as many subjects as possible, it’s hard to do better.

But What Does It All Mean?

That brings us to why we’ve bothered to write about BERT at all. Of course, when Google says that a change is the biggest in five years, that’s important. But how does that actually affect us down here in the trenches?

If you read (and read between the lines of) this article, you’ll see that some search experts with detailed knowledge of BERT’s inner workings point out that sites with poorly written (“weak” or “sloppy”) content are probably not going to be happy with the change.

For years Google has—sometimes quietly, sometimes not—been telling the world that good search results are based on good source material. That is: if you have high-quality, well-written content that users want to find and that uses (but doesn’t abuse) appropriate search terms, then that content is going to do well in search.

From that perspective, BERT has no appreciable effect on sites that are already doing what they should always have been doing. No WordWorks customer needs to worry about this (and none have reported any traffic changes).

Of course, we always stay on top of things like this (in addition to the fact that natural language processing is something that interests us). Whether or not we feel that an algorithm change of this sort will affect your business, we still need to understand what these changes mean. Sometimes we might have to react—but not this time.

This is another example of how working with Waltham WordWorks to generate your site content helps you in the long term. We do things right, and we do them right from the beginning. We don’t need to go back to fix problems related to weak content, because we don’t generate weak content in the first place. When you work with us, your pages are of the highest quality, written with the best practices for continuing search success.

To put your business expertise on display online, get in touch with Waltham WordWorks to learn how we can help you affordably showcase your business or industry knowledge and use it to attract new clients.