How to Stop Worrying and Love the Algorithm - More About Search Engines.

So in last week's post I introduced that idea that the internet has completely changed the way we find information -- from a "push" system where publishers compete to get the attention of the reader, to a "pull" system, where the publisher just sits back and lets the reader pull what they want from the system.

I don't know that I was fully clear about that last part so I will repeat it:

In the new internet paradigm, the information provider (the "seller") sits still and doesn't worry about selling the product. The customer finds you.

The very idea of this brings on panic in many. BUT... BUT... BUT... HOW CAN THEY FIND ME IN THIS HUGE CROWD OF JUNK??!!?? I'm LOST! I'm buried alive! Help help help!

I will start by telling you a joke my father used to tell. I'm really not sure if it's a traditional French Canadian story or if he got it from one of the Quebecois comics he followed. (If so, my guess is Michel Tremblay -- my apologies to Mr. Tremblay if I'm stealing his work....)

There was this baby bird who fell out of his nest on a cold morning. A man walked by and saw him shivering there, literally dying of cold. There was a warm, steaming pile of horse manure nearby, so the guy picked up the bird and stuck him in the manure to keep him warm.

After a little bit, the bird revived, and stuck his head up out of the manure, and started cheeping loudly for food. A cat heard the sound, and snatched the bird up and ate it.

There are three lessons to be learned from this story:

1.) If someone dumps you into a pile of merde, it may be for your own good.
2.) If someone snatches you out of the merde it isn't always to your benefit.
3.) If you're up to your beak in merde for goodness sake, KEEP YOUR MOUTH SHUT.

Which is my way of saying, if you find yourself swimming in the cosmic amount of doo-doo out there on the internet (or on Amazon) you don't need to panic and start screaming.

Sit back and enjoy yourself, because Google and Amazon have your back.

Yes, there are things you can do to help Google and Amazon do the job of helping customers find you. Some of those things are even promotional -- but for the most part, it's just a matter of going about your routine in ways that their algorithms can track and measure. (And also, of course, producing quality work. Just remember that to Google, every comment or forum posting is considered your "work.")

I'm going to try to give you a simplified version of what happens behind the scenes. This doesn't really talk about the user experience or anything going on at the surface, so I'm sorry if it's not clear how this benefits you. You're going to have to get your mind around how it works before you can understand the rest.

I'll talk mainly about Google here, partly because Google is a little simpler in terms of output, and partly because it's actually more important than Amazon -- and they're interactive with each other. (But that aspect is for another post.)

How does Google decide what results to give to someone doing a search? How do they filter out the junk which is screaming for attention with keywords and all those marketing tricks? How do they lead the searcher to YOU?

First, they examine everything going on out there in the web. They have computers gathering data about which pages link to which other pages, and whether users who search for a particular word respond well to those pages; whether they stay and read and click around the site, or if they hit the back button and keep looking.

But the biggest thing is the links. Google ranks each page with a kind of status or power -- which we sometimes call "Google Juice." If Google judges a site to have a high status, then not only will that site come out higher in search, but if the site links to any other page, those other pages will gain points from being associated with that powerful site.

So... here's an example I gave a while ago to Dean Wesley Smith:

1.) Dean's blog has a lot of juice for a couple of reasons all by itself. It has been around for a while, he posts regularly, and he gets a lot of traffic. Google also measures the "quality" of his content, based on measures they mostly keep secret, but we know they involve things like how long his posts are; whether they have a natural pattern of word use, or are just packed with keywords; whether the posts are consistent in terms of subject matter; whether it's all original content. It will also be judges on things like the fact that his visitors engage in long, exciting discussions in the comments. There is no spam on the site or in the comments. Nothing is forced, or machine leveraged -- the site is active in things that real live genuine readers do.

Google likes real live genuine readers.

2.) Google also measures the number of sites linking to Dean's blog, what are called "backlinks." The backlinks to his blog are also of high quality. He announces some posts on Twitter and such, but he doesn't spam or go around commenting on other people's blogs with "Nice post, visit my site." The sites linking to his may be lower status sites like mine, or high powered sites like Joe Konrath's -- but they all have good Google rankings for genuine content. That is, I may not be a high powered blogger, but I'm clearly not a link farm, I score well on the "quality" measures.

Furthermore, both Konrath and I benefit from linking to Dean, and not having links to spam sites, and also because there are legit links to our blogs from his. And as the low person on the totem pole, I benefit the most.

3.) Dean, Konrath and I -- and a bunch of other bloggers who are similarly linked -- are all writers, and write a lot about books and publishing, and genre fiction. So because we are already interconnected AND we have the same sort of readers, Google gives us all extra points in search on words related to our common topics. (And sometimes extra points for readers with a history of looking for such topics.)

4.) This is the magic part -- because of all the above associations, anything related to any of us becomes slightly related to the others. So that means anything Dean mentions on his blog gets a slight association with my blog and Konrath's blogs even if we don't talk about it -- and vice versa. And that sometimes even includes connections which don't appear on the web at all -- connections from real life.

So imagine this: Dean's wife Kris writes many books under pseudonyms, and neither she nor Dean reveal every pseudonym they write under. Say neither of them have ever mentioned a particular book on their websites, but at a Clarion reunion he happened to mention the title of one of Kris' books to me. I didn't know it was written by Kris, but I looked it up, and ended up talking about it on my blog.

Result? Without any promotional effort on the part of any of us — especially not Kris — and without my mentioning her name either, her book just got more likely to come up in searches by people who like her writing. That book benefits a little bit from any credibility Dean has, I have, and Konrath has. (And ironically, some of the credibility that I give to that book comes to me because of my indirect association with her, via Dean.)

So in the end, her book becomes associated with her, indirectly, even though nobody knows she wrote it. And from now on, anything she does will, in some miniscule way, promote that book nobody knows she wrote.

There's a reason why it's called the world wide WEB. It's all interconnected, and Google uses it all.

And that's just a really tiny part of it all.

No amount of promotion or money can fake the power in the huge amount of data points Google has on offer. It’s all just genuine, normal activity of people who are interested in the content, not the status.

That’s what’s coming to the publishing world.

So what's an unknown writer to do?

First you recognize that you aren't completely unknown. Google knows who you are. That may be a little scary, but that's the world we live in. The connections you need -- tiny ones -- are already there. You've got to build on them. While promotional activity might get some immediate attention, remember that's not where the Google juice comes from. Google won't make much use of it to help people find you. The juice will come from natural interactions, and quality content, and persistence.

And persistence may be your best leverage -- just stick around long enough.

So, remember: don't start yelling because you feel lost in a pile of crap. Google and Amazon both know where you are, and they will look after you.

Just learn to stop worrying and love the algorithm.

I hope I haven't confused anyone. I hope to clarify things in future posts -- especially to show how this works on the customer's end. Please ask questions in the comments if you need something clarified.

Next week we'll talk about how readers find books these days.

See you in the funny papers.