SEO Starter Guide: What’s all the drama?

[Music] welcome everyone to another episode of the search of the record podcast our plan for this series is to talk a bit about what's happening at google search how things work behind the scenes and who knows maybe have some fun along the way my name is john mueller i'm a search advocate on the search relations team here at google in switzerland today we have a special episode lined up with lizzy and michael hi everyone hi hi hello lizzy's been on our show a few times she's also on the social l

icense team and she's our tech writer but michael do you want to introduce yourself briefly sure yeah i'm michael or a lot of people also know me as wiz i worked on search from 2006 to 2015 and got to do a bunch of things with people who work on websites so if you've been around for a while like maybe you've seen me at conferences or in the forums or something like that i no longer work on search but i'm still at google and i write a documentation for internal tools now so good to see everyone w

oohoo fantastic to have you here thank you many of you are surely wondering why i have summoned you at this hour what started as a joke turned into reality someone a user commented on our documentation that it would be fantastic if we could have our documentation in audio format lizzy took the opportunity to notify us about this feedback and now we are going to do dramatic reading of our own documentation and the very best place to start with this journey is our seo starter guide [Music] how how

do i get my site on google inclusion in google search result is free and easy you don't even need to submit your site to google google is a fully automated search engine that uses web crawlers crawlers crawlers to explore the web constantly looking for sites to add to its index in fact the vast majority of sites listed in our search results aren't manually submitted for inclusion but found and added automatically automatically even when we crawl the web [Music] now the seo starter guide actuall

y predates most of us in the search relations team i know of very few google dinosaurs that have been around back when it started but i know for certain that wiz and john both have been around wiz would you like to say a few words about how this thing started and most importantly why sure absolutely i actually reached out to the original author to get the inside scoop on how this all happened so i had a little conversation with brandon who used to work on the search quality team and the first th

ing that he said is that most good things from our team it was matt's idea so matt cutts who we all know used to be in our videos and speak at conferences and really did a lot to get this search relations stuff going at google what we wanted to have was advice straight from google about seo because we would get asked about seo all the time both internally and externally internally we can't give any different advice than we give externally so it was like well let's see if we can create something

that we can give internally and externally and it will be great to have that it was also important for us to show that google isn't against seo you know sometimes we get questions about that and we weren't you know if it's done well this is what we wanted to show the best practices so that's how it came to be originally started as a pdf but then we also had a booklet version with the google bot character who's now everywhere it's been really cool to see that it's still going and still getting up

dated i totally remember the booklet i think we had a bunch of those printed as well and we brought them to conferences right yeah that's right and actually i brandon was saying that matt wanted to have something at conferences but in the initial versions we just did it as online but then independently the tokyo team sort of search relations team in tokyo they wanted a booklet as well and this was shortly after like chrome had come out and they had had that comic book with the chrome browser rel

ease and also at conferences i think a few of us had mentioned that we'd sometimes be asked do you have a manual for webmaster tools or like a booklet or something and we would sometimes be like the only booth that didn't have anything to hand out like everyone else had all this swag and printed materials to give out so anyway eventually we thanks to the tokyo team ended up doing the guide and a printed version and with that i got the illustrations of the great google popcorn character that we h

ave now oh yeah that was i think like the beginning of the googlebot character we still have that i don't know if you notice we also added a new character to the lineup have you seen it i heard there's a spider yeah the spider what's that spider's name again lizzy oh don't hurt me like that we still don't have a name so i guess this is another open call if you have any ideas you can tweet at us on twitter your suggestions for the name for the spider spider names we can call it i don't know four

eyes or something actually i don't know how many eyes spiders have so maybe that was insensitive sorry spiders for sensitive information use more secure methods a robot.txt file is not appropriate or effective way of blocking sensitive or confidential material it only instructs well-behaved crawlers that the pages are not for them but it does not prevent your server from delivering these pages to a browser that requests them also non-compliant or rogue search engines that don't acknowledge the r

obot's exclusion standard could disobey the instructions of your robots.txt finally a curious user could examine the directories or subdirectories in your robots.txt file and guess the url of the content that you don't want seen [Music] yeah one of the early sections in the guide is all about robots.txt it feels like one of those things that just remains the same for ever and ever how do you see that do you think all of that is still relevant where did that come from yeah i was rereading it and

i saw that it also mentions that you shouldn't use robots.txt for sensitive information i mean it's kind of like a sign that says don't look here and it tells you exactly where not to look and that actually also reminded me that well this was a while ago at this point we were noticing that we didn't have a lot of documents especially i think from government sources from korea and the team actually did a lot of work to figure out why is this happening and a lot of these sites were just blocked co

mpletely with robots txt and they did some investigation and asking around and they found out that there was actually a government document with recommendation i think it was in the security section that was like make sure that you block all robots with robots.txt so that's why they're you know this wasn't this good coverage and so the team actually did a lot of work to reach out to site owners and i think they even got that document changed yeah so it's it's still relevant though it's actually

fascinating because the issue still persists to some extent and we still have to do some outreach to government sites in korea due to the robots.txt rules that they set and the more fascinating thing to me is that they just blanket block every single robot it's not like they would allow certain robots that is more typical in other regions of the world they are just blanket block everything so i find that fascinating even though it's i think it's a misunderstanding of robots.txt and what it's sup

posed to do and not do so what's the best approach with robots text should they just allow everything or delete the file or is that all the same i mean it's for controlling crawling right like what do you want to have crawled or actually not have crawled right but since you are writing in plain text format in the robots.txt file you can't use it to hide stuff like for example if you have like a secret folder slash where you store your secret cat pictures then you don't want to put that in the ro

bust txt file because a human can also open it and then discover your secret folder so for that it's not good but it's good to manage for example cruel budget like if you have a section of your site that you don't think that it's that useful or if you have certain url patterns that are just a different version of the same contents that you allow crawling for that you can use the robust dxd disallow rules so you mentioned that these government sites had a misunderstanding of how like what was the

ir initial understanding like why were they oh we want to block everything did they think it was a secure method or like why was it tied to security i guess i don't know like we never found out why they were doing it we just asked them like what's up with that and can you please do it this other way yeah i'm not sure this john do you remember i remember this was mostly the korean team that was working on this yeah i think one of the issues there was on some of these sites they had personal infor

mation where you could look up something about your citizen id or something like that and the government sites were kind of worried that the site owner might not know where the confidential information is and kind of as a best practice it's just like oh just block everything then you'll be on the safe side on the one hand a little bit of understanding of how the web works a little bit like search engines go off and find stuff on websites but not that level of understanding that actually if you m

ake something public on the web then it'll be accessible but you don't have to make things public to publish them on the web you can put them behind the login you can kind of lock them down in more legitimate ways so it's kind of almost like a feeling that well these site owners don't really know what they're doing therefore they should just block everything which then of course leads the situation that nobody can look up anything about that site it's like where can i look up local government in

formation it's like well it's nowhere or rather we almost have to guess because if it's blocked by robot's text we wouldn't index the content on there but we might see links pointing there and we'd be like well maybe this site is relevant to you but we we don't really have any idea so that's really weird situation where that ends up i think also in korea there was kind of a unique situation with the search engines in general because the other internet sites there were more directory type sites w

here people would explicitly list websites rather than them going off and crawling the web to find stuff yeah i think i remember that same idea about i forget if it was like forms or portal kind of format which is maybe why the issue wasn't so obvious in the local markets unless you used a search engine that actually goes out and crawls [Music] in a world of mobile devices most people are searching on google using a mobile device the desktop version of a site might be difficult to view and use o

n a mobile device as a result having a mobile already site is critical to your online presence in fact starting in late 2016 google has begun experiments to primarily use the mobile version of the site's content for ranking parsing structure data and generating snippets oh i actually have a funny story about the mobile friendly test it was when it first launched i was like okay i'll just try my personal website on it and so i opened it in the the mobile friendly test and it's a super super simpl

e hand coded html page with html from the 90s so it's like a link and then a break and then another link and another break and then a link and i'm like there's no way that i have to do anything special for this to work on a phone it's the simplest possible thing and i put it in the test and it's like the tap targets are too close together or too small or something like that and i was like that's not right this is the simplest possible page there's no way that it's right and i was sharing an offi

ce with two other people on the team eric and mary and i said the test isn't right like it's saying that this very simple page isn't working and it says that the tap targets are too close together and i'm like look let me show you that it's that it's wrong it's the simplest page and so i opened up my phone and i opened up my home page and i'm like look so if i just want to tap on this link here to my blog and i went to tap and i completely missed the link and i ended up on the wrong page and so

that was a pretty humbling moment where i was like oh i guess the test is right we're all good it's so cool all of the the mobile friendly stuff that we had there and kind of the guidelines in general for mobile it's funny some of the first seo conferences i went to i was like well we have all of this mobile stuff and people in the audience were like but these phones is like people are never going to use phones to access the internet come on way too annoying to use these small devices to access

the internet i don't know it just felt so so interesting to have all of this information there so early and essentially over time we didn't end up changing too much like most of that just ended up working well with mobile phones so that's pretty cool well i guess apart from the the feature phone stuff where we still have some of that in our docs every now and then but i imagine nobody actually has these phones anymore so what's up with featured phones by the way i don't know i mean it's probably

not that they broke or that the batteries are empty now because they're pretty robust right so it's like my old feature phone compared to my smartphone the display is still okay and the case is still okay wait are you still using it i have it in a drawer and i was like every now and then i look through my drawers and it's like i need to throw out some old stuff and i'll be like oh but this feature phone it could probably still work if i charged it up imagine if the apocalypse came and smartphon

es no longer worked i would be the only one with internet access probably not no i mean the edge network was turned down here a few months ago oh my gosh so you're saying i'm keeping it for nothing that's terrible yeah so i shouldn't optimize my site for feature phones ask yourself does it spark joy probably yeah oh then keep it then keep it keep it okay i think the harder part is finding the right charger for it seems like a critical part critical part i don't know every now and then i get temp

ted to take a photo of the google home page on a feature phone just to show that it still works but maybe it doesn't work anymore it's just like keeping it for nothing so awkward i don't want to be dramatic but you should really be promoting your website putting effort into the offline promotion of your company or site can also be rewarding for example if you have a business site make sure its url is listed on your business cards letterheads posters etc you could also send out recurring newslett

ers to clients through the mail letting them know about any new content on the company's website [Music] yeah so there's this other section about promoting your website and i was looking what's in there and there's a section that just says know about social media and i was wondering what's up with that like when this guide was written what year was it and what were we meaning by that just to like know about social media i mean the year was 2008. and so what was social media back then twitter rig

ht was twitter twitter was i don't know what what to hear what was twitter there i think twitter just started i think that's roughly when it started we had google oh no wait google class oh google reader was also a social network right where you shared things with other people oh that's right google reader that's true it was yeah and buzz buzz don't you remember buzz wasn't that after 2000 i have no idea i think it was after it was like 2010 maybe but we definitely had delicious for example we d

efinitely had stumble upon we definitely had dig and based on personal experience and based on data from hundreds of my sites these were actually very very good for driving traffic to certain kinds of sites so back then it was less about ranking and more about just getting traffic i mean it depends what you wanted to do with your site same as today so there's this point in here about that you should avoid attempting to promote each new small piece of content and instead only do the big interesti

ng things do you think that we would be still recommending that today or is that something that we should go back and adjust like should people post uh when they've i don't know done a new blog post for example like we tweet when we do a blog post would that be all right i think it depends on the site and or the section of the site right because like if you have a blog and you post like once a week for example then you do want to let people know that you posted but if you are big news site say c

nn or wall street journal then you can possibly tweet everything for example like if you tweet everything it will be just like a constant flow like a writer's feed for example the professional feed where you literally have like a constant flow of uh headlines coming in and they are just like scrolling in front of you for that it's not appropriate i think so do you think it's about seo or is this just general best web practices it's a marketing thing right it's less about seo more about marketing

but this is the seo starter guide so you want to cut this part i don't know i that's what i'm asking is this related to seo like today or was it back when we wrote it i guess and should we still keep it i mean i guess that's something interesting to think about because it's sort of related because it's like you have a website and you're reading this document so there's some other things in here for example that section about custom 404 pages like oh this is like a nice thing that you should do

for users is to have this page and make it a little bit better for them to like navigate around your website and is it necessarily something like where do we draw the line of like oh this is okay to put in because it's directly related to search or like this is sort of a related topic that would be nice to have in and people are interested in it so we keep it does it spark joy gary um so where in the serving pipeline do social media posts get used well they don't i mean we have a twitter carouse

l right but otherwise they don't i'm inclined to say that we should cut this because it's way too much on the marketing side if you think about seo holistically it probably has some marketing as well but how people advertise their site which tweeting about every piece of content kind of is that probably shouldn't be covered by the seo you know there's also like another paragraph earlier in the promoting your website that says you should mail physical mail to people about your website i did that

it doesn't work oh wow so maybe more things that we can update i mean it also covers the spamming link requests the social media sites thing where i imagine in the early days we picked up links from social media sites and used those for pagerank and things like that and probably that's where some of this is coming from is like just don't link drop your sites urls and random social media sites and instead think about it a little bit more but i guess nowadays social media sites don't have do follo

w links anyway so it doesn't really matter right are you asking me yes i don't know why i stopped spamming a long while ago good good good okay but i mean should social media sites have do follow links or are they all nofollow now i don't know if they are all no follow i can imagine a few very select sites that don't care for these kind of things we should be able to tell these sites apart and just on our side no follow or markers no follow all the links that are coming from those sites i would

be very surprised if there was any big social media site out there we do follow links that we haven't already marked as nofollow internally for ourselves a title tag tells both users and search engines what the topic of a particular page is users because they definitely go to your html page and check the title tag the title tag should be placed within the head element of the html document you should create a unique title for each page on your site or don't we are just a voice recording brendan's

baseball cards buy baseball cards baseball use card prizes brandon's baseball cards provides a large selection of vintage and modern baseball cards for sale we also offer daily baseball news and events [Music] so one of my favorite sections of the seo starter guide is actually about titles and meta descriptions because they are actually helpful driving people to your site very targeted people if you are smart about them because what you write in the title tag and in the meta description may act

ually show up in the search results as you provided and we've seen this happening on our own sites like the search central site what's our site called google webmaster central site search no and we got lots of recommendations from our own seo team about what kind of titles we should use and we see them perform live basically we can see that they are changing the kind of visitors that we get from searches for certain queries so what kind of recommendations do we get from the seo team did they jus

t want to add like high traffic keywords into their titles or what were the recommendations i mean that was definitely a part of it like they did some keyword research to see like what words that they should be putting and they sent me this spreadsheet that had like the keywords and then they're suggested like okay we want to write this based off of this so it didn't make sense like how they were coming up with the title and the description they also had some interesting style choices like capit

alize every word whereas like in the docs if i'm writing a title our style guide for the documentation is just a sentence case thing so you just capitalize the first word whereas the title that they were suggesting had like every word capitalized and then like a pipe thing and then like the name of our site so extra stuff that we i wouldn't necessarily want for the heading or like title of the page so that was sort of interesting i'm curious does that make a difference for just people who click

on things in search results if things are capitalized every letter versus sentence case is that something that like looks more official have you noticed that this is a trend i went through some of the recommendations when they came back and i know that i was grumping around about some of the recommendations because i felt that they are irrelevant but then you were very suspicious you were like i don't think this is gonna work and we shouldn't have this pipe thing so the pipe thing that's actuall

y interesting because i said that we would just append the site name again whatever we determined for the site we would just append it again and that happened like we have right now in the search results results that have double name even though that's super long like we have the title and then google search central and then google search central again well you don't see the second part basically it just starts like google search central pipe and then google dot dot is it google developers becau

se that's like the site that we're under or is it like to a google search central i could look it up i don't want to because it's our own site but but yeah it's kind of weird i think uh so i was grumping around about that as well but the capitalization in my opinion didn't make any sense i don't think i said anything about that particular thing but i definitely thought that it doesn't make any sense i just thought it was interesting and i was wondering from like a user perspective like maybe the

re's nothing from like a search engine perspective but like if you're a user and you're scrolling does this like result look more official because of the capitalization and it's something that i had not thought about and it just seemed like an interesting oh we're gonna change it and we can support that because we can have like the title tag and then different heading for like what i want on the site so we don't have to align there but just the idea that from seo perspective they think that this

is more helpful and maybe it's a user thing maybe i really like our recommendation in the seo starter guide and i think that makes the most sense like for example the snippet that we have in the code blog on the guide is basically just saying brandon's baseball cards buy cards baseball news and card prices and that in my mind makes perfect sense because it conveys the information that is going to be on the page i do have one question though and that's the name of the site and where it comes fro

m because throughout the guide we get to see information about brandon's baseball cards and i have no idea what that is i mean i know what baseball cards are but not what the site is yeah this was actually a little controversial or questionable when we were writing the guide so it's called brandon's baseball cards because brandon was the original author of the guide and he was trying to think of a topic and baseball cards happened to be his childhood obsession but at the time again you know i th

ink we're talking 2008 baseball cards weren't really popular it seems like they were well past their their peak in terms of popularity and there were some questions like uh is this really going to translate well to other regions so when i was talking to brandon recently he's like you know now there's nfts and physical sports cards are trading for record highs and so he said that he feels a bit vindicated still relevant yeah but does the site actually exist the domain is registered and it uh redi

rects to a pretty good search engine my favorite search pretty good search engine is it steve oh you're not talking about steve that the steve search engine yeah i think that's pretty cool it's it's also cool i i think that the site exists because otherwise i'm sure random people will be like oh google is linking to this baseball card site i'm going to just reserve that domain name i mean not that people just go off and reserve domain names for random reasons yeah it never happened before yeah i

totally don't do that every month i have so many random domains that i bought as like a joke for like a one-off thing and then every time it comes up for expiration i'm like well what if somebody parks it or what if i you know want to use it or i think i linked to this at some point in a blog post and you know should i go back and delete the link or no i can just renew the domain and so this is now like over a decade later for some of these domains and so it's hundreds of dollars on one-off jok

es in this like recurring billing situation i need to get myself out of yeah but when you consider the value of domain age in google search you know oh why would you do that i thought it was really fresh content that we want oh no the really old content why let's fight it youtube gary you should do a session on domain age and authority on ranking or you know not creating compelling and useful content will likely influence your website more than any of the other factors discussed here users know

good content when they see it and will likely want to direct other users to it this could be through blog posts social media services email forums or other means organic or word of mouth buzz is what helps build your site's reputation with both users and google and it rarely comes without quality content [Music] i was reviewing the guide since you asked me to be on on the podcast and i was reading through it and i realized a lot of this stuff like writing for users or you know don't deceive sear

ch engines is just really evergreen like the advice doesn't change especially with writing for users i was thinking about you know i used to go to in-person events when that was a thing and my favorite part was always if we would do an in-person one-on-one site clinic so you're literally sitting at a table and someone comes up and you pull up their site on a laptop and take a look at it and so many times someone would pull up their site and i'd open it and i'd stare at the home page for a minute

and i'd read it and it would say something like we provide innovative solutions and pigmented liquid to enhance the aesthetics of residential living and i'd be like okay all right and then i would have to look at the site owner and kind of admit like so i don't really know what your site is about like what do you what do you do and they would say oh i'm a house painter and i was oh okay put that on your site that's what users will be searching for and that's what users will understand now i'm a

tech writer at google and i've realized like writing is hard lizzie i don't know if you've had the same experience but i found that if i start with writing i'll end up writing in this like really complicated language that's hard to understand but then if i'm explaining to someone in a meeting what this tool is about or something like that all of a sudden it's super easy to understand and i just go and i delete my draft and i write it out in plain english no i totally relate to that and i did no

t know you were a tech writer now until this moment i feel like a connection his sudden connection yeah yeah i am i can totally see that happening too and sometimes i do that with engineers just like let's have a meeting and you tell me explain verbally and we'll write it down live together and it comes out so much better than if they were to try to like uh write it down and then it's just a jungle of words verbally like you can tell someone to write how you speak but it's hard when you're stari

ng at a blank google doc and you sort of have this like other voice yeah when you're writing like that and it's hard to break out of that yeah definitely and i think it's also interesting that this is another one of those recommendations that can also be a little bit outside of seo and it's kind of something that you should do anyway even if search engines don't exist because i mean you might have like the perfect title and meta description and and that shows up in the search results but if some

one clicks through to it and then they get to your page and you know at that point seo is kind of over they're on on your page but if they don't understand what your site is about are they going to really sit there and try to figure it out are they just going to click back and go on to the next thing you bring up a good point because you said perfect titles and descriptions and like perfect for who the search engine or the user because that was some of our feedback that we had when we were given

the suggestions for what to put in our documentation for the titles and descriptions some of them were just like a jumble of keywords just like mashed together and okay if you were to think if you're a user looking at that in search results would that make sense it's just like a bunch of random words so it does seem like there's a balance of like like perfect for who yeah i'm a big fan of writing for users not search engines i believe that's in our guidelines focus on the user and all else will

follow i mean that's also a direction a lot of our new algorithms are taking as well or like i don't know not really new but like a lot of the machine learning stuff that we do in search is all about understanding what normal text is and understanding what what they actually meant with this piece of text where it's like well the keywords are not exactly in that text but it's a clear concept that's explained what this is about so that i think always still makes sense and i still run across sites

where i look at it i can see well probably a very expensive web designer put this together it looks really really nice but i have no idea what the site is actually trying to sell them to me and then at that point maybe you show up on search maybe you get traffic but if nobody clicks through and actually buys anything what use is that site yeah and that's also another reminder like if you said the algorithms are moving more in this like natural language direction there's always this kind of i do

n't even want to say it's a trade-off but if it's do you want to think like long term or short term and so if you're writing for search engines and maybe focusing on keywords in the title because that's like the hot ranking factor of the moment but then you have to keep up with it if your algorithm changes that's a lot of work and you have to constantly be like reading the latest advice and that kind of thing but you know as someone who's like stepped away from search for a few years i found tha

t if i just look at the basic principles of creating sites that users would like that hasn't changed and then you're chasing the same thing that the algorithms are chasing and so it's almost like search engines will eventually catch up to you if you've done something that's good for users and you can just focus on your site and not have to worry about every single new ranking factor or something like that i've even looked back in the internet archives version of the webmaster guidelines from 200

3 that was the oldest version that i was able to find and it had things like make pages for users not search engines don't deceive your users and the basic principles that are still there there's always technical reasons for technical accessibility and that kind of thing and if you want to focus on the latest and and greatest because yeah sometimes the algorithms aren't there yet but it's kind of nice that if you want to you can sort of sit back and focus on the long term you still love search d

on't you i do i use it every day yeah i mean it's how i find things it's it's amazing but also the the way you talk about it the way you uh talk about the older webmaster comes i think you miss us i do i do actually think about the team a lot there are just so many good things about it like both in terms of the work that we did and then also the way that we work together and i'm constantly bringing things up i've worked on a couple of other teams since search at google and with writing specifica

lly i've i've talked about how nice it was to have a global team of experts who were also writers because then we weren't doing just straight translations or just straight global versions of all of our content it was up to the local teams to figure out what's relevant for their region and they would share it with the team and if you remember with the mobile friendly update we initially had that label and the search results and if you just send i think it in english said something like this site

is mobile friendly or something like that and if you just send that to a translator who's like not on the search team and doesn't have context you might get interesting translations back that aren't quite right you know either if there's a different industry term or if you know mobile can mean like you know we think about it in terms of mobile phone but what if it also means like the ability to move i think some of the initial translations came back in that kind of strange way but fortunately be

cause we're the search team and we work in so many different languages we were able to before launch have the team review that and they come up with better ideas that sometimes were not literal translations but they actually got the point across a lot better and so that's the story that i like to tell the teams that i work on now when we talk about making something for a global audience or moving into a you know different area it's a great team also i heard some rumors about love stories related

to seo starter guide or something yes oh there is a love story and it's something that i just learned recently again from brandon who wrote the guide so it turns out after the guide was published there was someone on our team who was traveling to another office and he sat next to a woman on the plane who he didn't know and i guess they started talking and she asked him where do you work and he said google and the first thing that she said was oh i'm a small business owner and i just read the se

o starter guide it's great oh and they later went on to get married what oh my god so read the seo starter guide it might lead to a love connection well that escalated fast wow okay that's like so topical it's so crazy what is the chance of meeting a random person on the plane i just read your documentation wow it's about one in seven billion i don't know maybe maybe the chances are better now with like fewer people flying what do you think when was your last flight gary don't hurt me like that

[Music] and that's it for this episode we've been having fun with these podcast episodes and i hope you the listener have found them both entertaining and insightful too at any rate let us know how you're liking these and if there are any topics that we should be covering in future episodes and of course don't forget to like and subscribe and read the seo starter guide thank you and until next time bye bye bye bye [Music]