Python for SEO: Complete Guide (in 9 Chapters)

Chapter 1: Python Basics

Why Learn Python for SEO

Learning programming and data science is obviously good and gets you productive, but you have to put in the effort to make it happen. As an SEO, here are some reasons why learning Python might be a good idea for you.

How to Learn Python for SEO

To learn Python for SEO, start by using Google Colab and learn some of the basics such as the Python syntax, printing, variables and comments, and then move to installing Python libraries and debugging your code. In the following tutorial, experts share their favourite Python Tools, Scripts and give tips to help anyone who wants to learn Python For SEO. Read Python insights from Hamlet Batista, JR Oakes, Seth Black (creator of python seo analyzer tool), and more.

Easiest Way to Get Started with Python (without Installation)?

The easiest way to start with Python SEO without Python Installation is to use Google Colab and start practicing some of the Python basics instantly. Here is a simple tutorial to get you started.

How to Install Python

To install Python, you need to download the Python executable file. Depending on the operating system that you work with, different options are available. The most popular option to install Python is using the Official Package installer. In the data science community however, some prefer the Anaconda Python distribution for data science and machine learning. Anaconda comes with many other softwares that you may not need. It can become complex over time to work with. Here are the 3 ways to install Python:

• Install Python on MacOS (Video)
• Install Python on Windows (Video)
• Install Python With Anaconda [On Windows]

How to Learn Python For Beginners

To learn Python for Beginners, starts with a simple IDE that allow you to practice Python skills with the basics of Python. In this tutorial, you will learn how to master Python code, functions, and objects and develop the Python programming skills needed to use Python for Search Engine Optimization.

How to use Google Colab With Python

To use Google Colab with Python, create a Google account and start using Colab in your browser. Python and most python libraries are pre-installed. Thus, users don’t have to go through the lengthy process of installing Python and can start using Python on any machine instantly. In the following tutorial, you will learn more how to use Google Colab with Python.

How to Use VSCode with Python (recommended)

To use Visual Studio Code with Python, you will need to install both VS Code and Python on your machine, as well as the Python extension. Using a code editor like Visual Studio Code allows you to integrate more easily with Git and leverage the Python linting and debugging features.. I prefer this to Spyder IDE (below). In this tutorial, we are going to learn how to get started using VSCode with Python.

How to Work With Jupyter Notebook

To work with Jupyter Notebook, you need to have Jupyter Notebook installed on your machine or use Google Colab. Jupyter Notebooks offer a convenient interface to run Python code, add comments in the markdown syntax and share your work with others. It is free, runs on a browser, and is easy to get started, even for beginners.

How to Run Python With Spyder IDE

To run Python with Spyder IDE, you need to have it installed. It comes by default with Anaconda. This is a free open-source integrated development environment tool that you can use to run Python. I generally recommend alternative IDEs such as Google Colab or VS Code.

Python Libraries For SEO (Pandas, NumPy, Matplotlib and more…)

Learn the top Python libraries that you should learn before you get started using Python for SEO. In this post, you will learn how to use NumPy, Pandas, Matplotlib and Seaborn.

Data Visualization with Seaborn

Data visualization is a core component of understanding SEO data. Seaborn helps for better communication of data to the internal team.

Chapter 2: Technical SEO Challenges

Get Number of Indexed Pages on Multiple Sites with Python

Learn how to perform a site: search on Google to get a rough estimate of the number of indexed pages on Google. Using Python and Selenium, we will generate a CSV with the number of indexed pages for a list of URLs.

Find Rendering Problems On Large Scale Using Python + Screaming Frog

Knowing how Google renders your website is critical to mastering your SEO performance. Using Python and Screaming Frog you can solve rendering problems on a large scale.

Recrawl URLs Extracted with Screaming Frog (using Python)

Data manipulation is essential for Search Engine Optimization (SEO). This case study will help you improve basic data manipulation skills using Pandas.

SEO Keyword Cannibalization with Python

Keyword cannibalization is a major issue on large websites that deal with lots of keywords. Theses posts will show you how you can identify internal pages that compete for the same keywords using Python

• In-depth SEO Keyword Cannibalization Analysis in Python (by Jason Melman)
• Find Keyword Cannibalization issues with Search Console (by Natzir Turrado)

Randomize User-Agent With Python and BeautifulSoup (by JR Oakes)

How to use random user-agents when extracting information when using Beautifulsoup to grab the internal links for a web page.

Create a Simple XML Sitemap With Python

How to create a sitemap.xml file using a simple Python script and split it into files with less than 50,000 URLs.

Parse Robots.txt to a DataFrame with Python

In this post, I will show you how to parse a Robots.txt file and save it to Pandas Dataframe using Python.

Keyword Density and Entity Calculator With the Knowledge Graph API (by Greg Bernhardt)

Keyword density is a useful tool in determining a web page’s identity. In this step-by-step tutorial, I’ll show you how to calculate the word frequency for each page and sitewide using Python.

Forecasting Speed Improvements with PageSpeed API, Lighthouse and Python (by Daniel Heredia Mejias)

It is important to know which Pagespeed metrics are going to have a higher ROI and spot those quick wins. This post will show you how to forecast speed improvements with PageSpeed API and Lighthouse.

Chapter 3: Web Scraping With Python

How to Use Selenium With Python

Selenium is the perfect free browser automation platform and a very popular tool amongst SEOs. Learn how you can use Selenium to automate technical SEO tasks.

Web Scraping With Python and Requests

In this web scraping tutorial, you will learn how to extract SEO information from a website using Python Requests and BeautifulSoup. You will learn how to extract meta tags, headings and broken links from a web page.

Web Scraping with BeautifulSoup and Python

In this Python SEO tutorial, you will learn about how to do web scraping with BeautifulSoup. You will learn how to use the requests library to fetch web pages and the BeautifulSoup library to parse the HTML in Python.

Scrape Google without Being Blocked (with FREE Apify Credit)

Apify is my new favourite web scraping tool when it comes to scrape the web without getting blocked. They handle most of the hurdle behind web scraping and offer pre-built tool to scrape Google.

Chapter 4: Automation With Python

Python Script Automation Using Task Scheduler (on Windows)

The task scheduler is a Windows-based tool that lets you schedule tasks to run on a schedule. As long as your computer is open you can run scripts daily, weekly, monthly with this app.

Python Script Automation Using CRON (on Mac)

Cron is the tool that let users run script, commands or software automatically on a specified schedule. Crontab is the file that lists the jobs that cron will be executing.

How to Send an Email Using Python + Gmail

Tired of sending the same email over and over? Send email using Gmail and Python. Combined with the task scheduler, it is the perfect script to send weekly reports to your clients/boss.

Backup Google Search Console Data Into MySQL With Python

This guide will show you the easiest way to automatically backup your Google Search Console Data in MySQL using Python.

Build automated SEO Audits with Slack+Python (by Stefan Neefischer)

Get notified when a python SEO audit job returned some issues and attach detailed information in a file to the slack notification to take action right away!

Chapter 5: Use APIs With Python

How to Connect to a Google API

Google APIs are amazing! Learning them is one of the most important things that you should learn when learning technical SEO.

How to use Google Analytics Data API

This is the absolute beginner guide to help you connect to your Google Analytics API using Python.

How to Connect to Google Search Console API using Python

This tutorial is for SEOs that would like to use Google Search Console API with Python. This is the simplest way to learn how to make your first API call to the Google Search Console and Python.

Google Search Console Data From a List of URLs

With this Python script, I will show you how to extract Google Search Console data from a list of URLs. This is the perfect solution to get GSC data from URLs you have crawled.

Get All Your Search traffic With Google Search Console API (more than 50,000 keywords)

This post will show you how you can automatically extract all your search traffic and all your keywords from Google Webmaster Tool using Python and the Google Search Console API.

How to use Google’s Indexing API with Python

This post will show you how you can send indexation requests to Google’s Indexing API to automatically update or remove URLs in Google’s Index.

How to use Google’s URL Inspection API with Python

This post will show you how you can query Google’s URL inspection API to view the indexed, or indexable status of a given URL.

Chapter 6: Social Media Automation

Reddit API with Python

In this guide, we will learn how to use the Reddit API. Get top posts from a subreddit with the Reddit API, how to use Reddit API With Pushshift.io, how to get Reddit API credentials and how to post to Reddit with PRAW.

Slack API with Python

In this guide, you will learn how to get your credentials, send your first message and learn by example by sending Robots.txt changes alerts to Slack.

Linkedin API with Python

In this tutorial, you will learn how to get your credentials, authenticate the API using OAuth 2.0 and make your first text and link post using the LinkedIn API and Python.

Facebook Graph API with Python

In this guide, I will show you how to use Python and the Facebook Graph API to post to Facebook Groups.

Twitter API with Python

IIn this extensive guide, you will learn everything that you need to use the Twitter API with Python. I will cover the most important points (and missing points) from Twitter API documentation.

Chapter 7: SEO Split Testing

SEO Split Test Using Python + CausalImpact + Tag Manager

In this guide, I will provide you with everything you need to set up your own SEO split tests with Python, R, the CausalImpact package and Google Tag Manager.

Stratified Sampling Using Google Analytics + Python

For SEO Split-testing, you need to create groups of pages. In this post, you will learn how you can set up test and control groups for your SEO A/B tests.

SEO Split-Testing Experiments using Google Tag Manager

With Google Tag Manager, you can modify meta tags, a very useful feature to set up your own split-testing SEO experiment. Not-so reliable technique?

CausalImpact for SEO in Python

pyCausalImpact is a Python wrapper for the original causal inference machine learning package built by Google.

CausalImpact for SEO in R Programming [Complete DIY SEO Experiment]

CausalImpact is a machine learning package built in R that lets you compare predicted data with actual data. Here is how to run CausalImpact for SEO Split-testing Experiment.

Chapter 8: Machine Learning

Machine learning can be used to improve SEO in many different ways. From natural language processing to classification and predictions, many libraries are available to add to the SEO toolkit.

Machine Learning for SEO

Machine learning provides a lot of opportunities for SEO professionals to add to their toolkits. In this post, we will see how SEOs can leverage machine learning in their daily work.

Get BERT Score for SEO (by Pierre Rouarch)

Since Google has open-sourced its Natural Language Processing pre-trained machine learning model, SEOs have been all over the place. This simple API can be used to get the BERT score of a page for a specific Keyword.

OpenAI Notebooks for SEOs (External Link by Kristin Tynski)

Kristin Tynski from Fractl shared a ton of here work on marketing tools made using the OpenAI API. I have followed her work with eagerness and was very excited when she shared her Github Repository containing 22 notebooks helpful for SEOs! Huge thanks to her for her dedication!

Keyword Topic Clustering with Python SEO (Sklearn TF-IDF + AffinityPropagation)

Here, we will level up our coding skills by learning how to group keywords into topic clusters using Python and the Scikit-learn library. This keyword clustering technique is a useful practice to improve Python SEO skills.

Chapter 9: Web Development

Deploy a Flask App on Heroku (Deprecated)

Heroku lets you deploy a Flask app online for free.

This is a fantastic option to test dev environments, to make SEO tests in Google Search Console or to make a python a public app to showcase your work without paying for hosting and for a domain name.

Get Started With Django

In this step-by-step tutorial, you are going to learn how to make the most basic Django Application with Python.

Other Useful Guides For SEO

• Useful Python Functions for SEO
• Create Feature Image With Python (Pillow)
• Beginner Guide to RegEx For SEO
• Massive Python for Datascience guide – By Koray Tuğberk GÜBÜR (external link)
• How to Make a Twitter Bot With Tweetpy – By Miguel Garcia (external link)
• Use Custom Search Engine with Python – by Elias Dabbas (external link)
• How to get un-sampled Google Analytics Data With Python (& without) – By Dom Woodman (external link)
• Basic SEO Data Analysis With Python – by Hülya Çoban (external link)
• Google Trends API Wrapper (Pytrends)
• Get the most out of PageSpeed Insights API with Python (by Daniel Heredia Mejias)
• Projects and Resources to Learn Python SEO

About Python Programming Language

This is the end of the original guide on Python for SEO. Remember to share and link to this post if you enjoyed it.

Jean-Christophe Chouinard

SEO Strategist at Tripadvisor, ex- Seek (Melbourne, Australia). Specialized in technical SEO. Writer in Python, Information Retrieval, SEO and machine learning. Guest author at SearchEngineJournal, SearchEngineLand and OnCrawl.