A Fair Comparison Between SEO Crawlers: JetOctopus and Screaming Frog

In this article, Screaming Frog and JetOctopus SEO Crawler technologies and pricing will be compared, and it will be stated which SEOs can choose for which types of SEO projects for what reasons.

There is no marketing purpose or guidance in article writing. The aim of the article is to convey similar and different visions of different SEO Technologies to make the SEOs’ preferences more efficient. At the same time, SEOs need to have this type of comparison articles in order to make SEO Crawlers’ technologies better. For this reason, I suggest you read the article, not with the question of which is better, when to choose which one, and what else SEO software should have while SEO changes.

Before starting, you might wonder who is talking. Because as John Rupert Firth says, “You shall know a word by the company it keeps“. And I say, “A word gains trust via its source.“

My name is Koray Tuğberk GÜBÜR, and I am a Holistic SEO Expert. I own an SEO Agency called Holistic SEO & Digital. I manage SEO and Digital Marketing Campaigns with different aspects from Front-end to CRO, UX, or Data Science. My motto is  “Think like an SEO and Code like a Developer“.

From the first day of my internship, I am a user of Screaming Frog, and in this context, I strongly recommend you to read this tutorial.

So let’s get started.

Log Analysis for SEO and Crawler Choice for Log Analysis

Log Analysis is one of the ways for SEO professionals to listen to Google’s communication with your website. By performing Log Analysis, you can easily see which Search Engine’s in which type of crawlers scan your website. You can also divide the website into different segments to see which segment is crawled more, and see if there is a correlation with Google Search Console Data. So, if you are an SEO, which crawler should you choose in which perspective?

 

JetOctopus gets ahead at this point for two reasons. First and foremost, JetOctopus can combine Google Search Console Data and data from Log Files.

Second, JetOctopus does not require a set-up fee for the Log Analyzer. Although Screaming Frog is very reasonable and generous with its “annual payments”, the separate sale of Screaming Frog SEO Spider and Screaming Frog Log Analyzer puts JetOctopus ahead in this case.

Both SEO Crawl Technologies have a great advantage in terms of Log Analysis. But, JetOctopus has a more granular view, more descriptive charts, and data correlations for SEO.

On the other hand, Screaming Frog Log Analyzer provides some graphs and aggregation data based only on the data contained in the log file, but nothing more.

You can understand this just by looking at the number of Menu, Filter, and Chart in both tools. JetOctopus has the following menu items.

• 12 separate data charts in stages from 3 hours to the last month.
• 15 separate charts for Google, Bing, and Yandex Crawlers.
• With 11 different data charts and information points, insights interpreted with Organic Search Data for Google, Bing, and Yandex.
• Organic Search Visits for Google, Bing, and Yandex
• 4 data charts for problems encountered by Search Engines during scanning.
• 3 separate data charts showing the differences between fake, malicious bots and real Search Engine Crawlers.
• Log File Management section showing which bot visits which page and how many times a day
• Tables, charts, and data points with Status Codes, Request Size, Timing, Ajax URLs.
• Filtering of never crawled pages, slow pages, heavy pages, 1 time crawled pages, and Organic Performances
• Filtering all the above data for real users.
• Regression data prepared for pages that are valuable in terms of SEO

Lastly, JetOctopus has a unique mode for uploading Log files into its system. In Screaming Frog, you can perform a log analysis for only a certain timeline that is coming from your log files. But, what will happen if you have 5 different log files for 5 different months? For Screaming Frog Log File Analyzer, you will have to unify these files. And, some SEOs may not succeed to do it properly…

But, in JetOctopus, you just can “append log files” from old to new for a log analysis for a broader timeline.

 

Also, if you have duplicated log file rows for the same date, you can skip these dates or you can hold those rows too, it is up to you. I believe this is also another key difference.

So, if you want to save money, time and also get more SEO insights based on data blending, JetOctopus would be the right choice.

Since JetOctopus has more than 400 unique insight creative charts and tables in total, and Screaming Frog has many different data tables at a granular level, in order to keep our comparison study clean and readable, we will try to compare with fewer details in the following sections.

CPU Usage Comparison for SEO Crawls

SEO Crawlers can consume a significant amount of CPU while crawling. Therefore, CPU consumption should be considered when choosing a Crawler for an SEO Project. In Python, you can scrape a website while extracting certain information from a website, but in a short time, you will see that it is not the best option in terms of CPU usage.

Thus, usually, this type of task can be performed over a VPS Server or a Cloud-based system. In terms of CPU, JetOctopus should be the more efficient option, since it is a cloud-based system, you don’t need to worry about your CPU Usage.

Crawl Data Storage and Comparison Between Different Crawls

If you want to store your Crawl Data after your crawl has been finished and also if you want to make a comparison between different crawl reports from different dates, JetOctopus again is the more efficient choice.

Screaming Frog SEO Analyser actually doesn’t record your crawl report unless you export it. Also, if you want to store all of the information from your crawl, only one crawl can take 7 or 8 GB if the website is big or you store all the HTML Codes and Rendered Page View, etc.

In this section, I may say that JetOctopus also is a better option because, you actually don’t need to tell “keep these codes, information or rendered webpage” one by one, it is the default setting.

Bandwidth Usage Comparison for SEO Crawlers

JetOctopus uses its own internet and server resources when you perform an SEO crawl. Screaming Frog, on the other hand, uses the machine it is installed on and its network.

We can summarize this sentence as follows.

• When you run an SEO Crawl with ScreamingFrog, your internet may slow down or your internet bill may be bloated.
• JetOctopus scans for you from its own server and internet resource.

Previously, one of my clients created an “endless-loop” because of a pagination error. During an SEO crawl, I spend 40 GB unwittingly, and therefore I remember I had to buy additional packages, I must say, we are not in a very good spot in this type of area in Turkey. So, if the internet is slow or expensive in your country, JetOctopus stands out even more at this point.

Desktop Application Advantages for a Crawler

Some SEOs may prefer to use a desktop crawler and log analyzer. I also had this habit until last year. It might seem sweet to have an SEO Crawler that you can access with just a single click without any browser interactivity. I know this, also I had other kinds of SEO Tools to help me as desktop applications. Such as ScrapeStorm, URL Profiler, or Email Extractor.

And, you might have a strong limitless network, a strong CPU, and also storage capacity. In this case, if you have this kind of habit might, using a desktop application can be better to choose Screaming Frog.

Crawl Configuration Options, Details, and Necessary Effort to Configure an SEO Crawl

Crawl Configuration Options, Details, and Necessary Effort to Configure an SEO Crawl

To be honest, I don’t like to configure a crawler. How fewer questions asked to an SEO during the Crawl Configuration, it is better. Most of the crawl configuration questions have only two advantages.

• First, it can decrease the necessary time for crawling.
• Second, it can decrease the necessary resources for crawling.

So, why should I care about these questions during a crawl? Imagine that, there is a crawler that doesn’t limit you and just ask you which data sources you want to blend in with crawl output for better SEO Insights…

And to be honest, these questions are actually not for “Crawl Configuration.” It is for “Crawl Limitation”. Because most of the crawlers don’t have enough resources, even if they have enough resources, still there is a marketing and profit effect of pricing the configuration of a crawl.

Screaming Frog doesn’t bill you according to your crawl configuration but to be honest, it asks too many questions because during the crawl it consumes your own resources like network, storage, and CPU.

On the other hand, JetOctopus doesn’t ask these types of configuration (limitation) questions. Imagine that a crawler asks you whether you want to enable or disable Javascript during the crawl… Why should I want to disable it? Apparently enabling it will give more information for extracting SEO insights.

Note: A good SEO Crawler can see the difference between Javascript Enabled and Javascript Disabled versions of a website with just one crawl.

Crawl Speed for SEO Crawls

Crawl Speed is another aspect of crawlers. I personally don’t give so much weight to speed criteria but we all know that this review has been written for you. If you have multiple projects you might need to perform multiple crawls at the same as parallel to each other.

If you want to perform multiple crawls at the same time, the right choice is JetOctopus.

And, JetOctopus is actually the fastest SEO crawler on earth at the moment. Since Screaming Frog’s crawling capacity is actually about your own device/network capacity, I, unfortunately, can’t give credit for this section.

In both crawlers, you also can determine the crawl thread and affect the crawler’s speed. 

Note: If a crawler doesn’t use a quality server and IP Address, a fast crawler might create overloading on your web server. The server might refuse the crawler because it can think that the SEO Crawl is actually a DDoS Attack. Thus, creating a balance between Speed and Server Health is actually art for an SEO Crawler.

While saying good things about an SEO Crawler’s speed, these should be considered, but until now I didn’t encounter such an error with the “fastest SEO Crawler, JetOctopus”.

An SEO Crawler with SEO Advice

SEO Crawlers are not just data extractor tools today. They also point out improvements that can be easily made by offering Data and Insight. Screaming Frog actually has some similar features with its Crawl Analysis feature, but these analyzes are very basic and limited.

The Configuration options Screaming Frog has to perform analysis after crawl are as follows. At this point, although I find the Internal Link Score option included later to make sense, the remaining options do not fully offer an unusual feature. Because these are all data that should be extracted during a normal SEO Crawl.

 

On the other hand, JetOctopus has its own “SEO Insights” section.

 

In this section, if you are not familiar with SEO, you can take advantage of ready-to-go SEO Insights that are unique.

 

The most valuable quick insights from JetOctopus’ SEO Insights are listed below. 

• “Zero Click Pages”, 
• “Only one inlink Pages”, 
• “Non-indexable pages with GSC Impression”, 
• “Indexable Pages without Impression”, 
• “Pages with Impression not in Crawl”, 
• “Non-200 Status Code Pages with GSC Impressions”.

Since, I assume you are already an SEO and Digital Marketer, I won’t give the definitions and importance of these terms but if you are not familiar with SEO, you should know that these are important insights that an SEO Crawler can give to anyone.

And it is not finished yet… This was only the one section of SEO Insights. Below, you can see SEO Opportunities.

 

With Log Files, JetOctopus can make Log Analysis for you at some level and extract insights by blending Google Search Console and SEO Crawl Data with Log File Data. In this section, we see these are listed.

• “High bot visits with poor interlinking.”
• “Pages with one interlink and one bot visit.”
• “Well-interlinked web pages and low bot visit.”
• “Zero click pages on the first page of SERP.”
• “Candidate pages for the First Page of SERP.”

Also, with the KPI Section, you can determine your own KPIs.

With changing the date, you can create even more insight and also JetOctopus has another option.

Comparing Different Crawls from Different Times

In Screaming Frog, you perform a crawl, after that you can export it but you can’t compare two different crawls in the Screaming Frog. Of course, you can create some comparison graphs via Google Sheets and Data Studio, but it wouldn’t be the score for Screaming Frog.

In JetOctopus, you can check every detailed data about a website to see what has changed during the time between two scans. JetOctopus allows comparing the results of different crawls that have been performed at different times in the following areas.

• Indexed Web Page Data
• Web Page Loading Time Data
• Status Codes
• On-page SEO Elements (HTML Tags)
• Content
• internal Links
• External Links
• Anchor Texts
• Sitemaps

Even if there is more comparable data, it is sufficient to say this much.

Changing Dynamic Data Graphs Based on Different Dates

In Screaming Frog, you have a kind of static information, you can’t change the data graphics according to the different dates. In JetOctopus, you can choose different dates for creating comparisons or focusing on different times in a year. Thus, finding SEO Opportunities or improving new SEO Insights is actually easier with dynamic data charts based-on date filters.

You may store and choose to examine past crawls, and also you can integrate Google Search Console Data with a broader timeline as below.

 

While the subject has come to Google Search Console, it is necessary to make a comparison at that point.

How Should an SEO Crawler Use Google Search Console Data?

Google Search Console Data is an opportunity to make conclusions in terms of SEO performance and its relation to the Technical SEO Realities of a website during a period of time. Also, you may use Google Analytics, Google Trend Data along with your marketing and branding campaign data to aggregate and correlate to understand the Google Algorithm.

You can see how you can use the Google Search Console Data for SEO with JetOctopus with just menu items..

Screaming Frog connects to Google Search Console with the help of API, but it cannot provide enough insight with the data here. On the other hand, JetOctopus can make Google Search Console data interpretable with all Technical SEO realities and adaptable according to different dates and past search data.

Above, you see an example screenshot from Screaming Frog’s GSC Data Interpretation style. It is actually not an interpretation and it doesn’t include any SEO Insight, unless you extract them. You also can use Google Search Console for this. Below, you will see JetOctopus’ system for Google Search Console Data Interpretation..

This is just a subsection of the sub-section. It shows the Click data based on Position and Ranking CTR Relationship. Below, you will see Query Performance based on Word Amount in a Query and CTR, Click, Impression Data.

We see that most of your Click and Impression comes from queries with 4 words. It means that you can optimize your SEO Tags, Content Structure, internal and external link texts according to these terminologies. You also can use JetOctopus’ expert-level descriptive visualizations.

Above, you see CTR and Position distribution for aggregated queries of a web site and its CTR, Position relation. It is a nice insight to see where to focus more.

Keyword Cannibalization Explorer by an SEO Crawler

Concepts such as Keyword Cannibalization, Ranking Signal Dilution and Mixed Ranking Signal are important concepts for SEO. I have previously published an SEO Case study with JetOctopus that explains all of these concepts with SEO Theories real-world SEO obstacles. One of the reasons I compare Screaming Frog to JetOctopus is to help increase such cases in the SEO World with a more advanced SEO Vision.

To read Importance of Ranking Signal Dilution and Consolidation of Ranking Signals SEO Case Study, you may follow the link. In Screaming Frog, you can’t find anything related to the Keyword Cannibalization for an URL but in JetOctopus, you might find them easily.

And, all of these URLs can be used for other types of SEO Audit related tasks.

With a single click, you may see URL-based metrics for any kind of SEO related audit. Below, you will see an Internal and External Link report for a specific URL that I have chosen from the Cannibalization Report of JetOctopus.

Winning and Losing Pages Report from an SEO Crawler

Unfortunately, in Screaming Frog, we don’t have a report for the losing and winning situation of web pages for different dates and their comparison based on Google Search Console Data. We actually have just the sections below.

But, JetOctopus serves a different level of unique data charts and explanations for a web site’s situation. Normally, getting these insights can take hours for an SEO since it requires lots of manual work. In Python, you can automate these kinds of Google Search Console Report templates and calculations but most of the SEOs don’t know coding. So, the argument is still valid for most of the industry, here. Below you will see a win/lose comparison for different dates in JetOctopus.

Below, you can see the winning and losing web pages’ change graphic based on GSC Impression and Clicks Data with JetOctopus.

The censor here is for protecting the classified data but the main message can still clearly be seen.

Below, you can see the winning and losing web pages breakdown according to their drop and growth situations.

Keyword Report of SEO Crawlers based on Google Search Console

With Google Search Console, we can see in which queries a website goes up or down. Screaming Frog does not offer an advantage at this point, but JetOctopus can provide the following data.

• All Kewyord Data
• New Keywords
• Lost Keywords
• Zero Clicks Keywords
• Winners
• Losers

Below, you will see the all keywords data, you can see the change for every column based on a query and landing page match.

And, one of my favorites is actually below, “Zero Click Keywords”. It is a great source for SEO Opportunities.

You may choose to focus on these queries and search intents, you can consolidate these queries according to the main topic and search intent for creating specific web pages within a semantic structure in terms of Semantic SEO.

Lastly, you may see how many queries that you have an impression with how many indexed pages, for everyday.

And, all of this information is also available for based on web pages.

You can see the results for the web pages’ aggregated data for Google Search Console dimensions according to their last situations and compared dates.

Below, you may see an example animation from JetOctopus’s Google Search Console, All Web Pages report. My favorite section from this report is actually the “query amount”.

I know I have given too much weight on Google Search Console Data Interpretation, but this is one of the major needs for an SEO and also a key difference between Screaming Frog and JetOctopus.

And, I know in this section, I couldn’t talk about Screaming Frog, and it can be seen as “unfair” in your eyes, but as I said before, Screaming Frog doesn’t have an equivalent level of tool for this task, unfortunately.

SEO Related Explanations and Guides in User Interface

Having some helpful explanations on the user interface explaining technical SEO terms, metrics, importance and graphics to the user means that the SEO Crawler also educates the inexperienced website owners. Screaming Frog has an excellent Technical SEO Blog, so I can’t criticize them on this topic negatively.

But, you know that an inexperienced web site owner probably won’t read so much article, because they need key points and key insights in terms of SEO. So, I can say that in this section too, JetOctopus is a better option, if you don’t know what to do or what is what for SEO and Search Engine Algorithms with which reason.

You also can see the Screaming Frog’s detailed blog to learn about Technical SEO.

Creating Custom Dashboards with Extensive Amount of Filter Variations

An SEO Crawler can collect lots of data from different fields. It is important to have a detailed filter and export system for creating more insight and convincing the customer’s IT, Content and Marketing team along with Product Managers, CTOs and CMOs… (Seriously, we as SEOs have too many people in our lives to convince for the sake of our businesses’ goodness… I know that you feel me…)

Both Screaming Frog and JetOctopus actually have good enough filtering and exporting systems.
In Screaming Frog, we have an option as “Bulk Export” to export things we wonder about a website.

Also, in Screaming Frog, we have horizontal sections for filtering data such as Meta Descriptions, Meta Keywords or Canonical URLs. But most of these actually just change the order of the columns so that we can see the data we want first. When we choose an URL, we also have these filters at the bottom of the program’s window.

And we can export the data we need from this section, as we want. And, we have search bars so that we can filter other kinds of columns or rows as we want. So, even if Screaming Frog doesn’t have an improved “filter system” it actually has a button for everything to export. So, I can’t say that they don’t provide enough and I think Screaming Frog has a proper exporting and filtering system.

In JetOctopus, there is a “button-less” filter system. You can choose the section you want from the SEO Crawl Report so that you can see the actual related data, not everything to be sorted with a button.

And, you have columns, conditions and actual selections for aggregation, correlation and customization of a data table or graph.

For instance for Web Pages’ Google Search Console, you can add Bot Logs or Sitemap URLs as you want. You also can create your own KPIs.

You also have more than 20 special filters for every section according to the section’s nature and purpose.

And you have every kind of rule and logical operand with full freedom. Along with an actual date filter…

And lastly but most important: you can create thousands of different data tables, you can change data columns, add data columns or remove data columns as you want, also you can create comparing columns for different columns…

While I was starting this section, I thought I can give a win to Screaming Frog here, but I am leaving the end decision to your perception. If you want to perform a detailed SEO Strategy, you can take more from JetOctopus with less. You can use your creativity, instead of pressing buttons like it is 1999.

Btw. I’m not joking, the filters are changing for every section, for instance, this is for images.

So, instead of searching for a button and scrolling horizontally for finding the right filter, you just have the related filters for every section.

Custom Data Importing and Integrating into SEO Crawl Report

You think that this section is over right… But not. In Screaming Frog, we can’t integrate different and custom datasets into the SEO Crawl Output. You can do it in Excel or with Python, but not with Screaming Frog.

On the other hand, you can import any kind of data you want to an SEO Report.

In my data section, you can upload your CSV or other kind of data such as a Backlink Report, or “Brand Mention” report etc. to correlate and compare SEO Crawl Data for extracting more insight.

After you load your own dataset, you will see that you can choose your own column names and value names as you want. For instance, below, you will see that I have imported Ahrefs Backlink Report to the JetOctopus’ SEO Crawl Report.

Another key difference for any SEO…

Data Presentation Styles of SEO Crawlers

Some SEOs like working with datasets and data tables and some like working with descriptive data graphics with interactivity, and some SEOs love both! A good SEO Crawler should present both options. I believe that since the beginning of the review, you may see that JetOctopus has a better visualization and also tabular data presentation style. It is more descriptive and also interactive.

And this is a visualization from Screaming Frog, you need to turn your head 90 degrees to the right so that you can read the X-axis. And a graphic from JetOctopus is below.

 

You can do math on your own.

Security Audits of SEO Crawlers

For every SEO Crawler, security actually shouldn’t be an option. Since, it is a technical issue and also affecting the SEO, having a security audit system in an SEO Crawler can be a chance.

In this Section, we will favor the Screaming Frog, because JetOctopus doesn’t provide any kind of security audit for a website.

Screaming Frog performs the security audits below for every URL on a website, as below.

• HTTP URLs
• HTTPs URLs
• Mixed Content
• Form URL Insecure
• From on HTTP URL
• Unsafe Cross-Origin Links
• Protocol-Relative Resource Links
• Missing HSTS Header
• Missing Content-Security-Policy-Header
• Missing X-Content-Type-Options Header
• Missing X-Frame-Options Header
• Bad Content Type

For instance, in my website, I don’t use X-Frame Options and it is for protecting websites against security attacks via Iframes. So, I have learned this from Screaming Frog SEO Spider.

Grammar and Spelling Checks of SEO Crawlers

Grammar and Spelling Error Checking is actually an SEO Task. It affects your website’s E-A-T (Expertise, Authority, Trust) and also it affects the User Experience. And, it is a factor for your brand’s positioning, it will affect the users’ perception towards your brand. In this section, we also need to favor Screaming Frog, it has a grammar and spelling error checking section. But, again I can’t say that it always works well, because you need to specify the HTML Div that includes the main content of your web page.

And, it doesn’t work for every web site. But still, I think it is a great step for SEO Crawlers, so we can say that grammar and spelling error audit is a good plus for Screaming Frog SEO Spider.

After you have chosen Spelling and Grammar Check options, you need to specify the grammar and spelling check rules.

You also can choose the words for being ignored.

Below, you will see the Spelling and Grammar errors from my web site.

Screaming Frog says that my name is an error in terms of Grammar or Spelling but actually, they are not errors. So, as you can see that some of them are really errors, and you can find them to fix via Screaming Frog.

HTML Error Detection and Validation

HTML Errors can make it difficult for Search Engines to understand or crawl the website, but as John Mueller said:

“Google works on the web as how they find.”

John Mueller

This means that even if a website has HTML errors, Google can understand it and crawl it properly. However, it would be better, in any case, to clean up HTML errors that make web browsers slow down while rendering a web page and create more overhead for Search Engines.

If so, it would be helpful if an SEO Crawler could find HTML Errors and tell you.

In Screaming Frog we don’t have any option for “bulk HTML Validation”. But we can validate every web page’s HTML Errors with right click via W3C Validator.

In JetOctopus, we don’t have a bulk HTML Validator either, so in both cases, this can be an improvement call. In this section, we might say that Screaming Frog has a slight boost since it allows users to validate a web page’s HTML format with one click.

 

Above, you can see an HTML validation checking example with W3C Validator.

Page Speed Tests and Performance Information by SEO Crawlers

Performing Page Speed Tests should be done by an SEO and IT Member or an SEO who knows to code. Thus, an SEO Crawler also should focus on both. This is actually my Holistic SEO definition and vision but it is the story of another time. Page speed is actually a science, and it should be examined carefully with repetitive tests and different page layout, resource, and code scenarios. But, SEO Crawlers can help us to perform scaled Page Speed tests site-wide.

In JetOctopus, we actually have lots of information for Page Speed. But, most of this information is actually not oriented according to the advanced page speed metrics, they are mostly the response time and response size of the targeted website’s server during the crawl.

You can see the response times and response sizes from JetOctopus above.

You can see the Size Distribution of pages’ responses to the crawler. This is also similar to the Screaming Frog’s presentation. Below you will see response times according to the page depth (click depth).

Below, you will see the page size by depth from JetOctopus.

But, Screaming Frog has actually more in this area via Pagespeed Insights. Screaming Frog can use Pagespeed Insights API with your key for scaling the page speed tests and give diagnostics and suggestions for improving web pages’ situation in terms of speed.

Actually, you can perform the same thing with Pagespeed Insights API itself, and also all of these requests are coming from your own key and Google account. But, still it is important. Screaming Frog has all of the Page Speed Insights data dimensions for webmasters as below.

• Eliminate Render-blocking Resources
• Properly Size Images
• Defer Offscreen Images
• Minify CSS
• Minify Javascript
• Remove Unused CSS
• Remove Unused JS
• Efficiently Encode Images
• Serve Images in Next-gen Format
• Enable Text Compression
• Reduce Server Response Time
• Avoid Multiple Image Redirects
• Preload Key Requests (Doesn’t work on Firefox, remember.)
• Video Format for Animated Content
• Avoid Excessive DOM Size
• Reduce Javascript Execution Time
• Serve Static Assets with an Efficient Cache Policy
• Minimize Main Thread Work
• Text-visibility During Loading
• Request Errors

You may see that Screaming Frog uses the Pagespeed Insights API and extracts all of the Page Speed Insight diagnostics for URLs in a bulk way and tells which improvement will speed up the web page. As I say before, this actually can be done via PSI itself, but still having this kind of possibility on Screaming Frog is a positive side. In this section, JetOctopus also has a slight boost we need to mention,

JetOctopus can correlate Google Search Console Data, Custom Data you choose with the Page Speed. So, even if it is response size and response timing, still there might be some insights in certain situations such as excessive server latency or excessive amount of response headers which is not easy to find for every SEO.

Structured Data Test by SEO Crawlers

Structured Data is another aspect of SEO. Structured Data can be used for giving more information to the Search Engines in a deeper perspective. It says the content’s type, author’s identity, content’s sections and purpose/topic of every section along with web page’s images and functions. So, implementing Structured Data in a non-correct way or not auditing the competitors’ websites for making comparison would be an important mistake.

SEO Crawlers, today can give us information about Structured Data. In JetOctopus, unfortunately we don’t have any section about Structured Data extraction and validation. But, in Screaming Frog we can extract Structured Data from all of the URLs and perform audits for them.

For single URLs, you can perform Structured Data Validations and see the results with Screaming Frog.

Screaming Frog uses Google’s or Schema.org tools (Rich Snippet Results Testing or Schema Validation)  for Structured Data testing. So, actually you can perform these tasks by yourself too, but like in the previous Page Speed section, having everything together makes things easier.

Above, you will see all of the Structured Data information from my web site.

Word and Content Analysis by SEO Crawlers

Before beginning this section, I must say that word amount is not a ranking factor. But, in some cases, word factor in the main content can be a clear signal for Google to choose the main content and differentiate it from the boilerplate content. Also, word amount and information amount are usually correlative to each other.

For SEO, an article should be as short as possible.

For SEO, an article should be as long as necessary.

And, Word Amount in a Content and Average Word Amount in Title Tags can show clear signals for making SEO Analysis between competitors. In this context, we need to favor JetOctopus, naturally.

JetOctopus filters all of the thin content from a web page within a correlation with Google Search Console, Google Analytics and Internal Link Data. Of course, you can also use filters according to the word amount, you can try to see whether there is a correlation between word amount and web page performance.

In this example, I don’t have any web page with thin content and that’s why I can’t show any results here but it is nice to see that JetOctopus has a different perception for Thin Content and Low Content Words.

And, you can see the quick filters and other types of filters in the screenshot above. Performing an SEO Analysis for the Content and Word Amount to see whether a correlation exists or not is possible.

JetOctopus also has another unique perception for “words” in HTML Tags for SEO. It tries to calculate the average “unique” words in the HTML Tags related to SEO.

And, we know that from the Moz’s Title Tag experiment, any word, letter or expression that is not relative to the brand name or the topic in the title tag affects the SEO Performance negatively. Above, you see the average unique and non-unique words in Title Tags and Meta Descriptions. Below, you see the same information for the Heading Tags.

Making a comparison between competitors’ websites can show some clear signals. Also, if you click one of those bars, you will filter all of these URLs.

So, you may check all of these web pages’ Organic Performance, to make some changes and then check the organic search result changes, by watching for the SERP for every Search Engine, of course.

SEO Crawlers and Backlink Correlation Reports

Backlink crawling and aggregating or creating special metrics for backlinks are not specialties of Technical SEO Crawlers. But, SEO Crawlers can perform backlink analysis also. It is not hard for them. In this section, both Screaming Frog and JetOctopus have really nice features. But, Screaming Frog uses “APIs” as always for Backlink Data aggregation.

And, It can use only Majestic, Ahrefs and Mozscape APIs for backlink data aggregation. This is not the optimum choice for many reasons. Firstly, Screaming Frog will spend your own credit for Backlink Audit whenever you call a request from the APIs of these Backlink Tools. Second, these APIs and their membership may not be cheap for you. On the other hand, you can simply import your own dataset for backlinks to the JetOctopus’ “Data Importer” module so that you can use them for SEO Analysis. You don’t have to pay for an API and even if you don’t have any kind of backlink tool membership, you can request it from someone and still you will be able to aggregate the backlink data with other Technical SEO and Organic Performance Data sets.

So, because of this difference, we need to favor JetOctopus in this section. If you don’t agree with me, follow the below.

First, import your backlink data to the JetOctopus.

Then, give a name to your file and remember to specify the “URL Key” column. It should be your “Join Key” like in Data Studio, remember?

Now, JetOctopus will read your mind.

As an optimized SEO Crawler, JetOctopus clearly asks you which data set you want to correlate this data set with. I have chosen the Google Search Console Pages. See the result below.

I have a query amount, URL Rating, Total Backlink Amount, Internal Link Amount. Do you want more? Click Setup Columns as I told you before.

And we have a unique and customized dashboard now.

In Python, with data blending you can do this simply, but for most of the SEOs these tasks are not easy to do. With JetOctopus, it takes only 30 seconds. Because of this reason, we have to favor JetOctopus, naturally.

Indexation Check by SEO Crawlers

Indexation Check. Actually, we can guess the outcome of this section basically. Since, I have covered lots of things from the both of the SEO Crawlers. Screaming Frog has some HTML Tag audits for indexation such as “hreflang”, “canonical”, “noindex”. Also, it marks every URL as “indexable” or “non-indexable” according to these tags and status codes.

You may see an example above for the Indexability check of Screaming Frog. Besides this section, Screaming Frog doesn’t have a special area for indexability. It also has a section in Google Search Console for “impression with non-indexable page” but it is just a one-click area more than a section, actually. In JetOctopus, there is a special area for just Indexability.

In this section, also there are sub-sections with more than 30 data graphs and also lots of filters, custom columns, data sets and data aggregation and correlation possibilities. Indexation Problems can be checked according to the Click Depth with JetOctopus.

Also, you may categorize these indexation problems and with just one single click, you can filter them for further investigation.

Below, also you can see the Robots Tags’ distribution.

Below, also you can see the Robots Tags’ distribution.

Because of these quick filters, detailed segmentation and variated graphs, we need to favor JetOctopus in this section.

Web Site Segmentations and SEO Crawlers

What is website segmentation? For Google or other Search Engines, your website is actually just the sum of different web pages. Sometimes, the Search Quality Team even doesn’t care about the source or the domain, if the web page is quality enough, it can be enough to rank well too. But, also web pages’ characters are important. Because they have different characters. All web page categories have similar and mutual points such as design, function, products on them, or content on them. So, segmenting a website is important for Search Engines to understand the source of web pages.

But with which method? According to the URL? According to the Breadcrumb? Words in title tags? User behaviors on the website? Internal links? Header menu categorization? Or just contexts of words according to Natural Language Processing in the menu? There are lots of signals and methods for segmenting a web site.

So, if you want to have a proper SEO Crawler, you should be able to create web page segments with this crawler. No words needed here for Screaming Frog, because you can’t create different web page segments with Screaming Frog.

But in JetOctopus, you can create different web page segments. And, all of the data aggregation, interpretation, SEO Insights, crawling process, and perception of the website would change. JetOctopus can segmentify your website according to your internal link structure and URL structure, but if you think or want, you can customize the segmentation according to your project’s special needs.

By pressing “New Segment”, you can create new segments and groups in segments with new rules. Because of this flexibility, we need to favor JetOctopus.

Live Support and SEO Crawlers

Live Support is another important subject for SEO Crawler. You may encounter a bug or you may want to learn a specific thing. In JetOctopus, if you buy the full package, you have a “training call” chance. We don’t have this type of chance in Screaming Frog.

Also, on JetOctopus’ official website, even if you are not a subscriber or member, you can always use live chat to ask questions.

In Screaming Frog, we don’t have a live chat option, unfortunately.

SEO Crawlers, Internal Links and Anchor Text Comparison

Both of the SEO Crawlers have special sections for anchor texts. Anchor texts are contextual text fragments that show the purpose, topic and contribution of the link between two web pages for the user. It carries a value and meaning for the Search Engine Crawlers. Positioning them and using them naturally help SEO. In this context, an SEO Crawler should have a special section for anchor texts. In Screaming Frog, we can export all of the anchor texts from a web site.

And all of these actually show important data.

But, all of them are in an excel file. So, you need to dig deeper to create insights from now on. JetOctopus carries this further. For instance:

Sum of internal links, sum of image links and sum of internal no follow links along with links to not-indexable web pages. Internal links to the disallowed web pages can be seen. And also their anchor texts can be extracted.

And, there is another unique SEO Metric from JetOctopus, here. “Unique” anchor text. I have written this in my Hangikredi SEO Case Study. If you don’t use internal links naturally and conveniently for the user-flow, Google Algorithms will ignore them. Also, you need to use Anchor Texts, contextually and naturally. If you don’t do that, Google algorithms will ignore them too.

So, you may see and compare your unique anchor texts and your competitors’ anchor texts according to their uniqueness status. Also, there is a sweat warning there “This chart is not clickable for now”, so you will be able to see them soon. You may see the anchor text word amount below.

Anchor texts by page depth can be seen below.

And also, we can see unique image alt tags with the same logic.

We also can see the “relative” and “absolute” URL distribution within internal links.

So, most of these metrics are unique and also they can be seen with interactive graphs. We can filter them with only one click. In this context, we need to favor JetOctopus in this section.

Last Thoughts on JetOctopus and Screaming Frog Comparison

During this review, I mostly favored JetOctopus because it is more flexible and has more unique sides. I also favored some sections of Screaming Frog because JetOctopus doesn’t have an equivalent of these features. Both of these SEO Crawlers are actually advanced and well enough to satisfy a user. And, I even didn’t cover lots of features from both of the crawlers despite this article’s excessive length. Screaming Frog can send you into the wayback machine for any URL within a click, and JetOctopus can extract links for social media accounts or calculate link amount by page depth. I tried to show the key differences between both of the SEO Crawlers, if you think which one to buy.

I didn’t cover the Pricing. But, actually both of these crawlers are convenient in terms of prices.

JetOctopus can be used only for 20 Euros for a month. If you want to use everything from the crawler, you should choose 80 Euros for a month. It is not expensive based on its talents. And, the best part is that there is not a “domain limit”. There is a limit for the URL Amount but you can add many projects to your account.

And Screaming Frog is actually 248 Euro per year. It means that if you want to buy Log Analyser and also SEO Spider at the same time, you need to pay on average 20 Euros per month. So, actually you can pay 10 Euros more to have way much more with JetOctopus. But, still I can say that Screaming Frog is also another great tool with the best possible price for our industry.

During the review, I tried to cover both technologies and try to incite both technologies to make some improvements. If you need ready to go templates, insights, data filters and descriptive graphics with more granular data, you should choose JetOctopus, if you are an experienced SEO and you can spend time on excel files and Google sheets, you are also good with Screaming Frog.

I present my respects to both of these SEO Crawlers and Developers, Designers, and Minds behind them.

All Technical SEO Related Posts

Check out the in-depth list of technical seo posts.

• A Fair Comparison Between SEO Crawlers: JetOctopus and Screaming Frog

  

• Common Issues During Server Log Analysis

  

• Google Bot Crawl Budget Optimisation

  

• Silo Internal Linking

  

• Technical SEO Checklist

  

The full list shows the various technical seo strategies for ranking higher in Google SERPs.