Tag Archives: CloudFlare

Did Malaysia just block use of public DNS servers?!

Did MCMC just block the use of public DNS servers in Malaysia?! Here is what we know so far…

MCMC blocks public DNS servers only for business and government?

According to an FAQ posted by Maxis, the Malaysian Communications and Multimedia Commission (MCMC) has ordered all Internet service providers in Malaysia to implement DNS redirection for businesses, governments, and enterprises by 30 September 2024.

DNS redirection basically blocks the use of public DNS servers, by redirecting DNS queries to the ISP’s own DNS servers, where certain websites can be blocked by government directive. In other words – DNS redirection prevents people from circumventing the ISP’s own DNS servers (and the government’s block list).

The Maxis FAQ, which is titled Maxis Business DNS Redirection, states that ISPs in Malaysia were ordered to implement DNS redirection for business/enterprise/government users, blocking their access to public DNS servers.

However, it now appears that the directive may actually apply to everyone in Malaysia, not just business, enterprise, or government users. Or the ISPs may be implementing the directive across the board. Here is what we know so far…

Public DNS servers appear to be blocked in Malaysia!

As far as I can tell, MCMC has not officially announced its DNS redirection order to ISPs. All we are relying on so far is the Maxis FAQ on DNS redirection, which suggested that it applied only to businesses, enterprises, and government agencies.

Why is DNS redirection being implemented for Enterprise/Business/Government services?

DNS redirection is being implemented to assist in preventing the commission or attempted commission of an offence under any written laws of Malaysia or otherwise in enforcing the laws of Malaysia. By blocking access to harmful websites more effectively and quickly, this proactive measure helps ensure compliance. This is particularly important for Enterprise/Business/Government, as it will also reduce the risk of reputational damage and inadvertent commission of offence.

However, the Maxis FAQ also states, in a different section, that “all service providers must implement this measure, and it applies to to all users of their services“, our emphasis in bold below:

The implementation of DNS redirection is a regulatory requirement enforced by MCMC to ensure compliance with Malaysian laws and to protect users from harmful online content. All service providers must implement this measure, and it applies to all users of their services.

If that’s accurate, then the MCMC directive to block the use of public DNS servers may apply to consumer users as well, not just business, enterprise, or government users.

Regular Internet users in Malaysia have started reporting (here, here) that they are no longer able to use public DNS servers from Google or Cloudflare.

The affected ISPs so far appear to be Telekom Malaysia (Unifi), Time, and Maxis. Digi and Celcom appear to be unaffected … so far.

We tested on Unifi and Digi Broadband, and confirmed that we cannot connect to Google DNS (8.8.8.8 / 8.8.4.4) or Cloudflare DNS (1.1.1.1). You can test it out yourself using these methods:

Using any Internet browser

Using any Internet browser, just go to the Cloudflare public DNS website – https://1.1.1.1/, which will appear in your Internet browser as https://one.one.one.one/.

But if your ISP has implemented DNS redirect, both domains are no longer accessible. Our tests on Friday, 6 September 2024 show:

Unifi : Unable to connect
Digi Broadband : Connects normally

Using Traceroute / Tracert

For the more technically-inclined, you can try using traceroute (macOS / Linux) or tracert (Windows), to see if your computer can connect to your preferred public DNS server – for example, 1.1.1.1 (Cloudflare) or 8.8.8.8 (Google).

It’s pointless to use ping, because pings to the public DNS server IP will get redirected to the ISP’s own DNS server, and you will still get a response.

Using traceroute to connect to 8.8.8.8, our tests on Friday, 6 September 2024 show:

Unifi : Redirected

192.168.0.1 (192.168.0.1)
jhb-113-254-tm.net.my (203.106.113.254)
10.55.52.54 (10.55.52.54)
10.55.52.90 (10.55.52.90)
10.55.52.54 (10.55.52.54)
10.19.129.65 (10.19.129.65)

Digi Broadband : Not redirected

192.168.1.1 (192.168.1.1)
172.16.136.1 (172.16.136.1)
115.164.8.106 (115.164.8.106)
72.14.243.96 (72.14.243.96)
dns.google (8.8.8.8)

As you can see, the traceroute showed that requests to Google DNS (8.8.8.8) on the Unifi network were routed to a Telekom Malaysia server (10.19.129.65) instead.

On the other hand, the same requests on the Digi Broadband network still get routed to Google DNS (8.8.8.8) as intended.

So it does appear that some Internet service providers in Malaysia have started to block access to public DNS servers by redirecting DNS queries to their own DNS servers. However, it is uncertain if MCMC actually ordered those ISPs to implement DNS redirection for consumers.

The blocking of public DNS servers will not matter to people who don’t know how to use these public DNS servers. But it will matter a lot to Malaysian netizens who want to use public DNS servers for faster performance, better privacy and security, as well as bypass government censorship. Some will inevitably see this as a step towards China’s Great Firewall.

While the goal of blocking dangerous or scam websites is admirable, that is already served by existing bans of those websites on the ISP level for regular users. People who are tech-savvy enough to configure their computers to use public DNS servers would know better than to fall for those websites.

Hopefully, MCMC only issued that DNS redirection directive for business/enterprise/government users, and Malaysian ISPs will roll back the implementation for consumers.

Please Support My Work!

Support my work through a bank transfer / PayPal / credit card!

Name : Adrian Wong
Bank Transfer : CIMB 7064555917 (Swift Code : CIBBMYKL)
Credit Card / Paypal : https://paypal.me/techarp

Dr. Adrian Wong has been writing about tech and science since 1997, even publishing a book with Prentice Hall called Breaking Through The BIOS Barrier (ISBN 978-0131455368) while in medical school.

He continues to devote countless hours every day writing about tech, medicine and science, in his pursuit of facts in a post-truth world.

Recommended Reading

Go Back To > Internet | Business | Tech ARP

Support Tech ARP!

Please support us by visiting our sponsors, participating in the Tech ARP Forums, or donating to our fund. Thank you!

Malaysia orders DNS redirection for business and government!

Leave a reply

MCMC has ordered ISPs in Malaysia to implement DNS redirection for businesses, enterprises, and government agencies! Here is what you need to know…

Malaysia orders DNS redirection for business and government!

The Maxis FAQ, which is titled Maxis Business DNS Redirection, states that this directive by MCMC is meant to “protect users from harmful or illegal online content” by blocking access to “websites involved in online gambling, pornography, copyright violations, scams, and other illegal activities.”

Here is a quick primer on what DNS redirection would mean for Malaysian businesses, enterprises and government agencies.

What is DNS?

When you visit a website, you will normally key in a user-friendly name (like www.techarp.com), but the website itself uses a numerical IP address (like 192.0.2.1).

DNS translates that user-friendly name into the correct IP address, so your computer can access the website.

What is DNS redirection?

By default, most people use the DNS servers provided by their Internet service provider. However, some people choose to use alternative DNS servers from Google or Cloudflare for privacy, faster speeds, or to avoid censorship.

DNS redirection prevents that by redirecting all DNS queries from your phone or computer to the ISP’s own DNS servers, where certain websites can be blocked by government directive. In other words – DNS redirection prevents people from circumventing the ISP’s own DNS servers (and the government’s block list).

Is DNS redirection good or bad?

Blocking access to illegal, or dangerous websites is important for businesses, enterprises, and government agencies, because it reduces the risks of reputational damage, and prevents the inadvertent commission of offence by employees. Blocking scam websites is also an important way to prevent people and companies from getting scammed.

On the other hand, public DNS servers from Google and Cloudflare offer faster speeds with lower latency, and block tracking by ISPs or governments. They also offer enhanced security using DNSSEC – DNS Security Extensions.

Which Malaysian ISPs are affected?

According to the Maxis FAQ, this MCMC directive affects all ISPs offering both mobile and fixed Internet services in Malaysia.

Who will be affected by DNS redirection?

According to the Maxis FAQ, this MCMC directive primarily affects Business / Enterprise / Government customers who use public DNS services. But there are already reports that consumer DNS queries have also been redirected.

Interestingly, the Maxis FAQ also states, in a different section, that “all service providers must implement this measure, and it applies to to all users of their services“. So this MCMC directive may actually apply to consumer users, and not just business, enterprise, or government users.

Which DNS services are affected?

According to the Maxis FAQ, this directive currently affects public DNS services like Google DNS and Cloudflare. OpenDNS was not mentioned, but is likely to be affected too.

Private DNS servers, and encrypted DNS queries like DNS over HTTPS (DoH) or DNS over TLS (DoT) are not affected.

When must Malaysian ISPs implement DNS redirection?

According to the Maxis FAQ, MCMC ordered all Internet service providers in Malaysia to implement its DNS redirection directive by 30 September 2024.

Users on Time, Telekom Malaysia and Maxis networks have reported that DNS redirection is already in effect.

How to check if DNS redirection has been implemented?

DNS redirection is designed to block access to websites deemed illegal by the government of Malaysia. If you access a blocked website, it will be inaccessible.

You can also test by trying to go to the Cloudflare DNS website – https://1.1.1.1/ using your Internet browser. If you see “Unable to connect”, instead of the 1.1.1.1 website, then your DNS queries are being redirected to your ISP’s own DNS servers.

Alternatively, techies can use the tracert command (in Windows command prompt), or traceroute command (in macOS Terminal) to connect to the alternative DNS server (like 1.1.1.1 or 8.8.8.8), and see if it gets routed properly.

If the trace route shows your query ending in a different IP address than the alternative DNS server you selected (like 1.1.1.1 or 8.8.8.8), then it is being redirected.

Can you opt out of DNS redirection?

No, unfortunately – you are not allowed to opt out, because it’s a regulatory requirement by MCMC.

Please Support My Work!

Support my work through a bank transfer / PayPal / credit card!

Name : Adrian Wong
Bank Transfer : CIMB 7064555917 (Swift Code : CIBBMYKL)
Credit Card / Paypal : https://paypal.me/techarp

Dr. Adrian Wong has been writing about tech and science since 1997, even publishing a book with Prentice Hall called Breaking Through The BIOS Barrier (ISBN 978-0131455368) while in medical school.

He continues to devote countless hours every day writing about tech, medicine and science, in his pursuit of facts in a post-truth world.

Recommended Reading

Go Back To > Internet | Business | Tech ARP

Support Tech ARP!

Please support us by visiting our sponsors, participating in the Tech ARP Forums, or donating to our fund. Thank you!

Steam Is Suddenly Banned Entirely In Vietnam!

Leave a reply

Steam has suddenly been banned entirely in Vietnam! Here is what we know so far about this developing situation…

Steam Is Suddenly Banned Entirely In Vietnam!

Earlier this week, Steam appears to have been banned entirely across Vietnam, with Vietnamese players taking to Steam to complain about how Vietnamese internet service providers have blocked access to both the Steam app, and the Steam website.

Currently, neither the government of Vietnam, nor Steam, has commented on the matter. So it is unclear, at least officially, what led the Vietnamese government to ban Steam. However, it appears that domestic game publishers in Vietnam may be responsible for this ban…

Possible Reasons Why Steam Was Banned In Vietnam!

According to a translated story on VietNamNet, domestic game publishers were more than a little annoyed that Steam was releasing thousands of titles into their market “without having to ask for permission”, which they saw as “an injustice”.

Less controversially, domestic game developers pointed out that they are required to censor violence and sexual content from their games, while Steam freely offers such uncensored games to Vietnamese gamers.

VietNamNet then suggested that a possible middle path might be to use a restricted version of Steam in Vietnam, like what was done for China, serving a much more limited number of game titles that the authorities have approved.

Steam is Valve’s video game distribution service launched in 2003 as a software application that automatically delivers games and was expanded to distribute third-party titles.

Talking to VietNamNet, many domestic game publishers believe that they are being treated unfairly, when the Steam platform is releasing more than 100,000 unlicensed computer (PC) games into the Vietnamese market.

“Steam is releasing games freely into Vietnam, including violent games, adult games… but without having to ask for permission. This is an injustice to domestic publishers. This requires authorities to take management measures with this cross-border platform, otherwise, domestic game publishers will die,” a representative of a game publisher shared.

According to PV VietNamNet’s research, Steam is Valve’s video game distribution service launched in 2003 as a software application, automatically providing games and expanded to distribute game titles. of third parties from the end of 2005 until now.

Steam offers a variety of features such as technical management, game server connection with Valve Anti-Cheat anti-hacking solution, social networks and Streaming services. Besides, Steam’s application also automatically updates games, cloud storage, community features…

Steam is released globally, currently has more than 100,000 games distributed (including paid and free) as well as updating 6,000 – 8,000 games each year. This platform supports up to 24 languages and is easily accessible to all users around the world. According to VGC, 2023 has seen a series of games distributed on Steam, with 14,535 new games, an increase of nearly 2,000 games compared to 2022.

To use Steam is very simple, just download and install Steam Client, then register an account and proceed to buy games or play free games provided by this platform. Steam also has a built-in electronic wallet (Steam wallet) so players can recharge and pay for user needs (buying games, buying items on the market, items…).

As noted, a lot of shooting games are being released on Steam, which still leave the effect (red blood) intact. In Vietnam, games of the same genre must be edited, due to the violent nature that affects the players’ psychology. In addition, on this platform there are also games with adult content with revealing images, sexual intercourse…

In 2021, the Steam Global version was officially “banned” in the Chinese market, replaced by a Steam version specifically for China. The game library on this platform has just over 100 titles, most of which are Chinese games. Players can find some famous games like Dota 2, CSGO… but will not be able to find games rated 18+ which are very popular on Steam.

In addition to lacking about 110,000 games compared to the Global version, Steam China is also not allowed to have any community features, does not support Workshops, does not allow selling trading cards on the Community Market, and does not have a forum to Gamers communicate and support each other,…

New games that want to appear on this platform will also be strictly censored. Specifically, games released in China are not allowed to have blood, are not allowed to show images of “skulls and bones” and images that can hurt gamers.

According to sources from the management agency, with no contact point in Vietnam, the authorities are looking for solutions to prevent illegal cross-border services from being provided to Vietnamese users in the future next time.

Show less

It is unknown when Steam will be reinstated in Vietnam, if ever. What seems certain though is that Vietnamese gamers will not be able to access Steam in its current form…

A Possible Workaround To Access Steam In Vietnam!

Enterprising gamers have been able to bypass the ban on Steam in Vietnam by simply changing the DNS servers used by their computers, or routers.

Here is a quick guide on how to do that in Windows:

Go to Network & Internet settings.
Select the Change adapter options.
Right click on your Wi-Fi or LAN connection, and select Properties.
Double-click on Internet Protocol Version 4, or select it and click Properties.
Switch to Use the following DNS server addresses.
Then key in the alternate DNS addresses from one of these excellent options:

Cloudflare
Primary DNS : 1.1.1.1
Secondary DNS : 1.0.0.1

Google
Primary DNS : 8.8.8.8
Secondary DNS : 8.8.4.4

NordVPN
Primary DNS : 1.1.1.1
Secondary DNS : 1.0.0.1

After keying in the DNS servers you want, click OK, and you are done!

Please Support My Work!

Support my work through a bank transfer / PayPal / credit card!

Name : Adrian Wong
Bank Transfer : CIMB 7064555917 (Swift Code : CIBBMYKL)
Credit Card / Paypal : https://paypal.me/techarp

Dr. Adrian Wong has been writing about tech and science since 1997, even publishing a book with Prentice Hall called Breaking Through The BIOS Barrier (ISBN 978-0131455368) while in medical school.

He continues to devote countless hours every day writing about tech, medicine and science, in his pursuit of facts in a post-truth world.

Recommended Reading

Go Back To > Gaming | Software | Tech ARP

Support Tech ARP!

Please support us by visiting our sponsors, participating in the Tech ARP Forums, or donating to our fund. Thank you!

Cloudflare Services Down For Almost 2 Days In Major Outage!

Leave a reply

Cloudflare services have gone down in a major outage, due to a loss of power affecting its data centres! Find out what’s going on!

Updated @ 2023-11-04 : Added new information and updates
Updated @ 2023-11-03 : Added new information and updates
Originally posted @ 2023-11-02

Cloudflare Dashboard + Services Down In Major Outage!

At approximately 11:54 AM UTC on 2 November 2023, Cloudflare customers started experiencing problems with its APIs and Dashboard services. Any user logging into their accounts would only see this announcement:

The Cloudflare Dashboard is temporarily unavailable.

Please reload this page to try again. If the issue persists, please visit the Cloudflare Status page for up-to-date information regarding any ongoing issues.

Cloudflare Edge security features, and Cloudflare CDN caching services were not affected. But all other services relying on the Cloudflare API infrastructure went down – Alerts, Dashboard functionality, Zero Trust, WARP, Cloudflared, Waiting Room, Gateway, Stream, Magic WAN, API Shield, Pages, and Workers.

Cloudflare Services Down From Datacenter Power Loss!

Just over an hour later, Cloudflare reported that the outage was far worse than it initially reported, and they were still “actively investigating the problem”. They also provided a more detailed list of its services that have gone down:

The following products are currently impacted at the data plane / edge level, meaning that the full product functionality is either partially or fully affected: Logpush, WARP / Zero Trust device posture, Cloudflare dashboard, Cloudflare API, Stream API, Workers API, Alert Notification System.

These products are impacted at the control plane / core level, meaning that only the changes to the existing configuration are affected, but the product is functioning at the edge: Magic Transit, Argo Smart Routing, Workers KV, WAF, Rate Limiting, Rules, WARP / Zero Trust Registration, Waiting Room, Load Balancing and Healthchecks, Cloudflare Pages, Zero Trust Gateway, DNS Authoritative and Secondary, Cloudflare Tunnel, Workers KV namespace operations, Magic WAN.

At approximately 1:58 PM UTC, Cloudflare finally revealed that the outage was due to a loss of power at its datacenter, which caused services to fail simultaneously.

Cloudflare is assessing a loss of power impacting data centres while simultaneously failing over services.

We are continuing to work on a fix for this issue.

Several hours later – at 5:08 PM UTC, power was partially restored to its core North American data centre, and some core services were transferred to a backup data centre. In three subsequent updates, Cloudflare reports that it was seeing “gradual improvement” to its affected services.

However, at almost 11 PM that day – almost half a day after the outage started, Cloudflare was still working to restore full functionality.

At 2:40 AM UTC on 3 November, Cloudflare released an updated list of what is now functioning, and what still remains down:

Data plane impact (i.e. full product functionality is either fully impacted or partially impacted):
* Cloudflare Dash: degraded availability but accessible
* Stream API: not available
* Healthchecks: not available
* Radar: Degraded

Control plane impact (i.e. product functionality is online however changes to configuration is impacted):
* Cloudflare Pages: restored
* Magic Transit: not available to make changes
* Magic WAN: not available to make changes
* Argo Smart Routing: configuration now working, smart updates and analytics are offline
* Waiting Room: configuration updates are propagating with delay
* ZT Gateway: degraded but online
* DNS Secondary: not available to make changes
* Billing API: Degraded
* Cloudflare API: degraded but accessible

Some six hours later, they appear to be making some progress, although it seem obvious that they are a long way from a full restoration of services.

Data plane impact (i.e. full product functionality is either fully impacted or partially impacted):
* Logpush: more jobs are being restored
* Cloudflare dashboard: degraded availability but accessible
* Stream API: not available
* Healthchecks: not available
* Radar: Degraded

Control plane impact (i.e. product functionality is online however changes to configuration is impacted):
* CDN API: potentially impacted
* Access API: Degraded
* Magic Transit: configuration plane is working internally (contact your account team to make any urgent changes)
* Magic WAN: configuration plane is working internally (contact your account team to make any urgent changes)
* SOC Proactive Alerts: not available
* Argo Smart Routing: configuration now working, smart updates and analytics are offline
* Billing API: Degraded
* Email Noticatifications: Emails updates from Cloudflare are being delayed or not sent
* Lists: List configurations are working, but there may be delays in list updates propagating throughout the cloudflare network

Almost a day later – at 10:59 AM UTC, Cloudflare said that it was still continuing to work on restoring service for the impacted applications. It is unknown if power has been restored to their North American datacenter in full… or if they are facing other problems…

The good news is that – the Cloudflare Dashboard is now working, although certain services or features like Analytics are “temporarily unavailable”, and you may get error messages about certain API requests failing.

It was only at 6:42 AM UTC on November 4, 2023 – almost 2 days later, that Cloudflare finally reported that it restored all services to “operational status”.

Now, we wait to see whether it would reveal what actually happened in this exceptional downtime, for a company that prides itself on helping to keep its clients online and available…

Please Support My Work!

Support my work through a bank transfer / PayPal / credit card!

Name : Adrian Wong
Bank Transfer : CIMB 7064555917 (Swift Code : CIBBMYKL)
Credit Card / Paypal : https://paypal.me/techarp

Dr. Adrian Wong has been writing about tech and science since 1997, even publishing a book with Prentice Hall called Breaking Through The BIOS Barrier (ISBN 978-0131455368) while in medical school.

He continues to devote countless hours every day writing about tech, medicine and science, in his pursuit of facts in a post-truth world.

Recommended Reading

Go Back To > Business | Internet | Tech ARP

Support Tech ARP!

Please support us by visiting our sponsors, participating in the Tech ARP Forums, or donating to our fund. Thank you!

Cloudflare Went Down, Knocking Many Websites Offline!

Leave a reply

Many websites were knocked offline today, after Cloudflare suffered a major outage and went down!

Here is what you need to know…

Cloudflare Went Down, Knocking Many Websites Offline!

At around 6:30 AM UTC, Cloudflare – the popular content delivery network (CDN) went down, knocking large parts of the Internet offline.

This is ironic, because many websites rely on Cloudflare to not only speed up page loading for their users, but also to mitigate or prevent downtimes from DDoS attacks and server failures.

Cloudflare declared it a critical P0 incident at around 6:34 AM, noting that “connectivity has been disrupted in broad regions“. Instead of loading, all affected websites would show a 500 Internal Server Error message.

Tech ARP was affected, and so were many popular websites and online services like Discord, Omegle, Medium, Feedly, Epic Games, etc.

Cloudflare Now Up, After Down For An Hour!

They finally identified the issue about 23 minutes later. The fix took another 23 minutes, but by 7 AM UTC, they restored connectivity.

After monitoring for about an hour, the Cloudflare team declared the P0 critical incident “resolved”0 at 8:06 AM UTC.

Cloudflare has not revealed what caused the outage, but right now, most people are just glad that they resolved in within an hour!

Please Support My Work!

Support my work through a bank transfer / PayPal / credit card!

Name : Adrian Wong
Bank Transfer : CIMB 7064555917 (Swift Code : CIBBMYKL)
Credit Card / Paypal : https://paypal.me/techarp

Dr. Adrian Wong has been writing about tech and science since 1997, even publishing a book with Prentice Hall called Breaking Through The BIOS Barrier (ISBN 978-0131455368) while in medical school.

He continues to devote countless hours every day writing about tech, medicine and science, in his pursuit of facts in a post-truth world.

Recommended Reading

Go Back To > Internet | Business | Tech ARP

Support Tech ARP!

Please support us by visiting our sponsors, participating in the Tech ARP Forums, or donating to our fund. Thank you!

This iThemes Security Setting Can Block Search Engine Bots!

Leave a reply

We were recently beset by a sudden, MASSIVE drop in the number of search referrals from Google. We discovered that something was blocking search engine robots from accessing our sitemaps or crawling our website!

At first, we though it was due to a CloudFlare SSL configuration. But now, we have confirmed that it is due to an iThemes Security feature!

It is critical that you check and make sure that this iThemes Security setting is not preventing search engine robots from reading your sitemaps, and crawling your website.

iThemes Security

Formerly known as Better WP Security, iThemes Security is a very popular WordPress security plugin. While the paid Pro option offers a ton of cool security features, even the free plugin gives you the ability to lock down WordPress, fix common holes, stop automated attacks and implement a blacklist (this is a pivotal feature in this article).

What Happens If You Block Search Engine Bots?

if you block search engine bots accidentally or otherwise, you prevent them from indexing your website. This essentially makes your website “invisible” to search engines. Your website pages will no longer appear when people are searching for a relevant topic.

How does that happen?

Your website pages no longer appear in Google searches. If Google can’t see them, Google cannot display your pages in search results!
Even if your website pages do appear in Google searches, the links may be corrupt or nonsensical. Look at this example of this search result which leads to a bad link.

The description of your website page may also be nonsensical, as the example above also demonstrates.

This iThemes Security Setting Can Block Search Engine Bots!

A key feature of iThemes Security is the ability to set up a blacklist. iThemes Security will automatically populate the blacklist with the IP addresses after a number of failed attempts to login. This prevents a malicious attacker from trying to brute force its way into your system.

Unfortunately, it can falsely detect search engine bots as malicious hackers, and add them to the blacklist. That was precisely what happened to us.

Google reports that all of our sitemaps are inaccessible. Here is a view showing all five of our sitemaps were inaccessible.

If you click on the reported errors, they will all show HTTP 403 error (Forbidden).

You can verify if any search engine bot is being blocked by keying in the sitemap (or robot.txt or your website) at Redirect Checker. You can also try loading the sitemap or robot.txt in your own web browser.

If Redirect Checker or you have no problem accessing your sitemaps or robot.txt file, then something is blocking the Googlebot (or other search engine robots) from accessing your sitemaps, or crawling your website. That “something” is most likely iThemes Security’s blacklist.

The Solution

The solution is simple.

Log into your website’s WordPress admin panel.
Go to Security -> Settings.
Look for the Banned Users section, and click on Configure Settings.

In the Banned Users page, you will see a list of banned IP addresses.
Delete the whole list of banned IP addresses.

Uncheck the Ban Lists option.

Click Save Settings, and that’s it! The search engine robots will now be able to read your sitemaps!

Pro Tips

[adrotate group=”2″]

For those who still want to use the blacklist to block malicious attackers, here are the IP addresses used by various search engine robots.

Google will eventually read your sitemaps and reindex your website. But you can speed things along by :

Both guides were written with help from Kok Kee from Nasi Lemak Tech!

Support Tech ARP!

If you like our work, you can help support our work by visiting our sponsors, participating in the Tech ARP Forums, or even donating to our fund. Any help you can render is greatly appreciated!

How To Detect + Fix Sitemap Problems In Google Search Console

Leave a reply

If your website pages are not showing up in Google, you may have a problem with your sitemaps. This is a serious problem because it prevents Google from directing people to your website. Let us show you how to fix sitemap problems, and get Google to reindex your website!

Credit : This guide was written with help from Kok Kee from Nasi Lemak Tech!

How Serious Are Sitemap Problems?

Sitemap problems are HUGE, because they greatly sap the number of people who visit your website from a Google search. When we started having sitemap problems, the number of unique visitors to Tech ARP was instantly halved.

How does that happen?

Your website pages no longer appear in Google searches. If Google can’t see them, Google cannot display your pages in search results!
Even if your website pages do appear in Google searches, the links may be corrupt or nonsensical. Look at this example of this search result which leads to a bad link.

The description of your website page may also be nonsensical, as the example above also demonstrates.

How To Detect + Fix Sitemap Problems

Log into Google Search Console.
Go to the Sitemaps section (Crawl > Sitemaps)
It will show you the sitemaps for your website, as well as any sitemap errors.

Click on the error to find out what’s wrong.

You will need to use the information listed to figure out what’s wrong, and fix the problem.
a) if the sitemap was deleted, or is corrupt, you will need to generate a new sitemap
b) if the sitemap is inaccessible due to a permission setting, you need to change its permission setting.
c) if you are using CloudFlare, try disabling Always use HTTPS.

[adrotate group=”1″]

After you fix the sitemap problem, you will need to resubmit the sitemap to Google. Go into each sitemap information page, and click on the red Resubmit button at the upper right corner.

After submitting the new sitemap, you can refresh the page to check its status.

Repeat the steps for all of your sitemaps,
Google Search Console should no longer list any sitemap error.

Tech ARP

For the best in science + technology!

Tag Archives: CloudFlare

Did Malaysia just block use of public DNS servers?!

MCMC blocks public DNS servers only for business and government?

Public DNS servers appear to be blocked in Malaysia!

Please Support My Work!

Malaysia orders DNS redirection for business and government!

Malaysia orders DNS redirection for business and government!

Please Support My Work!

Steam Is Suddenly Banned Entirely In Vietnam!

Steam Is Suddenly Banned Entirely In Vietnam!

Possible Reasons Why Steam Was Banned In Vietnam!

A Possible Workaround To Access Steam In Vietnam!

Please Support My Work!

Cloudflare Services Down For Almost 2 Days In Major Outage!

Cloudflare Dashboard + Services Down In Major Outage!

Cloudflare Services Down From Datacenter Power Loss!

Please Support My Work!

This iThemes Security Setting Can Block Search Engine Bots!

iThemes Security

What Happens If You Block Search Engine Bots?

This iThemes Security Setting Can Block Search Engine Bots!

The Solution

Pro Tips

Suggested Reading

Go Back To > Articles | Home

Support Tech ARP!

How To Detect + Fix Sitemap Problems In Google Search Console

How Serious Are Sitemap Problems?

How To Detect + Fix Sitemap Problems

Suggested Reading

Go Back To > Guides | Home

Support Tech ARP!