WordPress has a nifty little feature that allows you to easily password protect content so it can accessible on the web but only people with the password can access it. It’s a simple and quick way to share content with a trusted audience.
We recently discovered that some of our password protected content had been indexed by search engines. This was alarming to say the least, but in our case, no sensitive information was exposed. We breathed a sigh of relief, but then thought, how could this have happened?
Finding The Cause
Our initial suspicion was the theme we’re using to power our website – the extremely popular Divi by Elegant Themes. It seemed highly unlikely to be caused by a plugin because the content being indexed wasn’t the password protected posts themselves, but rather the category and author pages listing them. Those pages are typically controlled by the theme.
To test our theory, we setup a clean test site and installed Divi alongside the default WordPress 2017 theme. We created a few password protected posts and categorized them. Then we checked the category and author pages with 2017 active and then again with Divi enabled.
As suspected, the category and author pages Divi generates include excerpts from the password protected posts, rather than presenting the password field as seen in the 2017 theme.
2017 Author Page
2017 Category Page
Divi Author Page
Divi Category Page
We also discovered that the Divi Blog Module outputs password protected content in excerpts and in search results, so you’ll want to double check that as well and be sure you don’t have any sensitive content being displayed through that module or through static search results listings. This could lead to more than just having private content exposed on category and author pages, since there are many ways you can output data using these methods.
We notified Elegant Themes of the issue on January 6th and they have acknowledged it and testing an update internally that should resolve the issue. They still don’t have an ETA but we’ll update this post when the release has been made public. (Update: Fix issued! See the bottom of the post for more.)
The reason we’re going public before Divi is patched is because the content being exposed is partially outside of the websites being affected; their content is being indexed by search engines and that data will remain public even after people update their sites with a patched version of the theme. We felt that if we weren’t aware of an issue if this nature, we’d want to know as soon as possible so we could start cleaning up the data that has been made public.
Interestingly, Elegant Themes actually has a blog post on how to password protect your content in which they outline how to install a plugin to protect category pages. This was published back in April 2017. Makes you wonder; how long has this bug been present?
Ramifications
By default, most WordPress sites list all category and author pages in sitemaps or menus which are then indexed by Google, Bing, Yahoo, etc. This means that password protected content exposed by Divi is not only visible to people searching within your site, but it’s also likely to have been indexed by search engines.
As the blog module and internal site search results can also include password protected content, there are many ways that your private data could be exposed, and probably other scenarios where Divi is exposing password protected content that we haven’t found yet.
In most cases, the content that has been indexed was probably just excerpts, but there are a lot of variables that could alter that. You may have a customized version of Divi that prints a longer excerpt or the entire post instead of an except to category and author pages, or a plugin could do the same thing. You will need to think through how your site is constructed and test thoroughly.
Remediation
So, what’s next? We’ll update this post if we hear anything from Elegant Themes and let you know when a fix is live, but for now here are the steps to take to get your private data removed from search engines.
- Review Your Site – Look around for posts that should be private but aren’t. Only posts can utilize the password protection feature, so if you’re unsure if you have any password protected posts just go to WP Admin->Posts and look for “– Password protected” beside the post title. Review category and author pages (www.domain.com/category/category-name and www.domain.com/author/author-name) to see if you have any password protected content listed. Also, check to see if you’ve used any Divi Blog Modules or linked to search results that may include private data.
- Check The Search Engines – Do some specific searches to see what’s been indexed. Search for exact matches. For example, if you have the words “Private Client Demo” in the content of your password protected post, search Google for “private client demo” and your domain name. Let’s say the category you’ve setup for your private data is “private-data”. Try searching Google for “site:www.domain.com/category/private-data”. Make a list of what you need to remove.
- Remove Private Content From Sitemaps – Yoast SEO allows you to easily remove author and category pages from your sitemap. Just install Yoast and go to the category or user you want to remove and select “Never Include”. You need to make sure you don’t have private content included in your sitemap or search engines will re-index it even if you manually remove it.
- Manually Remove URLs from Search Engines – You can request any URL be removed from a search engine index. Here’s how to do it on Google and Bing.
- Consider Redirection – Even after you have identified private content that has been exposed, removed it from your sitemaps, and requested URL removal from search engines, it’s still technically available on your site. You may want to install a redirection plugin and point URLs with private data to a safe location, at least until a fix is available from Elegant Themes.
Update – 1/17/18 8PM EST: It turns out this issue affected all themes developed by Elegant Themes and the Divi Builder plugin. ET has just released an Email campaign notifying customers and informing them of the updates they have published. They opted to make them available to all customers, even those whose subscriptions have expired. They have also provided a security patching plugin so you can fix the problem without upgrading your theme, if necessary. The Email from ET contains links to download the updates or security patcher plugin manually, or you can apply your updates via your WordPress dashboard normally.
They don’t mention anything about cleaning up your search engine footprint, so be sure to review the steps above and check to see what’s out there that shouldn’t be!
