Verbolia is designed to optimize which pages are visible to search engines, ensuring that only high-value pages are indexed. Here’s how it works:

• Parameterized Vpages (such as filtered versions or paginated pages like page 2, 3, etc.) are automatically marked with x-robots-tag: noindex, nofollow in the HTTP response header.
• The robots.txt file on the Vpage backend allows crawlers to access necessary backend resources and APIs without blocking essential functionality.

Manually excluding a page from indexing #

If you want to prevent a specific Vpage from being indexed by search engines, you can do it manually. In the keyword edition page of your specific keyword, there is a checkbox labeled “Make this page non indexable ”. When selected:

• The page will return a x-robots-tag: noindex directive to block indexing.
• The given keyword will be excluded from Googlebot visits, position tracking batches, and SEO performance charts.
• The page will not be included in your website’s sitemap, internal linking, or the Verbolia linking widget installed on your standard pages.

This gives you full control over what gets indexed .