Feeds and discovery

Overview

Feeds & discovery (/content/feeds) lists the machine-readable endpoints Obsurfable serves on each connected subdomain. These help search engines, RSS readers, and AI crawlers discover and understand your published blog:

Shortcut links from /content jump directly to each section (#robots, #sitemap, #rss, #humans-txt, #llms-txt).

Who it's for

• SEO teams submitting sitemaps and validating crawler access
• Technical marketers adopting llms.txt on the blog subdomain
• Anyone publishing on Obsurfable's hosted blog who wants standard discovery files without manual maintenance

Prerequisites

1. Pro plan.
2. At least one verified connected subdomain — feeds are generated per subdomain hostname.
3. Published posts for meaningful sitemap, RSS, and llms.txt content (empty blogs still serve valid endpoints).

Step-by-step

Open Feeds & discovery

1. From /content, use the feed shortcut buttons or go to /content/feeds.
2. If no subdomain is connected, click through to Connected subdomains first.

Review robots.txt

1. Open the robots.txt section (/content/feeds#robots).
2. Click the live URL for each subdomain — e.g. https://blog.example.com/robots.txt.
3. Confirm crawlers are allowed to access posts and sitemap paths.

Use the sitemap

1. Open the Sitemap section (/content/feeds#sitemap).
2. Visit https://<subdomain>/sitemap.xml to see all published post URLs.
3. Submit this URL in Google Search Console — also available from Index status.

Subscribe via RSS

1. Open the RSS feed section (/content/feeds#rss).
2. Use https://<subdomain>/feed.xml in feed readers or automation tools.
3. New published posts appear automatically — no manual feed updates.

Check humans.txt

1. Open the humans.txt section (/content/feeds#humans-txt).
2. Review credits and team information served at /humans.txt.

Review llms.txt on the blog

1. Open the llms.txt section (/content/feeds#llms-txt).
2. Visit https://<subdomain>/llms.txt — Obsurfable generates this from your blog and company context.
3. Compare with your main-site llms.txt in Setup; the blog subdomain has its own file focused on published posts.

Tips

• Submit sitemap.xml in Search Console after your first batch of published posts — pair with Index status.
• RSS is useful for internal Slack/email digests when your team publishes frequently.
• llms.txt on the subdomain complements your root-site llms.txt — it highlights blog content AI systems should retrieve for category questions.
• Feeds update automatically when you publish or unpublish — no redeploy or manual regeneration.
• Link to /sitemap.xml from your main site robots.txt if you want crawlers to find the blog sitemap from the apex domain.

FAQ

Are these feeds available without a connected subdomain?

No. Each feed is served on your connected subdomain hostname. Connect and verify a subdomain first.

Do drafts appear in the sitemap or RSS?

No. Only published posts are included in sitemap, RSS, and llms.txt output.

How is blog llms.txt different from Setup llms.txt?

Setup llms.txt is your main-site intent file used in site analysis. The blog subdomain serves a separate /llms.txt focused on hosted post URLs and blog metadata.

Can I customize robots.txt or llms.txt content?

These files are generated by Obsurfable based on your subdomain configuration and published posts. Custom overrides are not available in-app — export and self-host if you need full control.

Where do I submit the sitemap?

Google Search Console → Sitemaps → enter https://<subdomain>/sitemap.xml. You can also use sitemap actions on Index status.

Does RSS include full post content?

The RSS feed includes post metadata and excerpts/links suitable for syndication. Open individual post URLs for the full rendered article.