The Marfeel Editorial Crawler builds the editorial profile of every page by extracting metadata the same way Googlebot does. It pulls editorial dimensions such as section, author, publication date, and title so editors can understand how content is distributed and consumed, then optimize their production accordingly.
Marfeel tracks data about traffic, subscriptions, advertising, and engagement, and supports reporting across all of these editorial dimensions.
When Marfeel receives the first hit on a URL, the Editorial Crawler gets the canonical of the page and extracts as much metadata as possible from the server-side rendered markup.
Where does the Marfeel Editorial Crawler get metadata from
The Editorial Crawler auto-tracks metadata through five sequential strategies applied to server-side rendering markup:
- Custom Marfeel tagging
- Structure Data
- Microdata
- Open Graph og:*
- RDFa
Read more on how Marfeel detects specific metadata fields and how you can improve your SEO by providing them correctly:
- Publication & Last Update date
- Title
- Image & Video
- Authors & Bylines
- Sections & Sub-sections
- Tags & Tag Groups
- Content Metrics
- Ensuring all metadata detection by following best practices
When does the Marfeel Editorial Crawler crawl an article?
The crawler processes a URL on its first hit and re-crawls it any time the article content changes. Read all the details.
Not updating the last update date can impact positioning on Google News and the News Carousel, among other placements. Because Marfeel creates a visual representation of how Googlebot most likely sees your site, it does not update the title on its own. Read more information.
Unlocking private article metadata
When crawlers cannot retrieve metadata, for example because content is behind a paywall, requires registration, or contains sensitive metadata, publishers can send all metadata to Marfeel using the Metadata API endpoint.
Miscellaneous
- Whitelisting Marfeel crawlers.
- Setup the Article Body crawler.
- Tools to understand where Marfeel gets metadata from.
What metadata sources does the Marfeel Editorial Crawler use?
The Marfeel Editorial Crawler extracts metadata through five sequential strategies on server-side rendered markup: custom Marfeel tagging, structured data, microdata, Open Graph (og:*), and RDFa.
When does the Marfeel Editorial Crawler crawl an article?
The crawler processes a URL when it receives its first hit and re-crawls the article any time its content changes. See Editorial Crawler frequency for the full details.
How can publishers send metadata for paywalled or private content?
Publishers can use the Metadata API endpoint to send all metadata directly to Marfeel when crawlers cannot access content behind paywalls, registration gates, or sensitive metadata.