Issues

• Clarifying historical CC-NEWS WARC-Date and completed-response availability

  question
  commoncrawl/news-crawl#124 · Filter by author calyqo opened on Oct 2, 2026
  1

  

• Explore alternate links in news sitemaps

  commoncrawl/news-crawl#107 · Filter by author sebastian-nagel opened on Aug 13, 2026

• Malformed URL with null host may be silently dropped

  commoncrawl/news-crawl#95 · Filter by author lfoppiano opened on Jul 21, 2026

• Avoid following advertisements in news feeds and sitemaps

  commoncrawl/news-crawl#58 · Filter by author sebastian-nagel opened on Nov 14, 2023
  

• produce WET files?

  commoncrawl/news-crawl#55 · Filter by author chris-ha458 opened on Jul 19, 2023
  6

• Consider archiving of news feeds and sitemaps

  commoncrawl/news-crawl#54 · Filter by author sebastian-nagel opened on Dec 4, 2022

• Explore schema.org annotations for seed completions

  commoncrawl/news-crawl#53 · Filter by author sebastian-nagel opened on Dec 4, 2022

• Use wikidata to complete seeds

  commoncrawl/news-crawl#50 · Filter by author sebastian-nagel opened on Oct 18, 2022
  6

  

• Allow to follow news sites not providing RSS/Atom feed or news sitemap

  enhancement
  commoncrawl/news-crawl#41 · Filter by author sebastian-nagel opened on Jul 24, 2020
  2

• NewsSiteMapParserBolt: do not detect feeds as sitemaps

  commoncrawl/news-crawl#35 · Filter by author sebastian-nagel opened on Jan 9, 2020
  4

• Check cross-submits for sitemaps

  commoncrawl/news-crawl#32 · Filter by author sebastian-nagel opened on Jul 5, 2019
  4

  

• Extract publishing date

  commoncrawl/news-crawl#18 · Filter by author fhamborg opened on May 29, 2017
  2

• Improve feed parser robustness

  commoncrawl/news-crawl#13 · Filter by author sebastian-nagel opened on Nov 23, 2016
  4