add recursive search from jsonld

[lvoxpq]

Related Issues

No related issue

Proposed Changes:

The date wasn't being extracted for https://www.obsidiansecurity.com/blog/when-is-stdio-mcp-actually-a-vulnerability

The reason was that the json ld saved "datePublished" in a nested object and newspaper4k checked only for the 1st keys:

"blogPost": {
    "@type": "BlogPosting",
    "headline": "1-Click RCE in Flowise (CVE-2026-40933): When Is stdio MCP Actually a Vulnerability?",
    "description": "Security researchers at Obsidian Security discovered a one-click RCE in Flowise (CVE-2026-40933), an open-source platform for building LLM workflows and AI agents with over 52k GitHub stars. An attacker can fully compromise a server by convincing an authorized user to import a crafted chatflow. Import alone is enough to trigger arbitrary server-side code execution.",
    "author": {
      "@type": "Person",
      "name": "Fenix Qiao"
    },
    "datePublished": "2026-05-28T02:02:15.871Z",
    "dateModified": "2026-05-28T02:02:15.871Z",
    "image": "https://cdn.prod.website-files.com/67b3bf2003f9c3d795e75e7b/6a178fdf778e61a10285be94_Blog%20Image%20-%201-Click%20RCE%20in%20Flowise%20Alt%20(1).avif",
    "inLanguage": "en",
    "articleSection": "Research",
    "publisher": {
      "@type": "Organization",
      "name": "Obsidian Security",
      "logo": {
        "@type": "ImageObject",
        "url": "https://cdn.prod.website-files.com/677ed9267354cb4c9db8fde5/677efad55b45bceb25f14233_logo-dark.svg"
      }
    }
  }
}

How did you test it?

pytest -q and manually to extract the date from html.

pytest had 17 failures, but all of them are not related to the functionality (I decided not to install additional libraries)
Here is the list of failed tests:

========================================================== short test summary info ==========================================================
FAILED tests/e2e/test_source.py::TestSource::test_robots - ImportError: You must install protego before using the honor_robots_txt feature. 
FAILED tests/integration/test_article.py::TestArticle::test_follow_read_more_button - newspaper.exceptions.ArticleException: Article `download()` failed with Status code 404 for url None on URL https://finance.yahoo.com/m/...
FAILED tests/integration/test_cli.py::TestCLI::test_cli_output_format - ImportError: nltk is required for NLP features. Install it with: pip install 'newspaper4k[nlp]'
FAILED tests/integration/test_languages.py::TestLanguages::test_language_articles - ImportError: You must install pythainlp before using the Thai tokenizer. 
FAILED tests/integration/test_misc.py::test_multithread_requests - assert 503 == 200
FAILED tests/integration/test_requests.py::TestNetwork::test_detect_cloudflair - Failed: DID NOT RAISE <class 'newspaper.exceptions.ArticleException'>
FAILED tests/integration/test_source.py::TestSource::test_pickle_source - TypeError: cannot pickle '_thread.lock' object
FAILED tests/integration/test_source.py::TestSource::test_get_feeds - assert (20 - 2) <= 0
FAILED tests/unit/test_article.py::TestArticle::test_article_nlp - ImportError: nltk is required for NLP features. Install it with: pip install 'newspaper4k[nlp]'
FAILED tests/unit/test_article.py::TestArticle::test_known_websites - ImportError: nltk is required for NLP features. Install it with: pip install 'newspaper4k[nlp]'
FAILED tests/unit/test_cli.py::TestCLI::test_input_html - ImportError: nltk is required for NLP features. Install it with: pip install 'newspaper4k[nlp]'
FAILED tests/unit/test_languages.py::TestLanguages::test_stopwords_languages - ImportError: You must install indic-nlp-library before using the Bengali tokenizer. 
FAILED tests/unit/test_languages.py::TestLanguages::test_stopwords - ImportError: You must install pythainlp before using the Thai tokenizer. 
FAILED tests/unit/test_languages.py::TestLanguages::test_bengali - ImportError: You must install indic-nlp-library before using the Bengali tokenizer. 
FAILED tests/unit/test_languages.py::TestLanguages::test_nepali - ImportError: You must install indic-nlp-library before using the Nepali tokenizer. 
FAILED tests/unit/test_nlp.py::TestNLP::test_article_nlp - ImportError: nltk is required for NLP features. Install it with: pip install 'newspaper4k[nlp]'
FAILED tests/unit/test_nlp.py::TestNLP::test_summarize - ImportError: nltk is required for NLP features. Install it with: pip install 'newspaper4k[nlp]'
ERROR tests/unit/test_category_extractor.py::test_parse
ERROR tests/unit/test_google_news.py::test_google_news_default_language
ERROR tests/unit/test_google_news.py::test_google_news_language_init_param
ERROR tests/unit/test_google_news.py::test_google_news_config_language_respected_in_download
ERROR tests/unit/test_google_news.py::test_google_news_proxy_format
ERROR tests/unit/test_google_news.py::test_google_news_proxy_used_in_feedparser
ERROR tests/unit/test_google_news.py::test_google_news_no_proxy
ERROR tests/unit/test_source.py::test_source_build_offline
ERROR tests/unit/test_source.py::test_source_set_categories
ERROR tests/unit/test_source.py::test_source_set_feeds
ERROR tests/unit/test_source.py::test_source_generate_articles
ERROR tests/unit/test_source.py::test_source_download_articles
ERROR tests/unit/test_source.py::test_source_parse_articles
17 failed, 136 passed, 2 skipped, 5 warnings, 13 errors in 218.82s (0:03:38)

Notes for the reviewer

Please check if nested solution is needed. If not change to support "blogpost" key and then search for datePublished in this object.