alihan/news-please-content-crawler
1
0
Fork 0
mirror of https://github.com/fhamborg/news-please.git synced 2021-09-19 22:26:00 +03:00
Code Issues Projects Releases Wiki Activity

Update commoncrawl.py

Browse Source
This commit is contained in:
Felix Hamborg
2017-07-20 21:27:45 +02:00
committed by GitHub
parent e641bff064
commit 36d1dd6438
1 changed files with 2 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
newsplease/examples/commoncrawl.py
View File

@@ -243,8 +243,8 @@ class CommonCrawl:
def run(self):
"""
Main execution method, which consists of: get an up-to-date list of WARC files, and for each of them: download
and extract articles. Each articles are checked against a filter. Finally, for each valid article the method
on_valid_article_extracted will be invoked.
and extract articles. Each article is checked against a filter. Finally, for each valid article the method
on_valid_article_extracted will be invoked after the extraction of the article has completed.
:return:
"""
self.__setup__()