nutch 0.9安装与使用(install and running)

nutch-anotherbug.gif(14.8 K)

切换到幻灯片模式

1.下载安装Windows下的Linux模拟工具 Cygwin (因为nutch命令是基于linux的，如果在linux下安装使用，请跳过此步)

安装过程：http://www.cygwin.cn/site/install/

2.假设下载的nutch-0.9.tar.gz放在d:\下，将包解压：启动Cygwin

1 2	cd /cygdirve/d tar -zvxf nutch-0.9.tar.gz

3.在d:\nutch-0.9\下新建urls目录，里面建个文件，比如 nutch，内容如下：

1	http://anotherbug.blog.chinajavaworld.com/

4.修改d:\nutch-0.9\conf\crawl-urlfilter.txt文件
将

1 2	# accept hosts in MY.DOMAIN.NAME +^http://([a-z0-9]*\.)*MY.DOMAIN.NAME/

改为如下：

1 2 3

# accept hosts in MY.DOMAIN.NAME #+^http://([a-z0-9]*\.)*MY.DOMAIN.NAME/ +^http://anotherbug.blog.chinajavaworld.com/

5.修改 conf/nutch-site.xml，在configuration根节点里加入：

1 2 3 4 5

<property> <name>http.agent.name</name> <value>chinajavaworld java search engine</value> <description>chinajavaworld java search engine</description> </property>

6. 开始执行nutch命令，抓取网页

1 2	cd /cygdrive/d/nutch-0.9/  bin/nutch crawl urls -dir crawl -depth 3 -topN 50 >& crawl.log

7.以上指令执行完后，启动 nutch 自带的搜索应用(将nutch-0.9.war解压或让应用服务器自动解压)进行搜索测试：

修改 resin.conf

1 2 3 4

<host id="nutch.chinajavaworld.com" root-directory="."> <web-app id="/" document-directory="d:\resin\app\nutch"> </web-app> </host>

同时修改 nutch\WEB-INF\classes\nutch-site.xml，如下：

1 2 3 4 5 6 7 8 9 10 11 12 13

<?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="nutch-conf.xsl"?>   <!-- Put site-specific property overrides in this file. -->   <nutch-conf> <property> <name>searcher.dir</name> <value>d:\nutch-0.9\crawl</value> <description>path to nutch's searcher dir.</description> </property> </nutch-conf>

启动 Resin，同时将hosts中加入 127.0.0.1 nutch.chinajavaworld.com

访问http://nutch.chinajavaworld.com/，即可看到搜索测试页面，如附件。


附：crawl.log

crawl started in: crawl
rootUrlDir = urls
threads = 10
depth = 3
topN = 50
Injector: starting
Injector: crawlDb: crawl/crawldb
Injector: urlDir: urls
Injector: Converting injected urls to crawl db entries.
Injector: Merging injected urls into crawl db.
Injector: done
Generator: Selecting best-scoring urls due for fetch.
Generator: starting
Generator: segment: crawl/segments/20071227201306
Generator: filtering: false
Generator: topN: 50
Generator: jobtracker is 'local', generating exactly one partition.
Generator: Partitioning selected urls by host, for politeness.
Generator: done.
Fetcher: starting
Fetcher: segment: crawl/segments/20071227201306
Fetcher: threads: 10
fetching http://anotherbug.blog.chinajavaworld.com/
Fetcher: done
CrawlDb update: starting
CrawlDb update: db: crawl/crawldb
CrawlDb update: segments: [crawl/segments/20071227201306]
CrawlDb update: additions allowed: true
CrawlDb update: URL normalizing: true
CrawlDb update: URL filtering: true
CrawlDb update: Merging segment data into db.
CrawlDb update: done
Generator: Selecting best-scoring urls due for fetch.
Generator: starting
Generator: segment: crawl/segments/20071227201318
Generator: filtering: false
Generator: topN: 50
Generator: jobtracker is 'local', generating exactly one partition.
Generator: Partitioning selected urls by host, for politeness.
Generator: done.
Fetcher: starting
Fetcher: segment: crawl/segments/20071227201318
Fetcher: threads: 10
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/tag/442/
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/tag/1079/
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/30_0_0_-1_0/
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/tag/692/
fetching http://anotherbug.blog.chinajavaworld.com/feed.asp
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/45_0_0_-1_0/
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/0_0_0_-1_421
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/tag/46/
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/tag/23/
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/tag/543/
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/0_0_0_-1_1
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/tag/544/
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/2007/11/
fetching http://anotherbug.blog.chinajavaworld.com/entry/3943/0/
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/2008/1/
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/15_0_0_-1_0/
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/tag/413/
fetching http://anotherbug.blog.chinajavaworld.com/entry/3348/0/
fetching http://anotherbug.blog.chinajavaworld.com/entry/2769/0/
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/tag/202/
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/0_0_0_-1_1155
fetching http://anotherbug.blog.chinajavaworld.com/entry/3949/0/
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/60_0_0_-1_0/
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/tag/1568/
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/0_0_0_-1_1167
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/tag/2030/
fetching http://anotherbug.blog.chinajavaworld.com/atom.asp
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/tag/145/
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/tag/2041/
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/tag/2034/
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/tag/
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/tag/2035/
fetching http://anotherbug.blog.chinajavaworld.com/entry/3950/0/
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/0_0_0_-1_23
fetching http://anotherbug.blog.chinajavaworld.com/entry/3938/0/
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/tag/690/
Fetcher: done
CrawlDb update: starting
CrawlDb update: db: crawl/crawldb
CrawlDb update: segments: [crawl/segments/20071227201318]
CrawlDb update: additions allowed: true
CrawlDb update: URL normalizing: true
CrawlDb update: URL filtering: true
CrawlDb update: Merging segment data into db.
CrawlDb update: done
Generator: Selecting best-scoring urls due for fetch.
Generator: starting
Generator: segment: crawl/segments/20071227201638
Generator: filtering: false
Generator: topN: 50
Generator: jobtracker is 'local', generating exactly one partition.
Generator: Partitioning selected urls by host, for politeness.
Generator: done.
Fetcher: starting
Fetcher: segment: crawl/segments/20071227201638
Fetcher: threads: 10
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/0_0_0_-1_4
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/0_0_0_-1_298
fetching http://anotherbug.blog.chinajavaworld.com/entry/3943/0/rate.avg_user_rating.label
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/2007/12/20/
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/2007/12/13/
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/0_0_0_-1_405
fetching http://anotherbug.blog.chinajavaworld.com/entry/43/0/
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/0_0_0_-1_63
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/2007/12/15/
fetching http://anotherbug.blog.chinajavaworld.com/entry/3348/0/rate.avg_user_rating.label
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/0_0_0_-1_137
Error parsing: http://anotherbug.blog.chinajavaworld.com/entry/3348/0/rate.avg_user_rating.label: failed(2,200): java.lang.NullPointerException:
fetch of http://anotherbug.blog.chinajavaworld.com/entry/3348/0/rate.avg_user_rating.label failed with: java.lang.NullPointerException:
fetching http://anotherbug.blog.chinajavaworld.com/entry/3625/0/
fetching http://anotherbug.blog.chinajavaworld.com/entry/2769/0/rate.avg_user_rating.label
fetching http://anotherbug.blog.chinajavaworld.com/entry/3943/0/正在保存...
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/2007/12/9/
fetching http://anotherbug.blog.chinajavaworld.com/entry/3949/0/rate.avg_user_rating.label
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/0_0_0_-1_228
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/0_0_0_-1_3
fetching http://anotherbug.blog.chinajavaworld.com/entry/1426/0/
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/0_0_0_-1_1086
fetching http://anotherbug.blog.chinajavaworld.com/dwr/util.js
fetching http://anotherbug.blog.chinajavaworld.com/entry/3348/0/正在保存...
fetching http://anotherbug.blog.chinajavaworld.com/dwr/engine.js
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/
fetch of http://anotherbug.blog.chinajavaworld.com/u/123297/ failed with: java.net.SocketTimeoutException: Read timed out
fetching http://anotherbug.blog.chinajavaworld.com/entry/2769/0/正在保存...
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/2007/12/12/
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/2007/12/1/
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/2007/12/19/
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/0_0_0_-1_54
fetching http://anotherbug.blog.chinajavaworld.com/entry/3949/0/正在保存...
fetching http://anotherbug.blog.chinajavaworld.com/entry/3950/0/rate.avg_user_rating.label
Error parsing: http://anotherbug.blog.chinajavaworld.com/entry/3950/0/rate.avg_user_rating.label: failed(2,200): java.lang.NullPointerException:
fetch of http://anotherbug.blog.chinajavaworld.com/entry/3950/0/rate.avg_user_rating.label failed with: java.lang.NullPointerException:
fetching http://anotherbug.blog.chinajavaworld.com/common/UBBCode_help.js
fetching http://anotherbug.blog.chinajavaworld.com/js/scriptaculous/scriptaculous.js
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/0_0_0_-1_414
fetching http://anotherbug.blog.chinajavaworld.com/entry/3938/0/rate.avg_user_rating.label
Error parsing: http://anotherbug.blog.chinajavaworld.com/entry/3938/0/rate.avg_user_rating.label: failed(2,200): java.lang.NullPointerException:
fetch of http://anotherbug.blog.chinajavaworld.com/entry/3938/0/rate.avg_user_rating.label failed with: java.lang.NullPointerException:
fetching http://anotherbug.blog.chinajavaworld.com/entry/3348/1/
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/2007/12/14/
fetching http://anotherbug.blog.chinajavaworld.com/js/events.js
fetching http://anotherbug.blog.chinajavaworld.com/u/123297
fetching http://anotherbug.blog.chinajavaworld.com/entry/3795/0/
fetching http://anotherbug.blog.chinajavaworld.com/entry/3950/0/正在保存...
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/2007/12/23/
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/0_0_0_-1_2
fetching http://anotherbug.blog.chinajavaworld.com/u/123297/2007/12/16/
fetching http://anotherbug.blog.chinajavaworld.com/js/prototype/prototype.js
fetching http://anotherbug.blog.chinajavaworld.com/entry/3938/0/正在保存...
fetching http://anotherbug.blog.chinajavaworld.com/entry/2959/0/
fetching http://anotherbug.blog.chinajavaworld.com/common/UBBCode.js
fetching http://anotherbug.blog.chinajavaworld.com/entry/3804/0/
fetching http://anotherbug.blog.chinajavaworld.com/dwr/interface/Rate.js
Error parsing: http://anotherbug.blog.chinajavaworld.com/entry/2769/0/rate.avg_user_rating.label: failed(2,200): java.lang.NullPointerException:
fetch of http://anotherbug.blog.chinajavaworld.com/entry/2769/0/rate.avg_user_rating.label failed with: java.lang.NullPointerException:
Error parsing: http://anotherbug.blog.chinajavaworld.com/entry/3943/0/rate.avg_user_rating.label: failed(2,200): java.lang.NullPointerException:
fetch of http://anotherbug.blog.chinajavaworld.com/entry/3943/0/rate.avg_user_rating.label failed with: java.lang.NullPointerException:
Error parsing: http://anotherbug.blog.chinajavaworld.com/entry/3949/0/rate.avg_user_rating.label: failed(2,200): java.lang.NullPointerException:
fetch of http://anotherbug.blog.chinajavaworld.com/entry/3949/0/rate.avg_user_rating.label failed with: java.lang.NullPointerException:
fetch of http://anotherbug.blog.chinajavaworld.com/u/123297/0_0_0_-1_137 failed with: java.net.SocketTimeoutException: Read timed out
Fetcher: done
CrawlDb update: starting
CrawlDb update: db: crawl/crawldb
CrawlDb update: segments: [crawl/segments/20071227201638]
CrawlDb update: additions allowed: true
CrawlDb update: URL normalizing: true
CrawlDb update: URL filtering: true
CrawlDb update: Merging segment data into db.
CrawlDb update: done
LinkDb: starting
LinkDb: linkdb: crawl/linkdb
LinkDb: URL normalize: true
LinkDb: URL filter: true
LinkDb: adding segment: crawl/segments/20071227201306
LinkDb: adding segment: crawl/segments/20071227201318
LinkDb: adding segment: crawl/segments/20071227201638
LinkDb: done
Indexer: starting
Indexer: linkdb: crawl/linkdb
Indexer: adding segment: crawl/segments/20071227201306
Indexer: adding segment: crawl/segments/20071227201318
Indexer: adding segment: crawl/segments/20071227201638
Indexing [http://anotherbug.blog.chinajavaworld.com/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/common/UBBCode.js] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/common/UBBCode_help.js] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/dwr/engine.js] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/dwr/interface/Rate.js] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/dwr/util.js] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/entry/1426/0/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/entry/2769/0/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/entry/2959/0/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/entry/3348/0/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/entry/3348/1/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/entry/3625/0/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/entry/3795/0/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/entry/3804/0/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/entry/3938/0/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/entry/3943/0/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/entry/3949/0/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/entry/3950/0/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/entry/43/0/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/js/events.js] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/js/prototype/prototype.js] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/js/scriptaculous/scriptaculous.js] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/0_0_0_-1_1] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/0_0_0_-1_1086] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/0_0_0_-1_1155] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/0_0_0_-1_1167] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/0_0_0_-1_2] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/0_0_0_-1_228] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/0_0_0_-1_23] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/0_0_0_-1_298] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/0_0_0_-1_3] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/0_0_0_-1_4] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/0_0_0_-1_405] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/0_0_0_-1_414] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/0_0_0_-1_421] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/0_0_0_-1_54] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/0_0_0_-1_63] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/15_0_0_-1_0/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/2007/11/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/2007/12/1/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/2007/12/12/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/2007/12/13/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/2007/12/14/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/2007/12/15/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/2007/12/16/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/2007/12/19/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/2007/12/20/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/2007/12/23/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/2007/12/9/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
merging segments _ram_0 (1 docs) _ram_1 (1 docs) _ram_2 (1 docs) _ram_3 (1 docs) _ram_4 (1 docs) _ram_5 (1 docs) _ram_6 (1 docs) _ram_7 (1 docs) _ram_8 (1 docs) _ram_9 (1 docs) _ram_a (1 docs) _ram_b (1 docs) _ram_c (1 docs) _ram_d (1 docs) _ram_e (1 docs) _ram_f (1 docs) _ram_g (1 docs) _ram_h (1 docs) _ram_i (1 docs) _ram_j (1 docs) _ram_k (1 docs) _ram_l (1 docs) _ram_m (1 docs) _ram_n (1 docs) _ram_o (1 docs) _ram_p (1 docs) _ram_q (1 docs) _ram_r (1 docs) _ram_s (1 docs) _ram_t (1 docs) _ram_u (1 docs) _ram_v (1 docs) _ram_w (1 docs) _ram_x (1 docs) _ram_y (1 docs) _ram_z (1 docs) _ram_10 (1 docs) _ram_11 (1 docs) _ram_12 (1 docs) _ram_13 (1 docs) _ram_14 (1 docs) _ram_15 (1 docs) _ram_16 (1 docs) _ram_17 (1 docs) _ram_18 (1 docs) _ram_19 (1 docs) _ram_1a (1 docs) _ram_1b (1 docs) _ram_1c (1 docs) _ram_1d (1 docs) into _0 (50 docs)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/2008/1/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/30_0_0_-1_0/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/45_0_0_-1_0/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/60_0_0_-1_0/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/tag/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/tag/1079/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/tag/145/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/tag/1568/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/tag/202/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/tag/2030/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/tag/2034/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/tag/2035/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/tag/2041/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/tag/23/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/tag/413/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/tag/442/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/tag/46/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/tag/543/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/tag/544/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/tag/690/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Indexing [http://anotherbug.blog.chinajavaworld.com/u/123297/tag/692/] with analyzer org.apache.nutch.analysis.NutchDocumentAnalyzer@462a3a (null)
Optimizing index.
merging segments _ram_1e (1 docs) _ram_1f (1 docs) _ram_1g (1 docs) _ram_1h (1 docs) _ram_1i (1 docs) _ram_1j (1 docs) _ram_1k (1 docs) _ram_1l (1 docs) _ram_1m (1 docs) _ram_1n (1 docs) _ram_1o (1 docs) _ram_1p (1 docs) _ram_1q (1 docs) _ram_1r (1 docs) _ram_1s (1 docs) _ram_1t (1 docs) _ram_1u (1 docs) _ram_1v (1 docs) _ram_1w (1 docs) _ram_1x (1 docs) _ram_1y (1 docs) into _1 (21 docs)
merging segments _0 (50 docs) _1 (21 docs) into _2 (71 docs)
Indexer: done
Dedup: starting
Dedup: adding indexes in: crawl/indexes
Dedup: done
merging indexes to: crawl/index
Adding crawl/indexes/part-00000
done merging
crawl finished: crawl

文章来自: 本站原创
标签: 安装 使用 Nutch