首页 > Web开发 > 详细

【Jstorm 入门例子】

时间:2016-07-24 02:12:32      阅读:983      评论:0      收藏:0      [点我收藏+]

一、检查硬盘空间,启动jstorm环境

[root@hadoop0 ~]# cd /opt/bigdata/

[root@hadoop0 bigdata]# ls

apache-flume-1.6.0-bin ? ? ? ? apache-hive-2.0.1-bin.tar.gz ?hadoop272 ? ?

hbase-1.1.5-bin.tar.gz ?jstorm-0.9.6.2 ? ? ?kafka ? ? ? ?sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz ?taokeeper-monitor.tar.gz ?zookeeper ? ?apache-flume-1.6.0-bin.tar.gz ?apache-tomcat-7.0.69.zip ? ? ?hbase-1.1.5 ?hive2.0 ? ? ? ? ? ? ? ? jstorm-0.9.6.2.zip ?sqoop-1.4.6 ?stomr096 ? ? tomcat7

[root@hadoop0 bigdata]# df -h

Filesystem ? ? ? ? ? ? ? ? ? ?Size ?Used Avail Use% Mounted on

/dev/mapper/VolGroup-lv_root ? 18G ?9.3G ?7.2G ?57% /

tmpfs ? ? ? ? ? ? ? ? ? ? ? ? 250M ? 72K ?250M ? 1% /dev/shm

/dev/sda1 ? ? ? ? ? ? ? ? ? ? 485M ? 33M ?427M ? 8% /boot

1)启动zookeeper环境

[root@hadoop0 bigdata]# cd zookeeper/bin/

[root@hadoop0 bin]# ./zkServer.sh start

JMX enabled by default

Using config: /opt/bigdata/zookeeper/bin/../conf/zoo.cfg

Starting zookeeper ... STARTED

2)启动Tomcat环境

[root@hadoop0 bin]# cd ../../tomcat7/bin/

[root@hadoop0 bin]# ./startup.sh?

Using CATALINA_BASE: ? /opt/bigdata/tomcat7

Using CATALINA_HOME: ? /opt/bigdata/tomcat7

Using CATALINA_TMPDIR: /opt/bigdata/tomcat7/temp

Using JRE_HOME: ? ? ? ?/opt/jdk1.7.0_79

Using CLASSPATH: ? ? ? /opt/bigdata/tomcat7/bin/bootstrap.jar:/opt/bigdata/tomcat7/bin/tomcat-juli.jar

Tomcat started.

3)启动storm的nimbus 和supervisor?

[root@hadoop0 bin]# cd ../../jstorm-0.9.6.2/bin/

[root@hadoop0 bin]# ./jstorm nimbus &

[1] 1985

[root@hadoop0 bin]# ./jstorm sRunning: java -server -Djstorm.home=/opt/bigdata/jstorm-0.9.6.2 -Dstorm.options= -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib ?-Xms2g -Xmx2g -Xmn768m -XX:PermSize=128m ?-XX:SurvivorRatio=4 -XX:MaxTenuringThreshold=20 -XX:+UseConcMarkSweepGC ?-XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 -XX:CMSFullGCsBeforeCompaction=5 -XX:+HeapDumpOnOutOfMemoryError ?-XX:+UseCMSCompactAtFullCollection -XX:CMSMaxAbortablePrecleanTime=5000 ?-Dlogfile.name=nimbus.log -Dlog4j.configuration=File:/opt/bigdata/jstorm-0.9.6.2/conf/jstorm.log4j.properties ?-cp /opt/bigdata/jstorm-0.9.6.2/jstorm-client-extension-0.9.6.2.jar:/opt/bigdata/jstorm-0.9.6.2/jstorm-client-0.9.6.2.jar:/opt/bigdata/jstorm-0.9.6.2/jstorm-server-0.9.6.2.jar:/opt/bigdata/jstorm-0.9.6.2/lib/minlog-1.2.jar:/opt/bigdata/jstorm-0.9.6.2/lib/commons-logging-1.1.3.jar:/opt/bigdata/jstorm-0.9.6.2/lib/commons-cli-1.2.jar:/opt/bigdata/jstorm-0.9.6.2/lib/slf4j-api-1.7.5.jar:/opt/bigdata/jstorm-0.9.6.2/lib/libthrift7-0.7.0.jar:/opt/bigdata/jstorm-0.9.6.2/lib/jline-0.9.94.jar:/opt/bigdata/jstorm-0.9.6.2/lib/objenesis-1.2.jar:/opt/bigdata/jstorm-0.9.6.2/lib/disruptor-3.2.1.jar:/opt/bigdata/jstorm-0.9.6.2/lib/curator-client-2.5.0.jar:/opt/bigdata/jstorm-0.9.6.2/lib/fastjson-1.1.41.jar:/opt/bigdata/jstorm-0.9.6.2/lib/commons-exec-1.1.jar:/opt/bigdata/jstorm-0.9.6.2/lib/kryo-2.17.jar:/opt/bigdata/jstorm-0.9.6.2/lib/clj-time-0.4.1.jar:/opt/bigdata/jstorm-0.9.6.2/lib/json-simple-1.1.jar:/opt/bigdata/jstorm-0.9.6.2/lib/httpclient-4.3.2.jar:/opt/bigdata/jstorm-0.9.6.2/lib/plexus-utils-1.5.5.jar:/opt/bigdata/jstorm-0.9.6.2/lib/plexus-compiler-javac-1.8.1.jar:/opt/bigdata/jstorm-0.9.6.2/lib/zookeeper-3.4.6.jar:/opt/bigdata/jstorm-0.9.6.2/lib/joda-time-2.0.jar:/opt/bigdata/jstorm-0.9.6.2/lib/slf4j-log4j12-1.7.5.jar:/opt/bigdata/jstorm-0.9.6.2/lib/log4j-1.2.16.jar:/opt/bigdata/jstorm-0.9.6.2/lib/carbonite-1.5.0.jar:/opt/bigdata/jstorm-0.9.6.2/lib/commons-lang-2.5.jar:/opt/bigdata/jstorm-0.9.6.2/lib/netty-3.9.0.Final.jar:/opt/bigdata/jstorm-0.9.6.2/lib/http-20070405.jar:/opt/bigdata/jstorm-0.9.6.2/lib/commons-codec-1.6.jar:/opt/bigdata/jstorm-0.9.6.2/lib/metrics-core-3.0.1.jar:/opt/bigdata/jstorm-0.9.6.2/lib/asm-4.0.jar:/opt/bigdata/jstorm-0.9.6.2/lib/metrics-jvm-3.0.1.jar:/opt/bigdata/jstorm-0.9.6.2/lib/guava-16.0.1.jar:/opt/bigdata/jstorm-0.9.6.2/lib/clojure-1.5.1.jar:/opt/bigdata/jstorm-0.9.6.2/lib/jgrapht-0.8.3.jar:/opt/bigdata/jstorm-0.9.6.2/lib/plexus-compiler-api-1.8.1.jar:/opt/bigdata/jstorm-0.9.6.2/lib/jzmq-2.1.0.jar:/opt/bigdata/jstorm-0.9.6.2/lib/curator-framework-2.5.0.jar:/opt/bigdata/jstorm-0.9.6.2/lib/commons-io-2.4.jar:/opt/bigdata/jstorm-0.9.6.2/lib/servlet-api-2.5.jar:/opt/bigdata/jstorm-0.9.6.2/lib/httpcore-4.3.1.jar:/opt/bigdata/jstorm-0.9.6.2/lib/reflectasm-1.07-shaded.jar:/opt/bigdata/jstorm-0.9.6.2/lib/snakeyaml-1.11.jar:/opt/bigdata/jstorm-0.9.6.2/conf: com.alibaba.jstorm.daemon.nimbus.NimbusServer?

?

[root@hadoop0 bin]# ./jstorm supervisor &

[2] 2072

[root@hadoop0 bin]# Running: java -server -Djstorm.home=/opt/bigdata/jstorm-0.9.6.2 -Dstorm.options= -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib ?-Xms512m -Xmx512m -Xmn128m -XX:PermSize=64m -XX:+UseConcMarkSweepGC ?-XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 -XX:CMSFullGCsBeforeCompaction=5 -XX:+HeapDumpOnOutOfMemoryError ?-XX:+UseCMSCompactAtFullCollection -XX:CMSMaxAbortablePrecleanTime=5000 ?-Dlogfile.name=supervisor.log -Dlog4j.configuration=File:/opt/bigdata/jstorm-0.9.6.2/conf/jstorm.log4j.properties ?-cp /opt/bigdata/jstorm-0.9.6.2/jstorm-client-extension-0.9.6.2.jar:/opt/bigdata/jstorm-0.9.6.2/jstorm-client-0.9.6.2.jar:/opt/bigdata/jstorm-0.9.6.2/jstorm-server-0.9.6.2.jar:/opt/bigdata/jstorm-0.9.6.2/lib/minlog-1.2.jar:/opt/bigdata/jstorm-0.9.6.2/lib/commons-logging-1.1.3.jar:/opt/bigdata/jstorm-0.9.6.2/lib/commons-cli-1.2.jar:/opt/bigdata/jstorm-0.9.6.2/lib/slf4j-api-1.7.5.jar:/opt/bigdata/jstorm-0.9.6.2/lib/libthrift7-0.7.0.jar:/opt/bigdata/jstorm-0.9.6.2/lib/jline-0.9.94.jar:/opt/bigdata/jstorm-0.9.6.2/lib/objenesis-1.2.jar:/opt/bigdata/jstorm-0.9.6.2/lib/disruptor-3.2.1.jar:/opt/bigdata/jstorm-0.9.6.2/lib/curator-client-2.5.0.jar:/opt/bigdata/jstorm-0.9.6.2/lib/fastjson-1.1.41.jar:/opt/bigdata/jstorm-0.9.6.2/lib/commons-exec-1.1.jar:/opt/bigdata/jstorm-0.9.6.2/lib/kryo-2.17.jar:/opt/bigdata/jstorm-0.9.6.2/lib/clj-time-0.4.1.jar:/opt/bigdata/jstorm-0.9.6.2/lib/json-simple-1.1.jar:/opt/bigdata/jstorm-0.9.6.2/lib/httpclient-4.3.2.jar:/opt/bigdata/jstorm-0.9.6.2/lib/plexus-utils-1.5.5.jar:/opt/bigdata/jstorm-0.9.6.2/lib/plexus-compiler-javac-1.8.1.jar:/opt/bigdata/jstorm-0.9.6.2/lib/zookeeper-3.4.6.jar:/opt/bigdata/jstorm-0.9.6.2/lib/joda-time-2.0.jar:/opt/bigdata/jstorm-0.9.6.2/lib/slf4j-log4j12-1.7.5.jar:/opt/bigdata/jstorm-0.9.6.2/lib/log4j-1.2.16.jar:/opt/bigdata/jstorm-0.9.6.2/lib/carbonite-1.5.0.jar:/opt/bigdata/jstorm-0.9.6.2/lib/commons-lang-2.5.jar:/opt/bigdata/jstorm-0.9.6.2/lib/netty-3.9.0.Final.jar:/opt/bigdata/jstorm-0.9.6.2/lib/http-20070405.jar:/opt/bigdata/jstorm-0.9.6.2/lib/commons-codec-1.6.jar:/opt/bigdata/jstorm-0.9.6.2/lib/metrics-core-3.0.1.jar:/opt/bigdata/jstorm-0.9.6.2/lib/asm-4.0.jar:/opt/bigdata/jstorm-0.9.6.2/lib/metrics-jvm-3.0.1.jar:/opt/bigdata/jstorm-0.9.6.2/lib/guava-16.0.1.jar:/opt/bigdata/jstorm-0.9.6.2/lib/clojure-1.5.1.jar:/opt/bigdata/jstorm-0.9.6.2/lib/jgrapht-0.8.3.jar:/opt/bigdata/jstorm-0.9.6.2/lib/plexus-compiler-api-1.8.1.jar:/opt/bigdata/jstorm-0.9.6.2/lib/jzmq-2.1.0.jar:/opt/bigdata/jstorm-0.9.6.2/lib/curator-framework-2.5.0.jar:/opt/bigdata/jstorm-0.9.6.2/lib/commons-io-2.4.jar:/opt/bigdata/jstorm-0.9.6.2/lib/servlet-api-2.5.jar:/opt/bigdata/jstorm-0.9.6.2/lib/httpcore-4.3.1.jar:/opt/bigdata/jstorm-0.9.6.2/lib/reflectasm-1.07-shaded.jar:/opt/bigdata/jstorm-0.9.6.2/lib/snakeyaml-1.11.jar:/opt/bigdata/jstorm-0.9.6.2/conf com.alibaba.jstorm.daemon.supervisor.Supervisor?

./jstorm supervisor &kill 32721: No such process


bubuko.com,布布扣
?

二、开发应用程序

?参见附件,storm程序不用更改任何代码直接部署使用

永久链接:?http://gaojingsong.iteye.com/blog/2289928

三、部署应用

[root@hadoop0 bin]# ./jstorm jar /opt/bigdata/stormtest-0.0.1-SNAPSHOT.jar stormdemo.stormtest.App

Running: java -client -Xms256m -Xmx256m -Djstorm.home=/opt/bigdata/jstorm-0.9.6.2 -Dstorm.options= -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.jar=/opt/bigdata/stormtest-0.0.1-SNAPSHOT.jar -Dstorm.root.logger=INFO,stdout -Dlog4j.configuration=File:/opt/bigdata/jstorm-0.9.6.2/conf/aloha_log4j.properties -cp /opt/bigdata/jstorm-0.9.6.2/jstorm-client-extension-0.9.6.2.jar:/opt/bigdata/jstorm-0.9.6.2/jstorm-client-0.9.6.2.jar:/opt/bigdata/jstorm-0.9.6.2/jstorm-server-0.9.6.2.jar:/opt/bigdata/jstorm-0.9.6.2/lib/minlog-1.2.jar:/opt/bigdata/jstorm-0.9.6.2/lib/commons-logging-1.1.3.jar:/opt/bigdata/jstorm-0.9.6.2/lib/commons-cli-1.2.jar:/opt/bigdata/jstorm-0.9.6.2/lib/slf4j-api-1.7.5.jar:/opt/bigdata/jstorm-0.9.6.2/lib/libthrift7-0.7.0.jar:/opt/bigdata/jstorm-0.9.6.2/lib/jline-0.9.94.jar:/opt/bigdata/jstorm-0.9.6.2/lib/objenesis-1.2.jar:/opt/bigdata/jstorm-0.9.6.2/lib/disruptor-3.2.1.jar:/opt/bigdata/jstorm-0.9.6.2/lib/curator-client-2.5.0.jar:/opt/bigdata/jstorm-0.9.6.2/lib/fastjson-1.1.41.jar:/opt/bigdata/jstorm-0.9.6.2/lib/commons-exec-1.1.jar:/opt/bigdata/jstorm-0.9.6.2/lib/kryo-2.17.jar:/opt/bigdata/jstorm-0.9.6.2/lib/clj-time-0.4.1.jar:/opt/bigdata/jstorm-0.9.6.2/lib/json-simple-1.1.jar:/opt/bigdata/jstorm-0.9.6.2/lib/httpclient-4.3.2.jar:/opt/bigdata/jstorm-0.9.6.2/lib/plexus-utils-1.5.5.jar:/opt/bigdata/jstorm-0.9.6.2/lib/plexus-compiler-javac-1.8.1.jar:/opt/bigdata/jstorm-0.9.6.2/lib/zookeeper-3.4.6.jar:/opt/bigdata/jstorm-0.9.6.2/lib/joda-time-2.0.jar:/opt/bigdata/jstorm-0.9.6.2/lib/slf4j-log4j12-1.7.5.jar:/opt/bigdata/jstorm-0.9.6.2/lib/log4j-1.2.16.jar:/opt/bigdata/jstorm-0.9.6.2/lib/carbonite-1.5.0.jar:/opt/bigdata/jstorm-0.9.6.2/lib/commons-lang-2.5.jar:/opt/bigdata/jstorm-0.9.6.2/lib/netty-3.9.0.Final.jar:/opt/bigdata/jstorm-0.9.6.2/lib/http-20070405.jar:/opt/bigdata/jstorm-0.9.6.2/lib/commons-codec-1.6.jar:/opt/bigdata/jstorm-0.9.6.2/lib/metrics-core-3.0.1.jar:/opt/bigdata/jstorm-0.9.6.2/lib/asm-4.0.jar:/opt/bigdata/jstorm-0.9.6.2/lib/metrics-jvm-3.0.1.jar:/opt/bigdata/jstorm-0.9.6.2/lib/guava-16.0.1.jar:/opt/bigdata/jstorm-0.9.6.2/lib/clojure-1.5.1.jar:/opt/bigdata/jstorm-0.9.6.2/lib/jgrapht-0.8.3.jar:/opt/bigdata/jstorm-0.9.6.2/lib/plexus-compiler-api-1.8.1.jar:/opt/bigdata/jstorm-0.9.6.2/lib/jzmq-2.1.0.jar:/opt/bigdata/jstorm-0.9.6.2/lib/curator-framework-2.5.0.jar:/opt/bigdata/jstorm-0.9.6.2/lib/commons-io-2.4.jar:/opt/bigdata/jstorm-0.9.6.2/lib/servlet-api-2.5.jar:/opt/bigdata/jstorm-0.9.6.2/lib/httpcore-4.3.1.jar:/opt/bigdata/jstorm-0.9.6.2/lib/reflectasm-1.07-shaded.jar:/opt/bigdata/jstorm-0.9.6.2/lib/snakeyaml-1.11.jar:/opt/bigdata/stormtest-0.0.1-SNAPSHOT.jar:/root/.jstorm:/opt/bigdata/jstorm-0.9.6.2/bin:/opt/bigdata/jstorm-0.9.6.2/conf/jstorm.log4j.properties stormdemo.stormtest.App?

====Bolt==declarer========

======Spout=====declareOutputFields========

[INFO ?1999-06-12 20:05:14 CuratorFrameworkImpl:238 main] Starting

[INFO ?1999-06-12 20:05:15 ConnectionStateManager:194 main-EventThread] State change: CONNECTED

[INFO ?1999-06-12 20:05:16 ThriftClient:62 main] masterHost:hadoop0:7627

[INFO ?1999-06-12 20:05:16 ThriftClient:86 main] Begin to connect hadoop0:7627

[INFO ?1999-06-12 20:05:16 CuratorFrameworkImpl:238 main] Starting

[INFO ?1999-06-12 20:05:16 ConnectionStateManager:194 main-EventThread] State change: CONNECTED

[INFO ?1999-06-12 20:05:17 ThriftClient:62 main] masterHost:hadoop0:7627

[INFO ?1999-06-12 20:05:17 ThriftClient:86 main] Begin to connect hadoop0:7627

[INFO ?1999-06-12 20:05:17 CuratorFrameworkImpl:238 main] Starting

[INFO ?1999-06-12 20:05:17 ConnectionStateManager:194 main-EventThread] State change: CONNECTED

[INFO ?1999-06-12 20:05:18 ThriftClient:62 main] masterHost:hadoop0:7627

[INFO ?1999-06-12 20:05:18 ThriftClient:86 main] Begin to connect hadoop0:7627

[INFO ?1999-06-12 20:05:18 StormSubmitter:244 main] Jar not uploaded to master yet. Submitting jar...

[INFO ?1999-06-12 20:05:18 StormSubmitter:286 main] Uploading topology jar /opt/bigdata/stormtest-0.0.1-SNAPSHOT.jar to assigned location: /opt/bigdata/jstorm-0.9.6.2/data/nimbus/inbox/b44ccc23-d47d-47ac-92e8-1b260310a268/stormjar-b44ccc23-d47d-47ac-92e8-1b260310a268.jar

[INFO ?1999-06-12 20:05:18 StormSubmitter:305 main] Successfully uploaded topology jar to assigned location: /opt/bigdata/jstorm-0.9.6.2/data/nimbus/inbox/b44ccc23-d47d-47ac-92e8-1b260310a268/stormjar-b44ccc23-d47d-47ac-92e8-1b260310a268.jar

[INFO ?1999-06-12 20:05:18 StormSubmitter:135 main] Submitting topology simple in distributed mode with conf {"user.password":null,"user.name":null,"topology.workers":1,"topology.debug":true,"user.group":null}

[INFO ?1999-06-12 20:05:18 StormSubmitter:149 main] Finished submitting topology: simple

[root@hadoop0 bin]#?

?

四、验证

?


bubuko.com,布布扣
?

?

?

?


bubuko.com,布布扣
?

?

?

?


bubuko.com,布布扣
?

【Jstorm 入门例子】

原文:http://gaojingsong.iteye.com/blog/2312491

(0)
(0)
   
举报
评论 一句话评论(0
关于我们 - 联系我们 - 留言反馈 - 联系我们:wmxa8@hotmail.com
© 2014 bubuko.com 版权所有
打开技术之扣,分享程序人生!