二：HDFS 命令指南

时间：2017-08-30 22:07:48 阅读：368 评论：0 收藏：0 [点我收藏+]

命令具体选项请参考：

http://hadoop.apache.org/docs/r2.6.3/hadoop-project-dist/hadoop-hdfs/HDFSCommands.html

1.HDFS命令通过bin/hdfs执行，语法如下：

hdfs [--config confdir] [COMMAND] [GENERIC_OPTIONS] [COMMAND_OPTIONS]

配置文件用户命令基本选项命令选项

COMMAND_OPTION	Description
`--config confdir`	Overwrites the default Configuration directory. Default is `$HADOOP_HOME/conf`.
GENERIC_OPTIONS	The common set of options supported by multiple commands. Full list is here.
COMMAND_OPTIONS	Various commands with their options are described in the following sections. The commands have been grouped into and . 来源： http://hadoop.apache.org/docs/r2.6.3/hadoop-project-dist/hadoop-hdfs/HDFSCommands.html

2.GENREIC_OPTIONS基本命令如下：

GENERIC_OPTION	Description
`-conf <configuration file>`	Specify an application configuration file. 指定配置文件
`-D <property>=<value>`	Use value for given property. 指定属性
`-jt <local> or <resourcemanager:port>`	Specify a ResourceManager. Applies only to job. 指定资源管理器
`-files <comma separated list of files>`	Specify comma separated files to be copied to the map reduce cluster. Applies only to job.
`-libjars <comma seperated list of jars>`	Specify comma separated jar files to include in the classpath. Applies only to job.
`-archives <comma separated list of archives>`	Specify comma separated archives to be unarchived on the compute machines. Applies only to job.

3.用户命令

文件系统命令：hdfs dfs [GENERIC_OPTIONS] [COMMAND_OPTIONS] 参考： File System Shell Guide.

口令命令：hdfs fetchdt [GENERIC_OPTIONS] [--webservice <namenode_http_addr>] <path>

文件检查命令：hdfs fsck [GENERIC_OPTIONS] <path> [-list-corruptfileblocks | [-move | -delete | -openforwrite] [-files [-

locks -locations | -racks]]]] [-includeSnapshots]

版本信息 :hdfs version

4.管理命令

dfsadmi

平衡：hdfs balancer [-threshold <threshold>] [-policy <policy>]

启动回滚撤销datanode ： hdfs datanode [-regular | -rollback | -rollingupgrace rollback]

dfsadmin管理：hdfs dfsadmin [GENERIC_OPTIONS]

mover数据迁移工具 :hdfs mover [-p <files/dirs> | -f <local file name>] 类似于balancer，会检查块是否满足存储要求

namenode管理：

Usage: hdfs namenode [-backup] |
          [-checkpoint] |
          [-format [-clusterid cid ] [-force] [-nonInteractive] ] |
          [-upgrade [-clusterid cid] [-renameReserved<k-v pairs>] ] |
          [-upgradeOnly [-clusterid cid] [-renameReserved<k-v pairs>] ] |
          [-rollback] |
          [-rollingUpgrade <downgrade|rollback> ] |
          [-finalize] |
          [-importCheckpoint] |
          [-initializeSharedEdits] |
          [-bootstrapStandby] |
          [-recover [-force] ] |
          [-metadataVersion ]

secondary namenode:

hdfs secondarynamenode [-checkpoint [force]] | [-format] | [-geteditsize]

COMMAND_OPTION	Description
-checkpoint [force]	Checkpoints the SecondaryNameNode if EditLog size >= fs.checkpoint.size. If `force` is used, checkpoint irrespective of EditLog size.
-format	Format the local storage during startup.
-geteditsize	Prints the number of uncheckpointed transactions on the NameNode.

注意：

sbin/start-dfs.sh实际上就是启动了namenode datanode secondarynamenode zooker等

--script "$bin/hdfs" start namenode $nameStartOpt

--script "$bin/hdfs" start datanode $dataStartOpt

--script "$bin/hdfs" start secondarynamenode

--script "$bin/hdfs" start zkfc

通过SSH连接到nanenode上启动

5.存储策略

hdfs storagepolicies 列出策略

hdfs dfsadmin -setStoragePolicy <path> <policyName> 设置策略

hdfs dfsadmin -getStoragePolicy <path> 获取策略

<wiz_tmp_tag id="wiz-table-range-border" contenteditable="false" style="display: none;">

来自为知笔记(Wiz)

二：HDFS 命令指南

原文：http://www.cnblogs.com/skyrim/p/7455532.html

踩

(0)

评论一句话评论（0）

分享档案

更多>

2021年09月23日 (328)
2021年09月24日 (313)
2021年09月17日 (191)
2021年09月15日 (369)
2021年09月16日 (411)
2021年09月13日 (439)
2021年09月11日 (398)
2021年09月12日 (393)
2021年09月10日 (160)
2021年09月08日 (222)