【From】https://helpcdn.aliyun.com/document_detail/62958.html
YARN的授权根据授权实体,可以分为服务级别的授权、队列级别的授权。
详见Hadoop官方文档。
YARN 可以通过队列对资源进行授权管理,有Capacity Scheduler和Fair Scheduler两种队列调度。
这里以Capacity Scheduler为例。
队列也有两个级别的授权,一个是提交作业到队列的授权,一个是管理队列的授权。
ACL开关,设置为true。
yarn rmadmin/yarn kill
等命令,该值必须配置,否则后续的队列相关的acl管理员设置无法生效。user1,user2 group1,group2 #user和group用空格隔开
group1,group2 #只有group情况下,必须在最前面加上空格
EMR集群中需将has配置为admin的ACL权限。
yarn.scheduler.capacity.${queue-name}.acl_submit_applications
#queue-name=root
<property>
<name>yarn.scheduler.capacity.root.acl_submit_applications</name>
<value> </value> #空格表示任何人都无法往root队列提交作业
</property>
#queue-name=root.testqueue
<property>
<name>yarn.scheduler.capacity.root.testqueue.acl_submit_applications</name>
<value>test testgrp</value> #testqueue只允许test用户/testgrp组提交作业
</property>
yarn.scheduler.capacity.${queue-name}.acl_administer_queue
#queue-name=root
<property>
<name>yarn.scheduler.capacity.root.acl_administer_queue</name>
<value> </value>
</property>
#queue-name=root.testqueue
<property>
<name>yarn.scheduler.capacity.root.testqueue.acl_administer_queue</name>
<value>test testgrp</value>
</property>
单击右上角查看操作历史查看任务进度,等待任务完成。
Key | Value |
---|---|
yarn.acl.enable | true |
yarn.admin.acl | has |
<configuration>
<property>
<name>yarn.scheduler.capacity.maximum-applications</name>
<value>10000</value>
<description>Maximum number of applications that can be pending and running.</description>
</property>
<property>
<name>yarn.scheduler.capacity.maximum-am-resource-percent</name>
<value>0.25</value>
<description>Maximum percent of resources in the cluster which can be used to run application masters i.e.
controls number of concurrent running applications.
</description>
</property>
<property>
<name>yarn.scheduler.capacity.resource-calculator</name>
<value>org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.queues</name>
<value>default,q1,q2</value>
<!-- 3个队列-->
<description>The queues at the this level (root is the root queue).</description>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.capacity</name>
<value>0</value>
<description>Default queue target capacity.</description>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.user-limit-factor</name>
<value>1</value>
<description>Default queue user limit a percentage from 0.0 to 1.0.</description>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.maximum-capacity</name>
<value>100</value>
<description>The maximum capacity of the default queue.</description>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.state</name>
<value>STOPPED</value>
<!-- default队列状态设置为STOPPED-->
<description>The state of the default queue. State can be one of RUNNING or STOPPED.</description>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.acl_submit_applications</name>
<value> </value>
<!-- default队列禁止提交作业-->
<description>The ACL of who can submit jobs to the default queue.</description>
</property>
<property>
<name>yarn.scheduler.capacity.root.default.acl_administer_queue</name>
<value> </value>
<!-- 禁止管理default队列-->
<description>The ACL of who can administer jobs on the default queue.</description>
</property>
<property>
<name>yarn.scheduler.capacity.node-locality-delay</name>
<value>40</value>
</property>
<property>
<name>yarn.scheduler.capacity.queue-mappings</name>
<value>u:test:q1,u:foo:q2</value>
<!-- 队列映射,test用户自动映射到q1队列-->
<description>A list of mappings that will be used to assign jobs to queues. The syntax for this list is
[u|g]:[name]:[queue_name][,next mapping]* Typically this list will be used to map users to queues,for
example, u:%user:%user maps all users to queues with the same name as the user.
</description>
</property>
<property>
<name>yarn.scheduler.capacity.queue-mappings-override.enable</name>
<value>true</value>
<!-- 上述queue-mappings设置的映射,是否覆盖客户端设置的队列参数-->
<description>If a queue mapping is present, will it override the value specified by the user? This can be used
by administrators to place jobs in queues that are different than the one specified by the user. The default
is false.
</description>
</property>
<property>
<name>yarn.scheduler.capacity.root.acl_submit_applications</name>
<value> </value>
<!-- ACL继承性,父队列需控制住权限-->
<description>
The ACL of who can submit jobs to the root queue.
</description>
</property>
<property>
<name>yarn.scheduler.capacity.root.q1.acl_submit_applications</name>
<value>test</value>
<!-- q1只允许test用户提交作业-->
</property>
<property>
<name>yarn.scheduler.capacity.root.q2.acl_submit_applications</name>
<