首页 > 其他 > 详细

Hadoop RPC protocol description--转

时间:2015-08-31 21:11:31      阅读:194      评论:0      收藏:0      [点我收藏+]

原文地址:https://spotify.github.io/snakebite/hadoop_rpc.html

Snakebite currently implements the following protocol in snakebite.channel.SocketRpcChannel to communicate with the NameNode.

Connection

The Hadoop RPC protocol works as described below. On connection, headers are sent to setup a session. After that, multiple requests can be sent within the session.

FunctionTypeDefault
Header bytes “hrpc”
Version uint8 7
Auth method uint8 80 (Auth method SIMPLE)
Serialization type uint8 0 (protobuf)
IpcConnectionContextProto length uint32  
IpcConnectionContextProto bytes  

Sending messages

When sending a message, the following is sent to the sever:

FunctionType
Length of the next two parts uint32
RpcPayloadHeaderProto length varint
RpcPayloadHeaderProto protobuf serialized message
HadoopRpcRequestProto length varint
HadoopRpcRequestProto protobuf serialized message

varint is a Protocol Buffer variable int.

Note

The Java protobuf implementation uses writeToDelimited to prepend the message with their lenght, but the python implementation doesn’t implement such a method (yet).

Next to an rpcKind (snakebites default is RPC_PROTOCOL_BUFFER), an rpcOp (snakebites default isRPC_FINAL_PAYLOAD), the RpcPayloadHeaderProto message defines a callId that is added in the RPC response (described below).

The HadoopRpcRequestProto contains a methodName field that defines what server method is called and a has a property request that contains the serialized actual request message.

Receiving messages

After a message is sent, the response can be read in the following way:

FunctionType
Length of the RpcResponseHeaderProto varint
RpcResponseHeaderProto bytes
Length of the RPC response uint32
Serialized RPC response bytes

The RpcResponseHeaderProto contains the callId of the request and a status field. The status can beSUCCESSERROR or FAILURE. In case SUCCESS the rest of response is a complete protobuf response.

In case of ERROR, the response looks like follows:

FunctionType
Length of the RpcResponseHeaderProto varint
RpcResponseHeaderProto bytes
Length of the RPC response uint32
Length of the Exeption class name uint32
Exception class name utf-8 string
Length of the stack trace uint32
Stack trace utf-8 string

 

Hadoop RPC protocol description--转

原文:http://www.cnblogs.com/davidwang456/p/4773884.html

(0)
(0)
   
举报
评论 一句话评论(0
关于我们 - 联系我们 - 留言反馈 - 联系我们:wmxa8@hotmail.com
© 2014 bubuko.com 版权所有
打开技术之扣,分享程序人生!