首页 > Web开发 > 详细

netty权威指南学习笔记六——编解码技术之MessagePack

时间:2018-07-22 12:50:09      阅读:219      评论:0      收藏:0      [点我收藏+]

  编解码技术主要应用在网络传输中,将对象比如BOJO进行编解码以利于网络中进行传输。平常我们也会将编解码说成是序列化/反序列化

  定义:当进行远程跨进程服务调用时,需要把被传输的java对象编码为字节数组或者ByteBuffer对象。而当远程服务读取到ByteBuffer对象或者字节数组时,需要将其解码为发送时的java对象。这被称为java对象编解码技术。比如java的序列化。

  但是,java的序列化有一定的弊端;

  •   java序列化是java私有的协议,其他语言不支持,故而不能实现跨语言;
  •   其次,序列化后的码流太大;
  •   再次,序列化性能太低,耗时长。

  因此,通常不会选择java序列化作为远程跨节点调用的编解码框架。

  当前业界主流的编解码框架有:1)MessagePack高效的二进制序列化框架;2)Google 的Protobuf;3)Facebook的Thrift;4)JBoss Marshalliing

下面运行MessagePack的编解码

  这个示例在权威指南上,作者并没有给出完整代码,本博主刚开始运行也没有运行出来,经过网络搜索,参考相关文章运行了出来,其中潜在存在着一些坑,运行中本博主也发现一些现象也总结出来。

  一、首先我们需要引入相关jar包

 1         <dependency>
 2             <groupId>org.msgpack</groupId>
 3             <artifactId>msgpack</artifactId>
 4             <version>0.6.11</version>
 5 
 6         </dependency>
 7         <!-- 创建项目时已经存在,这里贴出来,但本博主不重复放包
 8               <dependency>
 9                   <groupId>org.javassist</groupId>
10                   <artifactId>javassist</artifactId>
11                   <version>3.22.0-GA</version>
12               </dependency>
13       -->

二、放上编解码代码,

坑一、一定要在继承的类后面加上泛型,这样,方法中的参数才能跟着发生变化

如过没有添加<>,则解码实现接口方法时直接生成的方法的参数如下:

编码代码

 1 package com.messagePack;
 2 
 3 import io.netty.buffer.ByteBuf;
 4 import io.netty.channel.ChannelHandlerContext;
 5 import io.netty.handler.codec.MessageToByteEncoder;
 6 import org.msgpack.MessagePack;
 7 
 8 public class MsgPackEncoder extends MessageToByteEncoder<Object> {
 9     @Override
10     protected void encode(ChannelHandlerContext channelHandlerContext, Object o, ByteBuf byteBuf) throws Exception {
11         MessagePack msgPack = new MessagePack();
12 //      编码,然后转为ButyBuf传递
13         byte[] bytes = msgPack.write(o);
14         byteBuf.writeBytes(bytes);
15     }
16 }

解码代码

 1 package com.messagePack;
 2 
 3 import io.netty.buffer.ByteBuf;
 4 import io.netty.channel.ChannelHandlerContext;
 5 import io.netty.handler.codec.MessageToMessageDecoder;
 6 import org.msgpack.MessagePack;
 7 
 8 import java.util.List;
 9 
10 public class MsgPackDecoder extends MessageToMessageDecoder<ByteBuf> {
11     @Override
12     protected void decode(ChannelHandlerContext channelHandlerContext, ByteBuf byteBuf, List<Object> list) throws Exception {
13 //      获取要解码的byte数组
14         final byte[] bytes;
15         final int length = byteBuf.readableBytes();
16         bytes = new byte[length];
17         byteBuf.getBytes(byteBuf.readerIndex(),bytes,0,length);
18 //      调用MessagePack 的read方法将其反序列化为Object对象
19         MessagePack msgPack = new MessagePack();
20         list.add(msgPack.read(bytes));
21     }
22 }

如果实现的接口后面没有添加泛型<ByteBuf>,则解码实现接口方法时直接生成的方法的参数如下:

1  @Override
2     protected void decode(ChannelHandlerContext channelHandlerContext, Object o, List list) throws Exception {
3 
4     }

 不过经过实验,将接收到的Object进行转换后仍然可以,要记好netty接收和传递信息都是经过ByteBuf进行的

 1     @Override
 2     protected void decode(ChannelHandlerContext channelHandlerContext, Object o, List list) throws Exception {
 3         ByteBuf byteBuf = (ByteBuf) o;
 4 //      获取要解码的byte数组
 5         final byte[] bytes;
 6         final int length = byteBuf.readableBytes();
 7         bytes = new byte[length];
 8         byteBuf.getBytes(byteBuf.readerIndex(),bytes,0,length);
 9 //      调用MessagePack 的read方法将其反序列化为Object对象
10         MessagePack msgPack = new MessagePack();
11         list.add(msgPack.read(bytes));
12     }

三、服务端代码

 1 package com.messagePack;
 2 
 3 import io.netty.bootstrap.ServerBootstrap;
 4 import io.netty.channel.ChannelFuture;
 5 import io.netty.channel.ChannelInitializer;
 6 import io.netty.channel.ChannelOption;
 7 import io.netty.channel.nio.NioEventLoopGroup;
 8 import io.netty.channel.socket.SocketChannel;
 9 import io.netty.channel.socket.nio.NioServerSocketChannel;
10 import io.netty.handler.logging.LogLevel;
11 import io.netty.handler.logging.LoggingHandler;
12 
13 public class EchoServer {
14     public void bind(int port) throws InterruptedException {
15         NioEventLoopGroup bossGroup = new NioEventLoopGroup();
16         NioEventLoopGroup workGroup = new NioEventLoopGroup();
17         try {
18             ServerBootstrap b = new ServerBootstrap();
19             b.group(bossGroup,workGroup)
20                     .channel(NioServerSocketChannel.class)
21                     .option(ChannelOption.SO_BACKLOG,1024)
22                     .childHandler(new LoggingHandler(LogLevel.INFO))
23                     .childHandler(new ChannelInitializer<SocketChannel>() {
24                         @Override
25                         protected void initChannel(SocketChannel socketChannel) throws Exception {
26                             socketChannel.pipeline()
27                                     .addLast("decoder",new MsgPackDecoder())
28                                     .addLast("encoder",new MsgPackEncoder())
29                                     .addLast(new EchoServerHandler());
30                         }
31                     });
32 //          绑定端口,同步等待成功
33             ChannelFuture f = b.bind(port).sync();
34 //          等待服务端监听端口关闭
35             f.channel().closeFuture().sync();
36         } finally {
37             bossGroup.shutdownGracefully();
38             workGroup.shutdownGracefully();
39         }
40     }
41     public static void main(String[] args) throws InterruptedException {
42         int port = 8080;
43         if(args.length>0&&args!=null){
44             port = Integer.parseInt(args[0]);
45         }
46         new EchoServer().bind(port);
47 
48     }
49 }

IO处理

 1 package com.messagePack;
 2 
 3 import io.netty.buffer.ByteBuf;
 4 import io.netty.buffer.Unpooled;
 5 import io.netty.channel.ChannelHandlerContext;
 6 import io.netty.channel.ChannelInboundHandlerAdapter;
 7 
 8 import java.util.List;
 9 
10 public class EchoServerHandler extends ChannelInboundHandlerAdapter {
11     int count;
12     @Override
13     public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
14         System.out.println("server receive the msgpack message : "+msg+"");
15         // 原路返回给客户端
16         ctx.writeAndFlush(msg);
17 /*        在EchoClientHandler中向服务端发送一个pojo对象,经过MessagePack编解码后,
18         在EchoServerHandler中的channelRead方法中打印的msg为pojo对象的toString方法内容,
19         不可以直接将msg转换为User,如果采用如下代码运行不成功*/
20 /*        List<User> users = (List<User>) msg;
21         System.out.println("到这里面来了,users是否为空:");
22         System.out.println(users!=null);
23         for(User u : users){
24             System.out.println("This is"+ ++count +" times server receive client request."+u);
25             ctx.write(u);
26         }
27 
28         ctx.flush();*/
29     }
30 
31     @Override
32     public void channelReadComplete(ChannelHandlerContext ctx) throws Exception {
33         ctx.flush();
34     }
35 
36     @Override
37     public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
38         ctx.close();
39     }
40 }

坑二、这里的坑比较大,在上述代码注释中已经说明了,下面运行结果是注释掉以上14~16行代码,放开20~28行代码时候运行结果,结果表明收到的消息转化为User数组时候,是空的,但是后台并没有报错,不知道为什么在我的IDEA上运行不下去但是不报错

技术分享图片

四、客户端代码

 1 package com.messagePack;
 2 
 3 import io.netty.bootstrap.Bootstrap;
 4 import io.netty.buffer.ByteBuf;
 5 import io.netty.buffer.Unpooled;
 6 import io.netty.channel.ChannelFuture;
 7 import io.netty.channel.ChannelInitializer;
 8 import io.netty.channel.ChannelOption;
 9 import io.netty.channel.nio.NioEventLoopGroup;
10 import io.netty.channel.socket.SocketChannel;
11 import io.netty.channel.socket.nio.NioSocketChannel;
12 import io.netty.handler.codec.DelimiterBasedFrameDecoder;
13 import io.netty.handler.codec.string.StringDecoder;
14 
15 public class EchoClient {
16     public void connection(int port,String host) throws InterruptedException {
17         NioEventLoopGroup workGroup = new NioEventLoopGroup();
18         try {
19             Bootstrap b = new Bootstrap();
20             b.group(workGroup)
21                     .channel(NioSocketChannel.class)
22                     .option(ChannelOption.TCP_NODELAY,true)
23                     .option(ChannelOption.CONNECT_TIMEOUT_MILLIS,3000)
24                     .handler(new ChannelInitializer<SocketChannel>() {
25                         @Override
26                         protected void initChannel(SocketChannel socketChannel) throws Exception {
27                             socketChannel.pipeline().addLast("msgpack decoder",new MsgPackDecoder())
28                                     .addLast("msgpack encoder",new MsgPackEncoder())
29                                     .addLast(new EchoClientHandler());
30 //
31                         }
32                     });
33 //            发起异步连接操作
34             ChannelFuture f = b.connect(host,port).sync();
35 //                          等待客户端链路关闭
36             f.channel().closeFuture().sync();
37         } finally {
38             workGroup.shutdownGracefully();
39         }
40     }
41     public static void main(String[] args) throws InterruptedException {
42         int port = 8080;
43         if(args.length>0&&args!=null){
44             System.out.println(args[0]);
45             port = Integer.parseInt(args[0]);
46         }
47         new EchoClient().connection(port,"127.0.0.1");
48     }
49 }

IO处理类

 1 package com.messagePack;
 2 
 3 import io.netty.channel.ChannelHandlerContext;
 4 import io.netty.channel.ChannelInboundHandlerAdapter;
 5 
 6 public class EchoClientHandler extends ChannelInboundHandlerAdapter {
 7     private int count;
 8 
 9     @Override
10     public void channelActive(ChannelHandlerContext ctx) throws Exception {
11        /* User user = getUser();
12         ctx.writeAndFlush(user);*/
13         User[] users = getUsers();
14         for(User u : users){
15             ctx.write(u);
16         }
17         ctx.flush();
18     }
19 
20     @Override
21     public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
22         System.out.println("this is client receive msg【  "+ ++count +"  】times:【"+msg+"】");
23        if(count<5){ //控制运行次数,因为不加这个控制直接调用下面代码的话,客户端和服务端会形成闭环循环,一直运行
24         ctx.write(msg);
25         }
26     }
27 
28     @Override
29     public void channelReadComplete(ChannelHandlerContext ctx) throws Exception {
30         ctx.flush();
31     }
32 
33     @Override
34     public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
35         ctx.close();
36     }
37     private User[] getUsers(){
38         User[] users = new User[5];
39         for(int i=0;i<5;i++){
40             User user = new User();
41             user.setId(String.valueOf(i));
42             user.setAge(18+i);
43             user.setName("张元"+i);
44             user.setSex("男"+String.valueOf(i*2));
45             users[i]=user;
46         }
47         return users;
48     }
49 
50     private User getUser(){
51             User user = new User();
52             user.setId("11");
53             user.setAge(18);
54             user.setName("张元");
55             user.setSex("男");
56         return user;
57     }
58 }

五、User代码

坑三、要传输的javabean一定要加上注解@message

 1 package com.messagePack;
 2 
 3 import org.msgpack.annotation.Message;
 4 
 5 @Message
 6 public class User {
 7     private String name;
 8     private int age;
 9     private String id;
10     private String sex;
11 
12     public int getAge() {
13         return age;
14     }
15 
16     public void setAge(int age) {
17         this.age = age;
18     }
19 
20     public String getId() {
21         return id;
22     }
23 
24     public void setId(String id) {
25         this.id = id;
26     }
27 
28     public String getSex() {
29         return sex;
30     }
31 
32     public void setSex(String sex) {
33         this.sex = sex;
34     }
35 
36     public String getName() {
37         return name;
38     }
39 
40     public void setName(String name) {
41         this.name = name;
42     }
43 
44     @Override
45     public String toString() {
46         return "User{" +
47                 "name=‘" + name + ‘\‘‘ +
48                 ", age=" + age +
49                 ", id=‘" + id + ‘\‘‘ +
50                 ", sex=‘" + sex + ‘\‘‘ +
51                 ‘}‘;
52     }
53 }

六、运行结果

客户端

技术分享图片

 服务端

技术分享图片

 七、上面运行的结果发现打印的数据都完全一样,这是因为没有考虑粘包/半包的处理,还不能正常工作,下面我们利用Netty的LengthFieldPrepender和LengthFieldBasedFrameDecoder,来解决上述问题,这里只需要对客户端和服务端添加相关的处理类就可以了,改动代码如下:

客户端

 1 .handler(new ChannelInitializer<SocketChannel>() {
 2                         @Override
 3                         protected void initChannel(SocketChannel socketChannel) throws Exception {
 4                             socketChannel.pipeline()
 5                                     .addLast("frameDecoder",new LengthFieldBasedFrameDecoder(65535,
 6                                             0,4,0,4))
 7                                     .addLast("msgpack decoder",new MsgPackDecoder())
 8                                     .addLast("frameEncoder",new LengthFieldPrepender(4))
 9                                     .addLast("msgpack encoder",new MsgPackEncoder())
10                                     .addLast(new EchoClientHandler());
11 //
12                         }
13                     });

服务端

 1  .childHandler(new ChannelInitializer<SocketChannel>() {
 2                         @Override
 3                         protected void initChannel(SocketChannel socketChannel) throws Exception {
 4                             socketChannel.pipeline()
 5                                     .addLast("framDecoder",new LengthFieldBasedFrameDecoder(65535,
 6                                             0,4,0,4))
 7                                     .addLast("decoder",new MsgPackDecoder())
 8                                     .addLast("frameEncoder",new LengthFieldPrepender(4))
 9                                     .addLast("encoder",new MsgPackEncoder())
10                                     .addLast(new EchoServerHandler());
11                         }
12                     });

其实两者改动的地方完全一样,下面看一下运行效果

客户端

技术分享图片

服务端

技术分享图片

 

这次运行结果显示的是0~5的数据,但是遗憾的是不知道为什么有重复了一下,而且之重复了部分的运行,暂时不理会。以后应用多了可能就明白了。或者有路过的朋友知道的留个言,谢谢!

  

netty权威指南学习笔记六——编解码技术之MessagePack

原文:https://www.cnblogs.com/xiaoyao-001/p/9348010.html

(0)
(0)
   
举报
评论 一句话评论(0
关于我们 - 联系我们 - 留言反馈 - 联系我们:wmxa8@hotmail.com
© 2014 bubuko.com 版权所有
打开技术之扣,分享程序人生!