首页 > 移动平台 > 详细

按字段分组的Mapper

时间:2016-03-06 10:02:50      阅读:322      评论:0      收藏:0      [点我收藏+]
<strong><span style="font-size:18px;">/***
 * @author YangXin
 * @info 按字段分组的Mapper
 */
package unitTwelve;

import java.io.IOException;
import java.util.regex.Pattern;

import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Mapper;

public class ByKeyMapper extends Mapper<LongWritable, Text, Text, Text>{
	private Pattern splitter = Pattern.compile("\t");
	private int selectedField = 1;
	private int groupByField = 0;
	
	protected void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException{
		String[] fields = splitter.split(value.toString());
		if(fields.length - 1 < selectedField || fields.length - 1 < groupByField){
			context.getCounter("Map", "LinesWithErrors").increment(1);
			return;
		}
		String oKey = fields[groupByField];
		String oValue = fields[selectedField];
		context.write(new Text(oKey), new Text(oValue));
	}
}
</span></strong>

按字段分组的Mapper

原文:http://blog.csdn.net/u012965373/article/details/50811604

(0)
(0)
   
举报
评论 一句话评论(0
关于我们 - 联系我们 - 留言反馈 - 联系我们:wmxa8@hotmail.com
© 2014 bubuko.com 版权所有
打开技术之扣,分享程序人生!