java8-模拟hadoop

hadoop的入门程序,java8也能实现

txt统计单词数量程序

@Test public void fileWordCount() throws IOException {     //特殊文件需要格式转换为txt     Files.readAllLines(Paths.get("D:\jd.txt"), StandardCharsets.UTF_8).parallelStream()             //将多个流融合为一个             .flatMap(line -> Arrays.stream(line.trim().split("\s")))             .filter(word -> word.length() > 0)             .map(word -> new AbstractMap.SimpleEntry<>(word, 1))             .collect(groupingBy(AbstractMap.SimpleEntry :: getKey, counting()))             .entrySet().forEach(System.out :: println); } 

List统计单词数量程序

@Test public void listWordCount(){     List<String> stringList = Arrays.asList("a","b","c","a");     stringList.stream()             .map(s -> new AbstractMap.SimpleEntry<>(s, 1))             .collect(groupingBy(AbstractMap.SimpleEntry :: getKey, counting()))             .entrySet().stream()             .forEach(System.out :: println);     System.out.println("---------------------------------------------------");     //通过自定义reduce统计,其实counting()也使用的是reduce     //记住:凡是在中间操作使用了map,接口定义都需要声明出来,直接使用lambda表达式会有1.无法读取method,2.类型检查不到 的问题     BinaryOperator<Integer> binaryOperator2 = Integer::sum;     //排序的转换规则接口     ToIntFunction<Map.Entry> sortMapFunction = (Map.Entry se) -> Integer.valueOf(se.getValue().toString()).intValue();     stringList             .stream()             .map(s -> new AbstractMap.SimpleEntry<>(s, 1))             .collect(groupingBy(AbstractMap.SimpleEntry::getKey,                     reducing(0, AbstractMap.SimpleEntry::getValue,binaryOperator2)))             .entrySet()             .stream()             .sorted(Comparator.comparingInt(sortMapFunction))             .forEach(System.out::println); }

脚本宝典为你提供优质服务
脚本宝典 » java8-模拟hadoop

发表评论

提供最优质的资源集合

立即查看 了解详情