java - Write an empty MapReduce job -
i want write empty mapreduce job, mean mapreduce job doing nothing, has mapper, reducer , main class. want test in hortonwoks sandbox 2.1. code:
import java.io.ioexception; import java.util.*; import org.apache.hadoop.fs.path; import org.apache.hadoop.conf.*; import org.apache.hadoop.io.*; import org.apache.hadoop.mapred.*; import org.apache.hadoop.util.*; public class mainclassname { public static class map extends mapreducebase implements mapper<intwritable, text, intwritable, text> { private final static intwritable 1 = new intwritable(1); private text word = new text(); public void map(longwritable key, text value, outputcollector<text, intwritable> output, reporter reporter) throws ioexception { output.collect(word, one); } } public static class cut down extends mapreducebase implements reducer<text, intwritable, text, intwritable> { public void reduce(text key, iterator<intwritable> values, outputcollector<text, intwritable> output, reporter reporter) throws ioexception { int info = 0; } output.collect(key, new intwritable(data)); } } public static void main(string[] args) throws exception { jobconf conf = new jobconf(mainclassname.class); conf.setjobname("jobname"); conf.setoutputkeyclass(text.class); conf.setoutputvalueclass(intwritable.class); conf.setmapperclass(map.class); conf.setcombinerclass(reduce.class); conf.setreducerclass(reduce.class); conf.setinputformat(textinputformat.class); conf.setoutputformat(textoutputformat.class); fileinputformat.setinputpaths(conf, new path(args[0])); fileoutputformat.setoutputpath(conf, new path(args[1])); jobclient.runjob(conf); } }
is correct? gives me error:
description resource path location type type mainclassname.map must implement inherited abstract method mapper.map(text, text, outputcollector, reporter) mainclassname.java /mainempty/src line 14 java problem
and want know java files must imported run simple job. lot. :)
your type arguments bit muddled. mapper taking <longwritable,text>
pair, , outputting <text,intwritable>
pair. class declaration says:
implements mapper<intwritable, text, intwritable, text>
which should read
implements mapper<longwritable, text, text, longwritable>
the rest looks ok.
java hadoop mapreduce hortonworks-data-platform
No comments:
Post a Comment