Sunday, 15 September 2013

java - Write an empty MapReduce job -



java - Write an empty MapReduce job -

i want write empty mapreduce job, mean mapreduce job doing nothing, has mapper, reducer , main class. want test in hortonwoks sandbox 2.1. code:

import java.io.ioexception; import java.util.*; import org.apache.hadoop.fs.path; import org.apache.hadoop.conf.*; import org.apache.hadoop.io.*; import org.apache.hadoop.mapred.*; import org.apache.hadoop.util.*; public class mainclassname { public static class map extends mapreducebase implements mapper<intwritable, text, intwritable, text> { private final static intwritable 1 = new intwritable(1); private text word = new text(); public void map(longwritable key, text value, outputcollector<text, intwritable> output, reporter reporter) throws ioexception { output.collect(word, one); } } public static class cut down extends mapreducebase implements reducer<text, intwritable, text, intwritable> { public void reduce(text key, iterator<intwritable> values, outputcollector<text, intwritable> output, reporter reporter) throws ioexception { int info = 0; } output.collect(key, new intwritable(data)); } } public static void main(string[] args) throws exception { jobconf conf = new jobconf(mainclassname.class); conf.setjobname("jobname"); conf.setoutputkeyclass(text.class); conf.setoutputvalueclass(intwritable.class); conf.setmapperclass(map.class); conf.setcombinerclass(reduce.class); conf.setreducerclass(reduce.class); conf.setinputformat(textinputformat.class); conf.setoutputformat(textoutputformat.class); fileinputformat.setinputpaths(conf, new path(args[0])); fileoutputformat.setoutputpath(conf, new path(args[1])); jobclient.runjob(conf); } }

is correct? gives me error:

description resource path location type type mainclassname.map must implement inherited abstract method mapper.map(text, text, outputcollector, reporter) mainclassname.java /mainempty/src line 14 java problem

and want know java files must imported run simple job. lot. :)

your type arguments bit muddled. mapper taking <longwritable,text> pair, , outputting <text,intwritable> pair. class declaration says:

implements mapper<intwritable, text, intwritable, text>

which should read

implements mapper<longwritable, text, text, longwritable>

the rest looks ok.

java hadoop mapreduce hortonworks-data-platform

No comments:

Post a Comment