java - Hadoop Custom Output format, when do all reducers end? -

java - Hadoop Custom Output format, when do all reducers end? -

i'm building custom output format hadoop , wondering if there way in output format know when reducers (recordwriters) finish ?

in order know 1 recordwriter completed, close method of recordwriter can used, executing cleanup when of recordwriters finish ?

you can utilize driver final clean instead of relying on outputformat. uncertainty if provides such feature(api). finalize method may lastly resort, not advisable @ all.

the waitforcompletion method of job returns after jobs finishes. :

boolean status = job.waitforcompletion(true); if(status){ // clean required successful jobs } else { // clean required failed jobs }

if clean irrelevant job's success/failure, remove if-else part. , if need method in outputformat class deletion, create static. eg :

job.waitforcompletion(true); customoutputformat.cleanup();

i hope should suffice need.

java hadoop mapreduce reduce outputformat