Monday 21 April 2014

Code For Deleting Output Folder If Exist In Hadoop MapReduce Jobs




Mostly Hadoop MapReduce Jobs operates with two arguments.
Input directory and Output directory.

Each time when we run our MapReduce job we need to give  non-existing folder as our output path. So while we are doing a Trail and Error method in our MR jobs. It is good if it automatically deletes  the output folder if exists.

Here is the code for that:
/*Provides access to configuration parameters*/
Configuration conf = new Configuration();
/*Creating Filesystem object with the configuration*/
FileSystem fs = FileSystem.get(conf);
/*Check if output path (args[1])exist or not*/
if(fs.exists(new Path(args[1]))){
   /*If exist delete the output path*/
   fs.delete(new Path(args[1]),true);
}

11 comments:

  1. Basically i know nothing about Hadoop.But feels great seeing an attempt to share things that u learn. Really nice. Will make use of this information someday for sure. :-)

    ReplyDelete
  2. Fortunately, Apache Hadoop is a tailor-made solution that delivers on both counts, by turning big data insights into actionable business enhancements for long-term success. To know more, visit Hadoop Training Bangalore

    ReplyDelete
  3. Actually I'm using scala . I have deleted the existing output folder and tried it. It is not throwing any exception . But my output file is not created in run time. I'm not able to access my output values what I was stored. What I want to do for this ?

    ReplyDelete
  4. I am really admired for the great info is visible in this blog that to lot of benefits for visiting the nice info in this website.
    Thanks a lot for using the nice info is visible in this blog.latestITjobs | Hadoop Training in Chennai |

    ReplyDelete
  5. This comment has been removed by the author.

    ReplyDelete
  6. https://www.digitalbrolly.com/digital-marketing-course-in-hyderabad/
    DIGITAL MARKETING COURSE IN HYDERABAD OUTCOMES
    Our Digital Marketing course in Hyderabad is a huge benefit to students, job seekers, freelancers and business owners. If you are a student or a job seeker this course will help you get a good job in Digital Marketing. If you are a freelancer or a business owner, this course would give you numerous tricks to generate more leads for your business. Honestly speaking, Digital Marketing is easy to learn. Executing and getting results from Digital Marketing isn’t difficult too. I would say it is a simple 3 step process.
    Step 01

    ReplyDelete
  7. nice article
    thanks for sheering us

    Azure admin training in hyderabad

    ReplyDelete