There may be scenerios where we need to make our custom pig scripts, which can take any arguments.
Below is an sample code for a Custom Pig Script.
Sample Pig Script
The "customparam.pig" loads an input with custom argument and generates a single field from the input bag to another bag and stores the new bag to HDFS.
Here the input,delimiter for input file,output and filed to seperate are given as custom arguments to Pig Scripts.
--customparam.pig
--load hdfs/local fs data
original = load '$input' using PigStorage('$delimiter');
--filter a specific field value into another bag
filtered = foreach original generate $split;
--storing data into hdfs/local fs
store filtered into '$output';
Pig Scripts can be run as Local or in MapReduce Mode.
Local Mode
pig -x local -f customparam.pig -param input=Pig.csv -param output=OUT/pig -param delimiter="," -param split='$1'
This is the sample "Pig.csv" file which is the custom input used in command line.The custom delimiter is ",".
Pig1,23.5,Matched
Pig2,6.88,Not Matched
Pig3,6.1,Not Matched
And seperating 2 nd column from the original bag to a new bag.Any field in Pig starts with $0,$1,$2,....So if we need to generate 2 nd column the split param should be "$1".
After executing the above command. The part file content will be
23.5
6.88
6.1
