My usecase is sorting and removing duplicates and getting count from a file that has json messages as a individual rows.
I am currently planning to use pandas for this purpose as its really fast. Below are the steps i am following
1) call a python function (called function) from IRIS classmethod(calling function)
2) the call python function will read the json file in a dataframe
3) perform sorting, dup removal, count in the dataframe
4) convert the dataframe into iris stream
5) return back the stream to iris calling function class method
When i try to write the stream into termial its coming as a %SYS.python object rather a iris stream object.
Below is what my questions are
1) why is the return a %Sys.python rather a iris stream object
2) is there a better way to implement sorting, dup removals n count of record, in a file within iris.
Thanks!