Multi-threading to improve performance
There is a large file that needs to be processed, with over 500 000 rows. Each row needs to be verified for various data and then all the results collated and reported into a single report. What is the most efficient way to do this? I tried calling the processing function with Job , where each function jobbed off would report results to a different node of a common global. But the jobbed functions are not updating their respective rows even though I am passing the global name and root node. What is the most efficient way to process this large file? Any Cache/Ensemble ideas are appreciated.