Large HL7 Batch

Hi

I need to process a very large Batch file which contains HL7 charges which we pull using FTP. I need to loop through the file and do some checks and then split the large Batch file in to individual messages. These individual messages then gets process through a different module to create a output batch of different format

My concerns are

1. I do not want to load the whole HL7 batch file because it will kill the server trying parsing that big file

2. I tried looking at the Complex Record mapping,  but I am not sure if its the right approach .

3. Looking at  Implementing the OnProcessInput() Method

I know its a common scenario in healthcare, has any one implemented this kind of scenario?

Thanks

VJ