STORE error means, your reached limit of memory per process. Since 2012.2 we have 256Mb per process by default and you can increase it up to 2Tb.  You can increase it, but I would recommend playing with different ways how to collect SQL Data, or on optimization in this SQL query. Or you can split such many rows by portions with less number of rows.

batch file means, that you want to do it outside of Ensemble. In this case you should use ccontrol command or iris if you use IRIS.

ccontrol stop <ensemble> quietly

ccontrol start <ensemble>

where you should replace <ensemble> with your instance name.

if you work on windows, you can find ccontrol tool in bin directory of installed Ensemble.

you can find more details about using ccontrol in the documentation

Better to use command JOB INT^SHUTDOWN instead of DO. When you use DO, the task will not be finished and will be marked as suspended, and you will have to resume it manually.

The example in the documentation to %XML.Reader, does not work for you?

    #include %occStatus
    // Create a new XML Reader class
    Set reader = ##class(%XML.Reader).%New()
    // Begin processing of the XML input
    Set sc=reader.OpenFile(filename)
    If $$$ISERR(sc) Do $system.OBJ.DisplayError(sc) Quit  
    // Associate a class name with the XML element name
    Do reader.Correlate("Person","Sample.Person")
    // read Sample.Person objects from xml file
    Set Count=0
    While reader.Next(.person,.sc) {
        Write person.Name_" imported.",!
        Set Count=Count+1
        Set sc=person.%Save()
        If $$$ISERR(sc) Do $system.OBJ.DisplayError(sc) Quit  
    }
    If $$$ISERR(sc) Do $system.OBJ.DisplayError(sc) Quit  
    Write Count_" Sample.Person instances found."

ROLLBACK should revert any changes in data which was done in a transaction, with some exceptions like $increment on Global.

You can look at this example.

Class User.Test Extends %Persistent
{

Property Name As %String;

Property CalcName As %String [ Calculated, SqlComputeCode = { set {*} = {Name} }, SqlComputed, SqlComputeOnChange = Name ];

Index ByName On CalcName;

Storage Default
{
<Data name="TestDefaultData">
<Value name="1">
<Value>%%CLASSNAME</Value>
</Value>
<Value name="2">
<Value>Name</Value>
</Value>
</Data>
<DataLocation>^User.TestD</DataLocation>
<DefaultData>TestDefaultData</DefaultData>
<IdLocation>^User.TestD</IdLocation>
<IndexLocation>^User.TestI</IndexLocation>
<StreamLocation>^User.TestS</StreamLocation>
<Type>%Storage.Persistent</Type>
}

}

Let's create first object

USER>s o=##class(Test).%New()

USER>set o.Name="testname"

USER>w o.%Save()
1

check saved in globals

USER>zw ^User.TestD,^User.TestI
^User.TestD=1
^User.TestD(1)=$lb("","testname")
^User.TestI("ByName"," TESTNAME",1)=""

and now open transaction, and do some changes.

USER>k

USER>TSTART

TL1:USER>set o=##class(Test).%OpenId(1)

TL1:USER>s o.Name="testname2"

TL1:USER>w o.%Save()
1
TL1:USER>zw ^User.TestD,^User.TestI
^User.TestD=1
^User.TestD(1)=$lb("","testname2")
^User.TestI("ByName"," TESTNAME2",1)=""

So, changes there, let's do rollback, and check data again.

TL1:USER>TROLLBACK

USER>zw ^User.TestD,^User.TestI
^User.TestD=1
^User.TestD(1)=$lb("","testname")
^User.TestI("ByName"," TESTNAME",1)=""

I think it will not be possible to restore backups from Caché/Ensemble to IRIS.

I'm sure that in most cases, it is possible to just rename CACHE.DAT to IRIS.DAT, and configure it exactly as it was in Ensemble, should work. And maybe it will work for you as well.

If you can easily repeat configuration on any just installed Ensemble server, do it in the same way for IRIS.

I think we don't have any other possibilities how to catch changes in source code. Do you know how to catch if any object in any class was changed, outside of this class, if you don't have any triggers there? No easy task. But I think it is possible to find some tricky ways if you don't care about just in time notifications. You can monitor any changes in the data, and filter by changing particular globals and subscripts. 

If you will describe what are you going to achieve exactly, it will be easier to help you.