How to prevent reentrancy inside same process ?

One way would be to add a check for the existence of a process-private global (PPG) node. Something like this:

lock +^TEMP("FOO"):0 //don't wait
quit:'$test
quit:$get(^||CRITSECT("FOO"))
set ^||CRITSECT("FOO")=1
//critical section
//...
kill ^||CRITSECT("FOO")
lock -^TEMP("FOO")

We can use the %SYS.LockQuery class and its List query function to check whether the global is already locked. If it is, we can skip attempting to acquire the lock.

Check for the specific process

ClassMethod LOCK(Lock, Mode)
{
    If '..IsLocked("^A","X") {
        Lock +^A
    }
    Else {
        Write "Locked"
    }
}
// X - Exclusive
// S - Shared
ClassMethod IsLocked(Global As %String, Mode As %String)
{
    #dim status = 0
    Set tResult = ##class(%SYS.LockQuery).ListFunc($J)
    While tResult.%Next() {
        If tResult.Mode=Mode&&(tResult.LockString=Global) Set status= 1
    }
    Return status
}

However, the above code only checks for a specific process and does not account for other processes with Xclusive or Shared locks. The sample below checks for all acquired locks, returning their status and lock type.

ClassMethod IsLocked(Global As %String, Mode As %String)
{
	#dim status = 0
	Set tResult = ##class(%SYS.LockQuery).ListFunc()
	While tResult.%Next() {
		If tResult.LockString=Global {
			If tResult.Mode?1(1"X",1"S") Set status= 1_","_tResult.Mode
		}
	}
	Return status #; status be like "1,S" or "1,X"
}

I don't entirely understand the problem, you ask for "inside same process" but further you write "protect some code for being called by multiple processes at same time" - same or multiple processes?

The only way (I know) to reenter a code inside the same process is, using recursion (there are neither threads nor events in ObjectScript). If you don't want reentrace, do not use recursion.

If you talk about multiple processes (running at the same time) then you have to use some kind of semaphore to let in just one process at any given time in a critical code section, see the examples below. I would use the version with device, because in all error cases (abnormal shut down, process error, etc) IRIS closes the device automagically - globals may have leftovers in case of errors.

Class DC.OnlyOne Extends %RegisteredObject
{

/// device numbers 20-46 and 200-223 are routine interlock devices
/// Choose one of them
/// 
ClassMethod UsingDevice(testTime = 0)
{
	set myDevice = 45
	open myDevice::0
	
	if $t {
	
		// run that critical section
		hang testTime
		set ans="Critical section is done"
	
	} else { set ans = "Critical section in use" }
	
	close myDevice
	quit ans
}

/// a more meaningful name then ^X would be reasonable
/// a KILL ^X in %ZSTART.mac, label SYSTEM is also advisable
/// for the case, the system was shut down abnormally
ClassMethod UsingGlobal(testTime = 0)
{
	if $i(^X)=1 {
		
		// run that critical section
		hang testTime
		set ans="Critical section is done"
		
	} else { set ans="Critical section in use" }
	
	if $i(^X,-1)
	quit ans
}

ClassMethod UsingLock(tim = 10)
{
	// see John's example
}

}