Perfect! The source code for that class method is exactly what I need, thank you.

Just for others who have the same problem, you can do the following:

do $System.Encryption.MD5HashInit()
do $System.Encryption.MD5HashUpdate(stream.Read(...)) // Read in each chunk stream
set hash = $System.Encryption.MD5HashFinal()
set checksum = $ZCONVERT(##class(%xsd.hexBinary).LogicalToXSD(hash), "L")