It appears to be enabled when I try here, but I haven't used recordmaps much so I don't have one fully configured. I see another setting for the encoding in the record map properties itself; perhaps this is where things should be configured? (That would also explain why the adapter setting is disabled: you want to configure this only once.)

One of the settings for this service (category Additional settings) is Charset. This allows you to specify the file charset. Did you try setting this to UTF-8?

Unfortunately, long strings can't be 3GB, but rather almost 3.5MB. For any sane usage of JSON this should be enough, but I've seen REST APIs where e.g. PDF streams are returned in base64-encoded JSON properties, that come uncomfortably close to that size.

It would be really nice if the %DynamicObject class added support for streams to deal with these large payloads!