@Timothy Leavitt to the rescue! 

Thank you, this has been the most successful approach by far.  I got this to work by defining a <script language='cache' runat='server'> block where within it I wrote a <script language='javascript'> block and then set my global variable there which I later user in my $(function() { . . . method.  

The only challenge is that since I was using write commands to write out a script within a script, it wouldn't let me simply w "</script>" at the end, so I had to kind of hack it and w "</"_script>"

That said, I obviously designed this poorly and the payload coming back didn't scale up and it's making the webpage slow overall.  This will get us up and working again, but I need to redesign this so it's more efficient.

For what it's worth, I thought for this particular tool I was designing that I would just return a whole payload to the client that represented the configuration for this particular setting not knowing that it scales up very quickly.  The idea was that instead of making calls to the server over and over, we would let them modify the whole configuration on the client and once it's done, send it back over. 

I guess fodder for another post!

Hi @Stephen Canzano and thank you for the reply!

I essentially tried a similar approach doing this

set stream = ##class(%Stream.TmpCharacter).%New()

do myDynamicObject.%ToJSON(stream)

Where now the whole dynamic object is set to stream.  The problem comes when I send that stream back to the client.  I need the data in that stream for a JavaScript function so I read from the stream into a variable on the server.  However because it's so big, reading the stream into a variable to use in the JavaScript is causing the MAXSTRING error as well.

I tried wrapping it in a REST API but under the hood in the %Rest.Impl class the %WriteResponse is just calling a %ToJSON on the dynamic object which is causing the error.  It doesn't seem to be handling streams or if it does, it still wants to get my dynamic object into a JSON string but it's just too big.

The data does change over time, so I don't think the JSON adaptor is the right approach.

Thanks for any additional advice!

@Ali Nasser digging into this more this morning I am trying to determine the best use cases be between SQLProcs and Calculated data.

There are some obvious differences like do you want something to be calculated when saved or updated.

But in general since a lot of my data exists, it's going to be calculating on the fly (I suppose I could write a script to save/update all the existing records for force a calculated field). I start to wonder though, are calculated fields and SQLProcs savinging me query performance or am I just offloading my post processing into the query itself?  Does that make sense?

The reality is that our swizzled references will knock us down a few pegs no matter how I configure it but I do hope wrapping it all in the query can be more efficient.  I have a feeling we'll see some gains from the filtering and sorting end since I'll be able to simply select this, that, this and that and then the filters and sorts are working with that table data already calculated.

@Ali Nasser thank you so much for this thoughtful response!

- I did see the new LIMIT and OFFSET SQL functionality, but we aren't yet on 2025 but we are soon to be there.  

- Thanks for the reference to swizzling - this is indeed I think what tends to slow things down.

- I did not know you could compute properties like that! I will have to experiment with this. I can see it working for many properties.  It does become a bit cumbersome in the class definition as they get larger and larger. I think we could use a mix of the triggers and the query runtime calcs, because some things static but others might changed based on other data.  

- Ok I admit I thought twice when I said you can't call methods from the SQL queries and I do recall seeing SQLProc keyword documentation and I think I need to dig into this more along with the calculated data.

- My ancedetoal though on base query plus ObjectScript was that the base query would run REALLY quick to just return a base set of ObjectIds that is the key for all the ObjectScript processing.  I just did try this approach and you're right, it does take a lot more processing. 

- Since we are replicating the DataTables client side behavior, yes, we are trying to search the whole row for "Sally" in that example. Right now I'm hacking this with a concatenated string in my ObjectScript processing and a search text, but it sounds like full-text search is a specialty and more delicate kind of processing that I shouldn't try to hack like I am.

We use a LOT of embedded SQL in our application and I have had to move to dynamic to make this work, however since dynamic queries aren't cached, I guess I thought they were less performant. But compared to a base embedded query with additional ObjectScript processing, I think you have given me some ideas to go back and make this work with Dynamic SQL without any additional COS processing.

Thank you for your help here! How wonderful!

The closest I've gotten here is to use OnPreDispatch() to hack the flow and call the ImplementationClass.Page() method but this opens a whole can of worms.

@Eduard Lebedyuk thank you!

I'm not sure that will work: even if I extend my implementation class with my dispatch class, the implementation class doesn't seem to fire off the Page() method which would call AccessCheck().  So Adding the ##SUPER() call into my subclassed AccessCheck() won't work because the subclass Page() and thus AccessCheck() methods are never called.

I think I'm trying to do something this wasn't designed to do? My dispatch class extends %CSP.REST which implements AccessCheck() and then forwards to an implementation class which also extends %CSP.REST. I don't see why the forward wouldn't call the %CSP.REST.Page() method but it's hard to track in the %CSP package.  

Am I understanding your suggestion on how to user ##SUPER() here?

Is there a way to call common code in a %RegisteredObject?  Like an OnBefore method (I don't see anything like that but maybe it happens a different way).  I think my issue is that Page() dispatches the request by calling the implementation method directly so any %CSP.REST or %CSP.Page functions aren't triggered.  I could add a middle layer to the dispatch and the implementation but then I still have the same problem: I want lower level access checks in the implementation method to trigger every time without having to call the access check in each method.  

It appears once a request is dispatched, that's really it.  The rest of the URL seems to parsed out with the arguments and the correct endpoint with args is called.

It seems the convention is that AccessCheck is only called ONCE from the dispatch class.

I can't seem to find clever way to get around this. It would be more helpful to override AccessCheck at the forwarded routes because then there's less parsing of URL to determine where I'm at and what's going on and what permissions I need to check and so on an so forth.

@Robert Cemper 
Thanks so much!  Glad I'm not alone in the struggle! 

So obviously there's good reason behind LineTerminator being a property than can be changed and adapted and we have to continue to think of how to handle them as we consume files.  You start to understand why so many other data formats beyond text files!

Further inspection reveals ..ATIndexOpen is an 'Index Open' generated method.  So it's trying to open some object with the ATIndex of that encrypted token.  Not sure how the that all connects . . .

Thanks, @Ron Sweeney 

Yes, I did put the token in the debugger and confirm only once scope is in the token and made sure that I copied and pasted that in the ValidateJWT() method so I didn't fat finger.  So the scope is indeed there.  My workaround is to not try to validate the scope within the library methods and just pull out the scope from the body and validate in a for loop.