Resolved by removed a lock on some global called ^oddProcess("default_project") or something like that.

How embarrassing for me.

Pro tip: check the names of your keys when you access them :)

Thanks all!

For some more context, I stopped trying to see what was going on by setting globals to debug.  I set the %response.TraceDump=1 so I can see the %session variable which is were I'm storing the value to the key of the object:

%session.Data("key")=array.%Get("key")

This is still no working as expected.  I even stepped through the code in the command line by calling the zLabel in the INT routine generated from the CSP and it worked as expected there.  

@Eduard Lebedyuk Thank you.  This IS true since I'm debugging a method defined on CSP page, I'm writing things to a global to check them on the terminal.  So that explains that.

How do you know that means that?

Thanks @Pravin Barton! This is helpful just to get a better send of how others use the framework.  We are still baby stepping into all of this and don't want to rush into implementing it as it's harder to untangle and "redo" once we're really rolling.  

Related to keeping your unit tests separate: so you don't export your unit test class files to a different ^UnitTestRoot?  Rather the ^UnitTestRoot is where the unit tests live in the cloned repository? 

@Timothy Leavitt thanks for the link to that post. I'm sure I've seen it before but more will make sense now that I have more experience with this.  

Related to the test coverage tool: do you have best practices or shortcuts as an individual developer?  Do you run the Test Coverage tool as your developing to see what lines of code are covered or not? We are thinking through how to arm the developer to write the most complete unit tests before sending to the remote and having Jenkins automatically rebuild everything and do the more extensive reporting.

Really appreciate the responses! 

@Dmitry Maslennikov This seemed to help as the changes to the HTML are instant with the changed setting, however the CSS didn't change.  After clearing the browser cache then I saw the changes.  Assuming there must be a browser setting that hold on to the CSS.  I'm not sure.  

We haven't quite solved this for our own app.  All the progress thusfar has been in the demo app from GitHub (the coffee shop).

Using the demo, the web folders are in the container.  I will try this though and see if it changes things.

After 30 mins or so both the CSS change (changed from Ariel to Times) and the HTML (added "TESTING!!!" in the heading) ended up refreshing.

Why is there this lag?  Must be the browser?

A update already: it seems the CSS has now updated on both browsers, but the HTML is still not updated.  This lag seems quite off.  When I update CSS locally (not Docker) it changes on refresh.  

@Guilherme Mendes Thank you.  That's exactly it.  That documentation was misleading, and I think it's fair to say so since there are so many ways to embed this and that into that and this!  

Ah ok, so just onClick=return fromAJavaScriptFunction()?

Define that in  <script language="javascript">, right?

Returning 0 on the server-side method didn't seem to stop the form, my file was still generated from the process that runs after checking the %reqeust.Data("sumbitButton",1) node.  But I think I'll get rid of this anyway.

<script language="Cache" method="Validate" arguments="startDate:%Date,endDate:%Date">
   
   s startDate=$ZDH(startDate,3)
   s endDate=$ZDH(endDate,3)
   if startDate>endDate {
  &js<
  alert('Invalid date range.');>
  quit 0
   }
   quit 1
 </script>

I turned on that auditing btw and I'm not seeing anything. But doing some other debugging it seems that the %Net.SMTP.Send() method is getting hung up.  If I step through it, it's endless lines of code.  I put a break after the Send() method and it never reaches it.  Not sure if it's infinite looping or just getting hung.

@Timothy Leavitt thank you!

I wouldn't have thought to do form logic OnPreHttp . . . the documentation suggests checking the %request.Data node for the form in the runat="server" script.  This does make sense if you plan to do a redirect, I think.

Thank you for helping me better understand the runat="server" timing.  It's been a while since I've touched PHP but this all rings a bell.  

@Warlin Garcia thank you!  In fact, I'm doing something like this but I must be missing something.

Form has an attribute:

onSubmit="return #server(..Validate(date1,date2))#"

Validate() is in the CSP code as such:

<script language="Cache" method="Validate" arguments="startDate:%Date,endDate:%Date">
   s startDate=$ZDH(startDate,3)
   s endDate=$ZDH(endDate,3)
   if startDate>endDate {
  &js<
         alert('Invalid date range.');
          return false;>
   }
 </script>

I get my alert, but my page continues to reload, which would be fine I guess, but the %request.Data object with the data from the form is still populated. 

So when the following block runs, the process that I don't want to execute on the server side keeps going when I want it to not run.  I guess I'm not sure what returning false to the onSubmit actually does?  What is does not seem to do is clear or not set the %request.Data form nodes.  

 <script language="Cache" runat="server">
 
  If ($Data(%request.Data("FormThatWasSubmitted",1))) {
         s startdate=%request.Data("STARTDATE",1)
         s startdate=$ZDH(startdate,3)
  
        s enddate=%request.Data("ENDDATE",1)
        s enddate=$ZDH(enddate,3)

        job process^routine(startdate, enddate)

</script>

Side note: I'm jobbing off that routine which sends and email . . . if I do the job in the foreground with a DO the email sense, if I use JOB the email does not send (file still generates).  Anyone know why?

I guess more generally, I'm not sure when using this tag is supposed to be used:

<script language="Cache" runat="server">

It's clearer when to use:

<script language="cache" method="OnPreHTTP">>

<script language="Cache" method="SomeMethod">

If I move my code from the tag with the runat="Server" argument, then I can't seem to access the %request.Data object.  If I keep my code in that block, I can't use any &js<> syntax.

Can someone explain in a kindergarten way to me where this lives in the HTTP lifecycle?