creating a test server

Caché, Change management, Continuous Integration, Containerization, Docker, Testing

hi, new here, and new to cache and deepsee.

i've been trying to setup a copy of our production server so we can use it for testing/development.

i did a full backup. moved it to the new server. ran the DBREST command. got it to restore but seems like permissions get all messed up. and it just generates a bunch of errors.

is there an easier/better way of doing this?

Answers

Answer by Robert Cemper 8 March 2018

backups typically backup a DB in Cachè but never a full installation. (e.g ref to registry and similar)
Do a fresh installation and next move / restore your DB

+ 1

Accepted answer

Comment by Chris Fine 8 March 2018

i did do a fresh install first. so i had a working test system, just once i did the restore it went to crap.

Comment by Robert Cemper 8 March 2018

IF you did a Restore of "%SYS" DB (.../mgr/CACHE.DAT)
then you may have destroyed essential instance specific installation information

Go back and do a fresh install. And don't touch %SYS during restore.

Comment by Chris Fine 9 March 2018

so i told it not to restore the mgr folder

but did restore the other two, now the name spaces are not showing?

is there a way to see where the namespaces are saved?

Comment by Robert Cemper 9 March 2018

there is an external config file in %cacheroot%/cache.cpf
that holds your configuration. It is loaded at startup.
You can verify it with any text editor and check especially directories. of your DBs
And you see also NAMESPACE definitions.

The important part is at the beginning.

[ConfigFile] Version=2017.226

[Databases] CACHESYS=C:\InterSystems\17E20\mgr\ CACHELIB=C:\InterSystems\17E20\mgr\cachelib\ CACHETEMP=C:\InterSystems\17E20\mgr\cachetemp\ CACHE=C:\InterSystems\17E20\mgr\cache\ CACHEAUDIT=C:\InterSystems\17E20\mgr\cacheaudit\ BLOCKS=C:\opt\blocks\data\ DOCBOOK=C:\InterSystems\17E20\mgr\docbook\ ENSDEMO=C:\InterSystems\17E20\mgr\ensdemo\ ENSEMBLE=C:\InterSystems\17E20\mgr\ensemble\ ENSEMBLEENSTEMP=C:\InterSystems\17E20\mgr\ensemble\ensembleenstemp\ ENSEMBLESECONDARY=C:\InterSystems\17E20\mgr\ensemble\ensemblesecondary\ ENSLIB=C:\InterSystems\17E20\mgr\enslib\ SAMPLES=C:\InterSystems\17E20\mgr\samples\ TEST=C:\opt\blocks\test\ USER=C:\InterSystems\17E20\mgr\user\

[Namespaces] %SYS=CACHESYS BLOCKS=BLOCKS DOCBOOK=DOCBOOK ENSDEMO=ENSDEMO ENSEMBLE=ENSEMBLE SAMPLES=SAMPLES USER=USER

Answer by Benjamin Spead 8 March 2018

For my team we always deploy on VMs, which makes it trivial to clone the VM to create our Dev and Test instances (this helps ensure complete equivalent for all Caché settings, OS patch level, OS settings, etc). When we moved to this approach it really helped to improve our testing stability and predictability (we have our Code and Data separated into different DB, so the Code DB is updated from source control and our Data DB can be updated weekly with restoring the backup of the Prod Data DB.

If you are currently deployed on bare iron this won't be an option for you. But if you are virtualized for production (or can move to it in the future) you may find that it saves a lot of hassle for your Dev and Test instances.

Comment by Chris Fine 8 March 2018

so going that way all i'd need to do is rename the cloned vm. we are virtual.

Comment by Robert Cemper 8 March 2018

That's easiest.
All you have to check is new IP-address

Comment by Benjamin Spead 8 March 2018

You would also need to be careful of the following:

integration endpoints (e.g. web service calls (especially those inserting data), POP3 mailbox retrieval, FTP fetches, etc)
email generation code in your app
task schedule
NFS mount points

You must be very careful that your non-production instances do not behave as production post-cloning (e.g. you would never want a test system generating emails to non-developers). We have all of our code-base instrumented so that with a single configuration global we can control production vs non-production behavior (including where to go for web service calls, etc). This allows us to have identical code across all environments, but only Prod will send email, fetch actual user files, insert real data into other systems, etc. We then have our application-specific configuration globals mapped to their own Config DB, which means:

when the Data DB is restored from LIVE it doesn't bring configuration data with it (i.e. we need to avoid a Test system suddenly thinking it is Prod because the global indicating Dev|Test|Prod is restored from Prod)
when we run a test build for CI on our Build servers (also cloned from Prod) we can start with an empty Code DB (and pre-populated Data and Config DBs) and everything "just works" :)

Glad to hear that you are already virtualized - that can really save you a lot of setup and config time. Let me know if you have any additional questions :)

(P.S. Chris, welcome to the world of Caché and DeepSee!! I hope you find the Community to be helpful.

note - if you are happy with this answer, please mark the Answer as Accepted so this thread falls off of the Unanswered Questions list)

Answer by Tom Fitzgibbon 8 March 2018

I agree with Benjamin: VMs (and containers) are the way to make testing environments. Once you set up your VM/container structure it's easy to create/delete identical testing instances. Obviously make sure that you carefully isolate your cloned Cache from the real world or it may start interacting with real world production.

Tom Fitzgibbon | tom@tec.today | 3474648531

Answer by Robert Cemper 8 March 2018

IF it was really only permissions you could run ^SECURITY from terminal prompt

USER>zn "%SYS" %SYS>d ^SECURITY 1) User setup 2) Role setup 3) Service setup 4) Resource setup 5) Application setup 6) Auditing setup 7) Domain setup 8) SSL configuration setup 9) Mobile phone service provider setup 10) OpenAM Identity Services setup 11) Encryption key setup 12) System parameter setup 13) X509 User setup 14) Exit Option? 1 1) Create user 2) Edit user 3) List users 4) Detailed list users 5) Delete user 6) Export users 7) Import users 8) Exit or Option? 2 1) Create role 2) Edit role 3) List roles 4) Detailed list roles 5) Delete role 6) Export roles 7) Import roles 8) Exit or Option? 3 1) Edit service 2) List services 3) Detailed list services 4) Export services 5) Import services 6) Exit Option? 4 Export which services? * => * Export to file name ServicesExport.xml => and so on...

and Import on your test server .....

But take care how you handle external connections!

+ 1

Answer by Evgeny Shvarov 8 March 2018

Containers were invented for it.

The idea is that you have a git repo with docker container in it and a new developer can just checkout the repo and call "docker build" or even "docker up" to get the full prepared environment with one server in a container, two servers for client-server apps, and more for complex solutions.

See @Luca Ravazzolo posts on (one, two) on containers with InterSystems. Also, see @Eduard Lebedyuk series on continuous delivery with Gitlab , it is a very related topic.

Comment by Chris Fine 9 March 2018

i'll have to mess around with that as well.