Internationalization Questions

I'm looking at adding multilingual support to a couple of open source projects I'm working on. The solutions are already developed in CSP so I am not looking for alternative approaches.

I'm wondering what would be the best approach for CSP and separate JavaScript files.

Initially I was wondering if I should bake the default system language text at compile time, or provide the end user with a language selection option at run time.

I came across $$$TEXT reading the docs...

https://docs.intersystems.com/latest/csp/docbook/DocBook.UI.Page.cls?KEY=GCSP_localization

and have also discovered the localization manager which looks to be really handy...

https://community.intersystems.com/post/cach%C3%A9-localization-manager-or-i18n-intersystems-cach%C3%A9

In the localisation manager example it gives the user a drop down to select a preferred language. It's not obvious to me at the moment what setting that drop down would change on the server to then affect the $$$TEXT macros at run time. What would I need to do, to do something similar.

I'm also unsure what $$$TEXT is doing at compile time. If I wanted to bake the language of the local system language into the code then how do I use $$$TEXT to do this. Again its not obvious at the moment how $$$TEXT knows the difference between a compile time implementation and a run time.

I'm also wondering if there is any advice on the best way to deploy the language pack (global) with an application on GitHub. If the application was developed such that it bakes the language at compile time, then how would you bundle the language into the build file. Would you just include all of the languages or have the user install the language of choice separately.

In general any additional advice or pointers would be welcome.

Thanks,
Sean.

  • + 1
  • 202
  • 15
  • 4

Answers

This is what I have found out so far, it wasn't clear in the documentation.

Snippets of strings (labels, phrases, sentences etc) are held in the global ^CacheMsg keyed on domain, language and some kind of hash of the string, e.g.

^CacheMsg("Foo","en",1243066710)="Hello World"
^CacheMsg("Foo","ru",1243066710)="Привет мир"


If I use $$$Text this way...

write $$$Text("Hello World","Foo","ru")


Then it compiles down to...

write $get(^CacheMsg("Foo","ru","1243066710"),"Hello World")


I assume therefore that I can dynamically control the language by doing something along the lines of...

write $$$Text("Hello World","Foo",%Session.Language)


where I set the language in the session object after the users selection.

Alternatively I can use $$$Text with just the first two parameters (no language specified) and it will compile to...

write $get(^CacheMsg("Foo",$s($mvv(58)="":"en",1:$mvv(58)),"1243066710"),"Hello World")


Where the language will either be set by the value of $mvv(58) or if null defaults to English.

I assume $mvv(58) is set by the system when its installed? But can't find any information on that at the moment.

The documentation talks about using an XML file to manage the languages, which seems a bit kludgy. I guess mainly to give to an interpretor to edit. I assume then that the Caché Localisation Manager just works off the ^CacheMsg global and not the XML files, and would be a better way to manage the localisations.

In terms of deployment I guess the best solution would be to just include all of the languages as a global export and have it imported with the main body of the code. 
 

Interesting to see that $$$TEXT references $mvv(58). IIRC the $mvv function is related to Caché MultiValue.

BTW, your reference to %Session.Language probably needs to be %session.Language instead.

For CSP/Zen sessions shouldn´t $mvv(58) be set as the Accept-Language sent by the User-Agent? It is not. Why?

 

Rodrigo is right:

http://docs.intersystems.com/latest/csp/documatic/%25CSP.Documatic.cls?PAGE=CLASS&LIBRARY=%25SYS&CLASSNAME=%25CSP.Session

says:

 

• property Language as %String;

The language (RFC 1766 format) in which pages in this CSP session are to be displayed.
The %session. Language attribute is used as the default language for csp:text, span and div tags if %response.Language is not set. If %session.Language is not set by the program, then it will default to the best fit with the HTTP_ ACCEPT_LANGUAGE CGI variable.
 

 

Hi, Robert!

User-agent is sending:

Accept-Language: pt-BR,pt;q=0.8,en-US;q=0.5,en;q=0.3

But %session.Language is being seen with the value "en", when not explicitly set by the application.

In ^%qCacheMessage global we´ve got entries for pt-br.

I want, for example, that the "No Results" text for the ZEN tablePane tag when there is no row returned is exhibited in portuguese as provisioned by the factory entry below:

^%qCacheMsg("%ZEN","pt-br",3559354800)="Nenhum resultado"

How to?

my personal experience in management portal:

changed my primary browser language from  "de"  to "en"  and the portal followed at next browser start.

So this might relate to ZEN somehow:
Do you use the $$$TEXT equivalent of ZEN ? $$$TextJS, $$$FormatText

for details and how-to see http://docs.intersystems.com/latest/csp/docbook/DocBook.UI.Page.cls?KEY=GZAP_localization

Does your Caché NLS seting support pt-br  ?  (mine includes German DEUW)

I´m not using $$$Text by myself. The "No Results" string is shown by ZEN tablePane itself, which is through $$$Text:

%ZEN.Component.tablePane.cls

If ((tRow = 0)&&(..initialExecute)) {
#; draw indicator of no results (unless deferred executed)
Set msgNoResults = $$$Text("No Results","%ZEN")
 

How do I include a language in Caché NLS definition?

I'm not sure this really your problem. It could be. 

The first TEST should be your Mgmt Portal.

Language definitions are predefined and loaded at installation time.
to change it later from Mgmt Portal: 
System > Configuration > Locale Definitions 

If you run on UNICODE it should be enough to change it to PTBW - Portuguese, Brasil, Unicode (comboBox)

ending 8 = 8 bit encoded
ensing W = WIDE = Unicode

If your system is any 8bit installation. Someone else may have an answer. It may cause some troubles on a running installation. I never touched this.

 

The Mgmt Portal works with a explicit definition by the user of which is his/her preferred language.

There it is working correctly when I choose Portuguese (Brazil).

The problem seems to be with the treatment of the Accept-Language HTTP header by CSP, which IMHO should be seting $mvv(58) accordingly, and it is not.

 

I haven't seen $mvv(58) before in any CSP application.
So this might be an action for WRC to find out why / where it is lost

Sorry,

(thanks John)

I'm now looking at how to implement this with JavaScript files.

In my JavaScript I will have lines of code like this...

popToast("danger","Unable to open document");


Since $$$Text can't be used in this context I wonder what else other developers are doing. I could fetch the JavaScript via a CSP page and replace the strings, but that feels kludgy.

My initial thoughts would be to create a $$$Text JavaScript function for continuity and use it in a similar way...

popToast("danger",$$$Text("MyApp","Unable to open document"));


The localisation file would be loaded on application start (or when the language is changed), something along the lines of...

/csp/myapp/localisation.cls?domain=MyApp&lang=default


Spin the global out into JSON...

"MyApp" : {
  "en" : {
    "1243066710" : "Hello World"
  }
}


$$$Text would be something like...

var $$$Text = function(domain,phrase) {
  var hash = someHashFunction(phrase)  // ??
  return locals[domain][defaultLanguage][hash]
}


The next question is, what hashing algorithm is Cache using to create the hash?

I will need to replicate that in JavaScript, or come up with some other hash approach.

I've also noticed that the samples does not use this hash, instead it uses a man made key, but using that key with $$$Text in COS just creates a hash which seems odd. Any explanations for that?
 

Hi, Sean!

Really good point. Maybe it makes sense to introduce a kind of REST API web-app and class to provide a general way to load text resources from serverside for js and other apps which utilize REST and JSON.

And yes, get the resources by id and languages constant. 

 

The next question is, what hashing algorithm is Cache using to create the hash?

CRC32

I will need to replicate that in JavaScript, or come up with some other hash approach.

Localization for Client-side Text

I assume $mvv(58) is set by the system when its installed? But can't find any information on that at the moment.

$mvv(58) - Default Language for a session either user defined or CSP based.

PS: Look at the source code %occMessages.inc - there's a lot of tasty there ;) or read the article in my blog, but it's in russian language :(

Hi, Sean!

I'm also wondering if there is any advice on the best way to deploy the language pack (global) with an application on GitHub. If the application was developed such that it bakes the language at compile time, then how would you bundle the language into the build file. Would you just include all of the languages or have the user install the language of choice separately.

I would suggest to export global in XML and store the file in sources. Pro: no losses, no conversion issues, same import/export as for code stuff. Like:

D $System.OBJ.Export("CacheMsg.GBL","cachemsg.gbl.xml")

Con: large size. But lang resources don't seem to be example of really large global.

For deployment purposes, e.g. what to upload to Github's Releases section, I like recently discussed recipe to export/import global with gz packing on the fly.

 

Thanks Evgeny,

As its config data its a shame to have it converted to Base64 and then wrapped up in XML.

Would be nice to see it as JSON in clear text so that it display nicely on GitHub.

Guess this might not work for certain types of binary data, but should work in this instance.

Agreed. Do you have any import/export Global-to-JSON function?

I have something that can be adapted. It will need to handle $LB.

If you can share something which even not cover $LB it would be already suitable for a lot of "Config-in-Global" cases.