SHA256 Signing with RSA PSS padding

Hi everyone,

I have a project which requires the sending of JSON messages to an external service provider using REST. The service provider requires the message contents to be signed.

Their instructions:

  1. Add a header called "Date" with the the date and time in a specific format - done
  2. Add the client's certificate password in a field in the header - done
  3. Create a string which consist of the {Date}{newline}{Password}{newline}{etc}{Message Body}.
    1. Convert to a UTF8 byte array
    2. SHA256 sign the value with the certificate and private key and use RSA PSS padding
    3. Base 64 Encode the value and place it in a Signature field in the header.

I've done the following:

  1. Set up X.509 credentials using the certificate and private key files
  2. Created the string to sign as per their instruction
  3. Performed a $zconvert, 'O', 'UTF8' on the string
  4. Used %SYSTEM.Encryption -> RSASHASign() and Base64Encode()

This does not seem to be correct, as the service provider keeps rejecting the messages.

Is there a way to specify the RSA padding to be PSS?
Am I using the wrong method?
Does this method actually use PSS padding and I should look for the problem somewhere else?
Are these methods endian-ness aware?

Thank you in advance.

  • 0
  • 0
  • 5715
  • 7
  • 1


You can try to do it using the OpenSSL libraries, which comes complete with Caché/Ensemble/etc.
To google: "openssl rsa-pss sign", "openssl SHA256 with RSA PSS padding"

Here is a small example on Windows, where it is assumed that

  • cert.pem is your certificate:
    -----END CERTIFICATE-----
  • key.pem is your private key:
    -----END RSA PRIVATE KEY-----

So (test.bat):

@echo off
echo Delete all temporary files
del /Q /F test.txt test.sig pubkey.pem test.b64
echo Extract the public key from certificate (only be done once)
openssl x509 -pubkey -in cert.pem -noout > pubkey.pem
echo Create test file (test.txt)
echo bla-bla-bla test123 {Date}{newline}{Password}{newline}{etc}{Message Body} > test.txt
echo Create signature (test.sig)
openssl dgst -sha256 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:-1 -sign key.pem -out test.sig test.txt
echo This step is only for information/verification.
echo Verify signature (The result should be: "Verified OK")
openssl dgst -sha256 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:-1 -signature test.sig -verify pubkey.pem test.txt
echo Convert signature to Base64 (test.b64)
echo You can this step be make on COS.
openssl base64 -in test.sig -out test.b64 -nopad

Or on COS:

#include %systemInclude
#include %occErrors
main() public {

  try {
    n $namespace

    if '##class(%File).Exists(filePubKey{
      ; Only be done once
      ; Extract the public key from certificate
      cmd=$$$FormatText("openssl x509 -pubkey -in %1 -noout > %2",fileCert,filePubKey)
      d $zf(-1,cmd)
    i=fileMsg,fileSig,file64 ##class(%File).Delete(i)
    file.Write("{etc}{Message Body}")
    w $$$FormatText("Create signature (%1)",fileSig),!
    cmd=$$$FormatText("openssl dgst -sha256 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:-1 -sign %1 -out %2 %3",fileKey,fileSig,fileMsg)
    d $zf(-1,cmd)
    w $$$FormatText("Convert signature to Base64 (%1)",file64),!
    cmd=$$$FormatText("openssl base64 -in %1 -out %2 -nopad",fileSig,file64)
    d $zf(-1,cmd)
    ;here we read our file test.b64 (file64) and place it in a Signature field in the header
    "Error "ex.DisplayString(),!

  i=fileMsg,fileSig ##class(%File).Delete(i)

Thank you. If it was possible to up-mark the answer more than once, I would have done it.

There is a difference in the values created by Cache and by OpenSSL

You can convert binary openssl output to base64 and write that to file:

cmd=$$$FormatText("openssl dgst -sha256 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:-1 -sign %1 %2 | base64 > %3",fileKey,fileMsg,file64)

base64 is available in most linux flavors, and on Windows in various GNU CoreUtils builds and in Git (usually under C:\Program Files\Git\usr\bin\).


Also in an a business operation filenames should be generated randomly (probably via ##clss(%File).TempFilename(ext)) to avoid conflicts.

Personally, I prefer to use all out of the box, so as not to produce zoo libraries/technologies/languages, etc.

Really, both operations possible to execute at a time, for instance so:

w $$$FormatText("Create a signature and convert it to base64 (%1)",file64),!
cmd=$$$FormatText("openssl dgst -sha256 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:-1 -sign %1 %2 | openssl base64 -out %3 -nopad",fileKey,fileMsg,file64)
d $zf(-1,cmd)

I've created a config file from some random example and placed it in the bin directory. I can run it from there in the command prompt, but the $zf does not execute it.

Would you mind sharing your openssl config file and where it should reside? Which paths should be configured on Windows?

Will the user require the %CallOut service to be available to do this on a locked down install(production environment)

Ensemble runs under other user, so:

1. Compare environment variables from terminal and Ensemble (path is especially important):

do $zf(-1,"set > vars.txt")

2. Check that ensemble can access all required files.

3. Check working directory.

4. (Optional) Provide full paths to all files and binaries.


Since you didn't specified which service you're using it's hard to simulate your doubt.

However I did a quick research and noticed that 'padding' actually refers to OAEP, this could be the method you want to use to encrypt. I'm not sure though.

/// This method performs RSA encryption as specified in
/// PKCS #1 v2.1: RSA Cryptography Specifications, section 7 Encryption Schemes.
/// <br><br>
/// Input parameters:
/// <br><br>
/// Plaintext - Data to be encrypted.
/// <br><br>
/// Certificate - An X.509 certificate containing the RSA public key to be used for encryption,
/// in PEM encoded or binary DER format.
/// Note that the length of the plaintext can not be greater than the length of the modulus of
/// the RSA public key contained in the certificate minus 42 bytes.
/// <br><br>
/// CAfile - The name of a file containing trusted Certificate Authority X.509 Certificates in PEM-encoded format, one of which was
/// used to sign the Certificate (optional).
/// <br><br>
/// CRLfile - The name of a file containing X.509 Certificate Revocation Lists in PEM-encoded format that should be checked
/// to verify the status of the Certificate (optional).
/// <br><br>
/// Encoding - PKCS #1 v2.1 encoding method (optional):<br>
///     1 = OAEP (default)<br>
///     2 = PKCS1-v1_5<br>
/// <br><br>
/// Return value: Ciphertext.
ClassMethod RSAEncrypt(
Plaintext As %String,
Certificate As %String,
CAfile As %String,
CRLfile As %String,
Encoding As %Integer) As %String