Stefan Cronje · Jul 10, 2017

SHA256 Signing with RSA PSS padding

Hi everyone,

I have a project which requires the sending of JSON messages to an external service provider using REST. The service provider requires the message contents to be signed.

Their instructions:

  1. Add a header called "Date" with the the date and time in a specific format - done
  2. Add the client's certificate password in a field in the header - done
  3. Create a string which consist of the {Date}{newline}{Password}{newline}{etc}{Message Body}.
    1. Convert to a UTF8 byte array
    2. SHA256 sign the value with the certificate and private key and use RSA PSS padding
    3. Base 64 Encode the value and place it in a Signature field in the header.

I've done the following:

  1. Set up X.509 credentials using the certificate and private key files
  2. Created the string to sign as per their instruction
  3. Performed a $zconvert, 'O', 'UTF8' on the string
  4. Used %SYSTEM.Encryption -> RSASHASign() and Base64Encode()

This does not seem to be correct, as the service provider keeps rejecting the messages.

Is there a way to specify the RSA padding to be PSS?
Am I using the wrong method?
Does this method actually use PSS padding and I should look for the problem somewhere else?
Are these methods endian-ness aware?

Thank you in advance.

1 0 8 9,521
Log in or sign up to continue


Since you didn't specified which service you're using it's hard to simulate your doubt.

However I did a quick research and noticed that 'padding' actually refers to OAEP, this could be the method you want to use to encrypt. I'm not sure though.

/// This method performs RSA encryption as specified in
/// PKCS #1 v2.1: RSA Cryptography Specifications, section 7 Encryption Schemes.
/// <br><br>
/// Input parameters:
/// <br><br>
/// Plaintext - Data to be encrypted.
/// <br><br>
/// Certificate - An X.509 certificate containing the RSA public key to be used for encryption,
/// in PEM encoded or binary DER format.
/// Note that the length of the plaintext can not be greater than the length of the modulus of
/// the RSA public key contained in the certificate minus 42 bytes.
/// <br><br>
/// CAfile - The name of a file containing trusted Certificate Authority X.509 Certificates in PEM-encoded format, one of which was
/// used to sign the Certificate (optional).
/// <br><br>
/// CRLfile - The name of a file containing X.509 Certificate Revocation Lists in PEM-encoded format that should be checked
/// to verify the status of the Certificate (optional).
/// <br><br>
/// Encoding - PKCS #1 v2.1 encoding method (optional):<br>
///     1 = OAEP (default)<br>
///     2 = PKCS1-v1_5<br>
/// <br><br>
/// Return value: Ciphertext.
ClassMethod RSAEncrypt(
Plaintext As %String,
Certificate As %String,
CAfile As %String,
CRLfile As %String,
Encoding As %Integer) As %String

Thank you. If it was possible to up-mark the answer more than once, I would have done it.

There is a difference in the values created by Cache and by OpenSSL

You can convert binary openssl output to base64 and write that to file:

cmd=$$$FormatText("openssl dgst -sha256 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:-1 -sign %1 %2 | base64 > %3",fileKey,fileMsg,file64)

base64 is available in most linux flavors, and on Windows in various GNU CoreUtils builds and in Git (usually under C:\Program Files\Git\usr\bin\).


Also in an a business operation filenames should be generated randomly (probably via ##clss(%File).TempFilename(ext)) to avoid conflicts.

I've created a config file from some random example and placed it in the bin directory. I can run it from there in the command prompt, but the $zf does not execute it.

Would you mind sharing your openssl config file and where it should reside? Which paths should be configured on Windows?

Will the user require the %CallOut service to be available to do this on a locked down install(production environment)

You can try to do it using the OpenSSL libraries, which comes complete with Caché/Ensemble/etc.
To google: "openssl rsa-pss sign", "openssl SHA256 with RSA PSS padding"

Here is a small example on Windows, where it is assumed that

  • cert.pem is your certificate:
    -----END CERTIFICATE-----
  • key.pem is your private key:
    -----END RSA PRIVATE KEY-----

So (test.bat):

@echo off
echo Delete all temporary files
del /Q /F test.txt test.sig pubkey.pem test.b64
echo Extract the public key from certificate (only be done once)
openssl x509 -pubkey -in cert.pem -noout > pubkey.pem
echo Create test file (test.txt)
echo bla-bla-bla test123 {Date}{newline}{Password}{newline}{etc}{Message Body} > test.txt
echo Create signature (test.sig)
openssl dgst -sha256 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:-1 -sign key.pem -out test.sig test.txt
echo This step is only for information/verification.
echo Verify signature (The result should be: "Verified OK")
openssl dgst -sha256 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:-1 -signature test.sig -verify pubkey.pem test.txt
echo Convert signature to Base64 (test.b64)
echo You can this step be make on COS.
openssl base64 -in test.sig -out test.b64 -nopad

Or on COS:

#include %systemInclude
#include %occErrors
main() public {

  try {
    n $namespace

    if '##class(%File).Exists(filePubKey{
      ; Only be done once
      ; Extract the public key from certificate
      cmd=$$$FormatText("openssl x509 -pubkey -in %1 -noout > %2",fileCert,filePubKey)
      d $zf(-1,cmd)
    i=fileMsg,fileSig,file64 ##class(%File).Delete(i)
    file.Write("{etc}{Message Body}")
    w $$$FormatText("Create signature (%1)",fileSig),!
    cmd=$$$FormatText("openssl dgst -sha256 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:-1 -sign %1 -out %2 %3",fileKey,fileSig,fileMsg)
    d $zf(-1,cmd)
    w $$$FormatText("Convert signature to Base64 (%1)",file64),!
    cmd=$$$FormatText("openssl base64 -in %1 -out %2 -nopad",fileSig,file64)
    d $zf(-1,cmd)
    ;here we read our file test.b64 (file64) and place it in a Signature field in the header
    "Error "ex.DisplayString(),!

  i=fileMsg,fileSig ##class(%File).Delete(i)

Personally, I prefer to use all out of the box, so as not to produce zoo libraries/technologies/languages, etc.

Really, both operations possible to execute at a time, for instance so:

w $$$FormatText("Create a signature and convert it to base64 (%1)",file64),!
cmd=$$$FormatText("openssl dgst -sha256 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:-1 -sign %1 %2 | openssl base64 -out %3 -nopad",fileKey,fileMsg,file64)
d $zf(-1,cmd)

Ensemble runs under other user, so:

1. Compare environment variables from terminal and Ensemble (path is especially important):

do $zf(-1,"set > vars.txt")

2. Check that ensemble can access all required files.

3. Check working directory.

4. (Optional) Provide full paths to all files and binaries.