Openssl and compatibility with S/MIMEv3.1

RT Developers
1

Hello all,

After switching to FastCGI to fix problems with mod_perl and env
variables (thanks to Alex Vandiver’s suggestion), I have found another
issue in my quest to achieve a fully functional S/MIME RT deployment.

Some of the S/MIME encrypted mails sent by Windows+Outlook users, were
not correctly decrypted by RT/openssl. The error shown is appended at
the end of the message for reference.

After some headbanging I discovered the “openssl smime” command that
RT uses is only fully compatible with S/MIMEv2 (PKCS#7) [1].
S/MIMEv3.1 (the IETF standard), standarized the use of Cryptographic
Message Syntax (CMS) format. For this, openssl developers implemented
a new command (openssl cms) that implements all S/MIME operations and
is supposed to deprecate “smime” option [2].

My tests show that “openssl cms” can decrypt all messages that I threw
at it, both PKCS#7 and CMS.
For encryption/signing, cms should produce messages which are
backwards compatible with PKCS#7, as long as you don’t use some
specific options [3]. These are not used in RT, AFAIK.

I have copied rt4/lib/RT/Crypt/SMIME.pm to
rt4/local/lib/RT/Crypt/SMIME.pm, replacing “smime” with “cms” in every
command issued by RT, and it seems to work well. However, it would
require more thorough testing.

I plan to do a proper modification and pull request, but I thought it
would be better asking in the list: is there any reason I am missing
to keep using “openssl smime”?

Thank you in advance,

Marcos.

[1] /docs/manmaster/man1/smime.html
[2] smime - OpenSSL PKCS#7 vs. S/MIME - Information Security Stack Exchange
[3] /docs/manmaster/man1/cms.html

Error in RT log for S/MIMEv3.1 CMS messages:
[8282] [Mon Dec 8 15:02:04 2014] [error]: openssl exited with error
code 2 and error: Error reading S/MIME message
3493945685672:error:0D078089:asn1 encoding
routines:ASN1_ITEM_EX_D2I:missing
eoc:tasn_dec.c:470:Type=PKCS7_ENVELOPE
3493945685672:error:0D08303A:asn1 encoding
routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:751:
3493945685672:error:0D08403A:asn1 encoding
routines:ASN1_TEMPLATE_EX_D2I:nested asn1
error:tasn_dec.c:579:Field=d.enveloped, Type=PKCS7
3493945685672:error:0D0D106E:asn1 encoding
routines:B64_READ_ASN1:decode error:asn_mime.c:193:
3493945685672:error:0D0D40CB:asn1 encoding
routines:SMIME_read_ASN1:asn1 parse error:asn_mime.c:528:
(/opt/rt4/sbin/…/lib/RT/Crypt/SMIME.pm:604)

2

I plan to do a proper modification and pull request, but I thought it
would be better asking in the list: is there any reason I am missing
to keep using “openssl smime”?

Thanks for the thorough investigation! I was not aware of openssl cms. I think the only reason I can think to use “openssl smime” is
that “openssl cms” was introduced in OpenSSL 1.0. Though it was
backported to 0.9.8h, it is not enabled by default there. As such,
Ubuntu Lucid and Debian squeeze, which are both distributions still
nominally in some amount of support, and which meet all other
requirements to run RT 4.2, would be left in the lurch.

Since the command-line forms of “openssl smime” and “openssl cms” are
compatible, and we’re already running openssl list-standard-commands,
care to switch the logic to check for cms then smime – storing the
value in a $self->OpenSSLCommand, like is done with ->OpenSSLPath ?

Thanks again for doing the research.

  • Alex