Blog
All Blog Posts | Next Post | Previous Post
Why direct RSA string encryption fails for large strings and how CMS envelopedData solves it?
Today
Unlike other asymmetric cryptographic algorithm, RSA can be used to sign a message and verify its signature, as well as encrypt and decrypt a message. When developers implement public-key cryptography, an initial thought is to take a plaintext string, encrypt it directly with a recipient's RSA public key, and send the output. Soon enough, however, strings grow slightly longer, and the application crashes with errors like
Data too large for key size or RSA_padding_check_failed.Understanding why RSA fails for arbitrary payloads (I. e., large strings) and how standard protocols like Cryptographic Message Syntax (CMS) solve this is essential for building robust security architectures. TMS Cryptography Pack already used CMS structures to sign PDF files with PAdES and now includes CMS envelopedData structures for encrypted objects.
The core problem: the mathematics of RSA key size
RSA relies on modular exponentiation. Data is treated as a single large integer m, which must strictly satisfy the condition:
m < n, where n is the RSA modulus (the product of two prime numbers, p and q).
1. Key size limits max plaintext size
A 2048-bit RSA key means the modulus n is 2048 bits (256 bytes) long. Consequently, the raw numerical value of your input data cannot exceed 256 bytes (resp 384 and 512 for 3072 and 4096-bit keys).
2. Padding overhead reduces capacity further
In practice, raw ("textbook") RSA is never used because it is deterministic and vulnerable to mathematical attacks. Secure implementations apply padding schemes like OAEP (Optimal Asymmetric Encryption Padding) or PKCS 1.5 to introduce randomness and integrity checks. OAEP shall be preferred as it is more secure.
Padding consumes space within the 256-byte block:
| RSA Key Size | Padding Scheme | Overhead | Max Data Length |
| 2048 bits (256 B) | PKCS#1 v1.5 | 11 bytes | 245 bytes |
| 2048 bits (256 B) | OAEP (SHA-256 + MGF1) | 66 bytes | 190 bytes |
| 4096 bits (512 B) | OAEP (SHA-256 + MGF1) | 66 bytes | 446 bytes |
Attempting to encrypt a 500-character JSON string or a multi-line document directly with a 2048-bit RSA key using OAEP fails instantly.
Why "chunking" RSA is a bad anti-pattern
A common workaround is splitting long strings into smaller chunks, encrypting each with RSA, and concatenating the result. Do not do this.
- Performance: RSA is computationally expensiveorders of magnitude slower than symmetric encryption.
- Ciphertext expansion: encrypting ten 100-byte chunks results in ten full 256-byte blocks (2560 bytes total).
- Security & integrity risks: standard RSA padding isn't designed for multi-block streaming. Reordering, dropping, or swapping individual chunks becomes trivial for an attacker unless complex custom framing is added.
The standard solution: hybrid cryptography via CMS envelopedData
To encrypt arbitrary-length data efficiently, asymmetric and symmetric cryptography are combined into a hybrid cryptosystem.
CMS envelopedData (defined in RFC 5652) is the industry standard for structuring this process. It decouples message confidentiality from key management.
How CMS EnvelopedData Works
- Symmetric payload encryption: A unique, random symmetric key (e.g., a 256-bit AES key) is generated. The input payloadregardless of whether it is 10 bytes or 10 gigabytesis encrypted using fast symmetric ciphers like AES-GCM or AES-CBC.
- Asymmetric key wrapping: The small, fixed-size symmetric key (32 bytes for AES-256) is encrypted using the recipient's RSA public key (
KeyTransRecipientInfo). Because 32 bytes easily fits within RSA-OAEP size constraints, payload size limitations disappear. - Standardized Envelope: CMS packages the encrypted payload, the RSA-encrypted symmetric key, recipient identifiers (certificate serial numbers or key IDs), and algorithm identifiers into a self-contained ASN.1 structure.
Key advantages of using CMS envelopedData
- Arbitrary length: encrypt strings or streams of any size without size checks or chunking hacks.
- Performance: high-speed symmetric algorithms handle the heavy lifting; slow asymmetric operations only run once per message on a 32-byte key.
- Multi-recipient support out of the box: The same payload can be made readable by multiple parties simply by adding extra encrypted key blocks (
RecipientInfo), encrypting the underlying payload only once (this will be added in a future release). - Interoperability: supported natively across major platforms and libraries (OpenSSL, Java Bouncy Castle, .NET
EnvelopedCms).
Example
As usual, we tried to make things as simple as possible for developers. This example requires a TForm, 2 TButtons, a TEdit and a TMemo.
procedure TMainForm.GenerateCMSBtnClick(Sender: TObject);
var
PlainText: string;
s: string;
CMS: TCms;
begin
PlainText := TextEdit.Text; // the text to encrypt
CMS := TCms.Create(nil); // the real work is done in the TCMS class
try
CMS.encType := TRSAEncType.oaep; // default is PKCS 1.5
s := CMS.GenerateCMSEnvelopedData(PlainText, '..\..\Certs\MyCert.pem', '.\MyLittleCMS.pem'); // path to files
MainMemo.Lines.Add(s); // put the result in the TMemo, in PEM format (base64 + header and footer)
finally
CMS.Free;
end;
end;
procedure TMainForm.ValidateCMSBtnClick(Sender: TObject);
var
s, IssuerDER: string;
CMS: TCms;
begin
CMS := TCms.Create(nil);
try
CMS.encType := TRSAEncType.oaep;
s := CMS.DecryptCMSEnvelopedData(
'.\MyLittleCMS.pem', '..\..\Certs\MyKey.pem', IssuerDER); // Issuer is not used in this version
MainMemo.Lines.Add(s);
finally
CMS.Free;
end;
end;
Conclusion
RSA was never designed to encrypt raw data directlyit was built to establish shared secrets or transport small keys. Facing RSA size limitations is a sign that hybrid encryption is required. By adopting CMS envelopedData, you follow an RFC-standard pattern that handles arbitrary message sizes cleanly, securely, and efficiently.
This feature has been added to TMS Cryptography Pack in version 5.3.2.0 with AES-CBC (256 bit-key only for encoding) and RSA either OEAP or PKCS#1.5 (2048, 3072 and 4096-bit keys) for recipient's key encryption.
Bernard Roussely
This blog post has not received any comments yet. Add a comment.
All Blog Posts | Next Post | Previous Post