Symmetric Encryption/Decryption in the AWS SDK for C++ with std::iostream

Cryptography is hard in any programming language. It is especially difficult in platform-portable native code where we don’t have the advantage of a constant platform implementation. Many customers have asked us for an Amazon S3 encryption client that is compatible with the Java and Ruby clients. Although we are not ready to release that yet, we have created a way to handle encryption and decryption across each of our supported platforms: the *nix variations that rely mostly on OpenSSL; Apple, which provides CommonCrypto; and Windows which exposes the BCrypt/CNG library. Each of these libraries provide dramatically different interfaces, so we had to do a lot of work to force them into a common interface and usage pattern. As of today, you can use the AWS SDK for C++ to encrypt or decrypt your data with AES 256-bit ciphers in the CBC, CTR, and GCM modes. You can use the ciphers directly, or you can use the std::iostream interface. This new feature is valuable even without the high-level client we are hoping to provide soon, so we are excited to tell you about it.

Let’s look at a few use cases:

Suppose we want to write sensitive data to file using a std::ofstream and have it encrypted on disk. However, when we read the file back, we want to parse and use the file in a std::ifstream as plaintext. Because we use AES-GCM in this example, after encryption, the tag and iv must be stored somewhere in order for the data to be decrypted. For this example, we will simply store them into memory so we can pass them to the decryptor.

#include <aws/core/Aws.h>
#include <aws/core/utils/crypto/CryptoStream.h>
#include <fstream>

using namespace Aws::Utils;
using namespace Aws::Utils::Crypto;

int main()
{
    Aws::SDKOptions options;
    Aws::InitAPI(options);
	{
		//create 256 bit symmetric key. This will use the entropy from your platform's crypto implementation
		auto symmetricKey = SymmetricCipher::GenerateKey();

		//create an AES-256 GCM cipher, iv will be autogenerated
		auto encryptionCipher = CreateAES_GCMImplementation(symmetricKey);

		const char* fileToEncrypt = "./encryptedSensitiveData";

		CryptoBuffer iv, tag;

		//write the file to disk and encrypt it
		{        
			//create the stream to receive the encrypted data.
			Aws::OFStream outputStream(fileName, std::ios_base::out | std::ios_base::binary);

			//now create an encryption stream.
			SymmetricCryptoStream encryptionStream(outputStream, CipherMode::Encrypt, cipher);
			encryptionStream << "This is a file full of sensitive customer secrets:\n\n";
			encryptionStream << "CustomerName=John Smith\n";
			encryptionStream << "CustomerSSN=867-5309\n";
			encryptionStream << "CustomerDOB=1 January 1970\n\n";
		}

		 //grab the IV that was used to initialize the cipher
		 auto iv = encryptionCipher->GetIV();

		 //since this is GCM, grab the tag once the stream is finished and the cipher is finalized
		 auto tag = encryptionCipher->GetTag();

		//read the file back from disk and deal with it as plain-text
		{
			 //create an AES-256 GCM cipher, passing the key, iv and tag that were used for encryption
			 auto decryptionCipher = CreateAES_GCMImplementation(symmetricKey, iv, tag);

			 //create source stream to decrypt from
			 Aws::IFStream inputStream(fileName, std::ios_base::in | std::ios_base::binary);         

			 //create a decryption stream
			 SymmetricCryptoStream decryptionStream(inputStream, CipherMode::Decrypt, cipher);

			 //write the file out to cout using normal stream operations
			 Aws::String line;
			 while(std::getline(stream, line))
			 {
				 std::cout << line << std::endl;
			 }
		}        
	}
    Aws::ShutdownAPI(options);
    return 0;
}

What if this time we want to put the encrypted data from plaintext on local disk into Amazon S3? We want it to be encrypted in Amazon S3, but we want to write it to disk as plaintext when we download it. This example code is not compatible with the existing Amazon S3 encryption clients; a compatible client is coming soon.

#include <aws/core/Aws.h>
#include <aws/core/utils/crypto/CryptoStream.h>
#include <aws/s3/S3Client.h>
#include <aws/s3/model/PutObjectRequest.h>
#include <aws/s3/model/GetObjectRequest.h>

using namespace Aws::Utils;
using namespace Aws::Utils::Crypto;
using namespace Aws::S3;

int main()
{
    Aws::SDKOptions options;
    Aws::InitAPI(options);
	{
		//create 256 bit symmetric key. This will use the entropy from your platform's crypto implementation
		auto symmetricKey = SymmetricCipher::GenerateKey();    

		const char* fileName = "./localFile";
		const char* s3Key = "encryptedSensitiveData";
		const char* s3Bucket = "s3-cpp-sample-bucket";

		//create an AES-256 CTR cipher
		auto encryptionCipher = CreateAES_CTRImplementation(symmetricKey);

		//create an S3 client
		S3Client client;

		//Put object into S3
		{
			//put object
			Model::PutObjectRequest putObjectRequest;
			putObjectRequest.WithBucket(s3Bucket)
					.WithKey(s3Key);

			//source stream for file we want to put encrypted into S3
			Aws::IFStream inputFile(fileName);

			//set the body to a crypto stream and we will encrypt in transit
			putObjectRequest.SetBody(
					Aws::MakeShared<CryptoStream>("crypto-sample", inputFile, CipherMode::Encrypt, *encryptionCipher));

			auto putObjectOutcome = client.PutObject(putObjectRequest);

			if(!putObjectOutcome.IsSuccess())
			{
				std::cout << putObjectOutcome.GetError().GetMessage() << std::endl;
			}
		 }

		 auto iv = GetIV();

		 //create cipher to use for decryption of AES-256 in CTR mode.
		 auto decryptionCipher = CreateAES_CTRImplementation(symmetricKey, iv);     

		 {
			//get the object back out of s3
			Model::GetObjectRequest getObjectRequest;
			getObjectRequest.WithBucket(s3Bucket)
					.WithKey(s3Key);

			//destination stream for the decrypted result to be written to.
			Aws::OFStream outputFile(fileName);

			//tell the client to create a crypto stream that knows how to decrypt the data as it comes across the wire.
			// write the decrypted output to outputFile.
			getObjectRequest.SetResponseStreamFactory(
					[&] { return Aws::New<CryptoStream>("crypto-sample", outputFile, CipherMode::Decrypt, *decryptionCipher);}
			);

			auto getObjectOutcome = client.GetObject(getObjectRequest);

			if(!getObjectOutcome.IsSuccess())
			{
				std::cout << getObjectOutcome.GetError().GetMessage() << std::endl;
				return -1;
			}
			//the file should now be stored decrypted at fileName        
		}
	}
    Aws::ShutdownAPI(options);

    return 0;
}

Unfortunately, this doesn’t solve the problem of securely transmitting or storing symmetric keys. Soon we will provide a fully automated encryption client that will handle everything, but for now, you can use AWS Key Management Service (AWS KMS) to encrypt and store your encryption keys.

A couple of things to keep in mind:

• You do not need to use the Streams interface directly. The SymmetricCipher interface provides everything you need for cross-platform encryption and decryption operations. The streams are mostly for convenience and interoperating with the web request process.
• When you use the stream in sink mode, you either need to explicitly call Finalize() or make sure the destructor of the stream has been called. Finalize() makes sure the cipher has been finalized and all hashes (in GCM mode) have been computed. If this method has not been called, the tag from the cipher may not be accurate when you try to pull it.
• Because encrypted data is binary data, be sure to use the correct stream flags for binary data. Anything that depends on null terminated c-strings can create problems. Use Strings and String-Streams only when you know that the data is plaintext. For all other scenarios, use the non-formatted input and output operations on the stream.
• AES-GCM with 256-bit keys is not implemented for Apple platforms. CommonCrypto does not expose this cipher mode in its API. As soon as Apple adds AES-GCM to its public interface, we will provide an implementation for that platform. If you need AES-GCM on Apple, you can use our OpenSSL implementation.
• Try to avoid seek operations. When you seek backwards, we have to reset the cipher and re-encrypt everything up to your seek position. For S3 PutObject operations, we recommend that you set the Content-Length ahead of time instead of letting us compute it automatically. Be sure to turn off payload hashing.

We hope you enjoy this feature. We look forward to seeing the ways people use it. Please leave your feedback on GitHub, and as always, feel free to send us pull requests.