Format-Preserving Encryption

For my capstone project to complete my Online Master of Science in Cybersecurity, my focus was on providing a reference implementation of Format-Preserving Encryption, specifically the FF3-1 specification as outlined by NIST 800-38G.

What is Format-Preserving Encryption

Format-Preserving Encryption (FPE) is a form of encryption that maintains the format of the original item that is being encrypted. If you wanted to encrypt your Social Security Number, which is a 9 digit number, a valid FPE ciphertext would also be a 9 digit number. It would look random to the naked eye (heck it would look just like a Social Security Number to someone who didn't know any better), but it's no longer your SSN. If an adversary happens to compromise the database where this encrypted SSN is stored, it is absolutely useless to them. Unless they also managed to grab the key, in which case the attacker is either very thorough or someone didn't follow best practices for cryptographic key management.

Other secure encryption methods generally garble the original input to where it looks like random noise, adding pesky things like initialization vectors and padding and usually increasing the length in the process. While these methods should absolutely be used in best case scenarios, there remains ample opportunity to reduce the risk of compromise in systems which either can't, or simply won't, support the latest and greatest cryptography.

Why the Interest?

In my career, I've worked for small community banks, educational service providers, and financial payment processors, among others. Without divulging any privileged information, I've spent my fair share of time around systems with access to both Personally Identifiable Information (PII) and Payment Card Industry (PCI) data, which includes credit and debit card numbers. There is always room for secure, accessible, straight-forward methods to safeguard sensitive information.

For new applications, sure, fire up AES-GCM or CHACHA20 or just enable the latest cloud thingamajig to protect your precious data. But for legacy applications and databases (and there are plenty of those out there...), you can't always apply the latest and greatest encryption methods. Databases might have strict column widths, and applications might be used to dealing with the real McCoy¹.

In either case, test coverage is often spotty at best, and I can neither confirm nor deny whether a two-day encryption-related upgrade once turned into a month-long fire drill, including lost sleep and holidays.

What's the goals?

My goals were simple:

• Provide a reference implementation of the NIST FF3-1 standard.
• Integrate said implementation with the popular PyCryptodome library
• Validate the implementation against the NIST provided test vectors
• Document the API operation, use cases, and best practices
• Release under the BSD-2 clause license, allowing "free as in speech" usage rights

Okay, where are the deets?

You can find my capstone presentation below - full link is here:

I apologize for the low technical quality. Since I was using a Teams presentation, I did this in one take (with a few practice runs) so a lot of obvious mistakes and weird transitions remain.

The presentation should be worthwhile even if you aren't familiar with cryptography, and I tried my best to follow the ACM Guide to a Successful Presentation.

If you are more the reading type or the presentation picqued your interest, the corresponding academic paper Format-Preserving Encryption is written in ACM format and goes into much further detail, including background, the full deliverables, threat model, use cases, and potential future work.

I should note that my only original contribution is my Python implementation of FF3-1, which closely follows the NIST 800-38G standard. My implementation may very well be suboptimal, and could have unknown issues, but I refrained from reviewing any related implementations during development. The code is well documented, and interested parties should feel free to review the implementation.

Sadly, my pull request to PyCryptodome has not yet been accepted into the project. However, the code remains available on GitHub.

References