Cryptography Job Interview Questions and How to Answer Them

Cryptography interview questions test whether you understand the math behind the algorithms, not just which library function to call. Hiring managers at companies like Google, Cloudflare, and Matrix ask questions that separate candidates who have implemented crypto from those who have only imported a library.

This post covers the questions that appear most frequently in cryptography and security engineering interviews. Each question comes with a technical answer that cites the relevant RFC or NIST publication, so you can verify the details yourself. The questions span symmetric and asymmetric encryption, hash functions, encryption modes, authentication mechanisms, and common attacks.

The answers here go deeper than a flashcard. A good interview answer explains the mechanism, names the standard, and identifies at least one attack or limitation. Memorizing "AES is symmetric and RSA is asymmetric" will not get you past a technical screen. Understanding why RSA requires coprime exponents and why GCM is preferred over CBC will.

You can follow along with the AES Encrypt/Decrypt tool and the RSA Encrypt/Decrypt tool to test the algorithms as you review each answer.

This is the most common opening question. The expected answer goes beyond the textbook definition.

Symmetric encryption uses one key for both encryption and decryption. AES is the standard, defined in FIPS 197. It is fast (hardware implementations exceed 10 Gbps), produces small ciphertexts, and is used for bulk data encryption. The problem is key distribution: both parties need the same key, and getting it to them securely is a separate challenge.

Asymmetric encryption uses a key pair: a public key for encryption and a private key for decryption. RSA is the most common example, with the algorithm published in 1977. It solves the key distribution problem because the public key can be shared openly. The trade-off is speed: RSA-2048 encryption is roughly 100x slower than AES-128, and RSA ciphertexts are at least 256 bytes regardless of plaintext size.

In practice, systems use both. TLS, defined in RFC 8446, uses asymmetric key exchange (ECDHE) to establish a shared secret, then uses that secret with symmetric encryption (AES-GCM) for the actual data. This hybrid approach gets the key distribution benefit of asymmetric crypto and the performance of symmetric crypto. The Diffie-Hellman Key Exchange tool demonstrates the key agreement step.

A strong answer also mentions that asymmetric encryption has size limits. RSA-2048 can encrypt at most 245 bytes of plaintext with OAEP padding. For anything larger, you encrypt a symmetric key with RSA and encrypt the data with AES. This is how PGP and S/MIME work.

Interviewers ask this to test whether you can explain the actual algorithm, not just name it. RSA has three steps.

Key generation: Choose two large primes p and q. Compute n = p * q. Compute the totient phi(n) = (p-1)(q-1). Choose a public exponent e such that 1 < e < phi(n) and gcd(e, phi(n)) = 1. Compute the private exponent d = e^(-1) mod phi(n) using the extended Euclidean algorithm. The public key is (n, e). The private key is (n, d).

Encryption: C = m^e mod n, where m is the plaintext (as an integer less than n).

Decryption: m = C^d mod n.

The security relies on the difficulty of factoring n. If you can factor n into p and q, you can compute phi(n) and derive d. The RSA Encrypt/Decrypt tool lets you inspect these values.

A common follow-up: why must e be coprime with phi(n)? Because d is the modular inverse of e modulo phi(n), and a modular inverse exists only when gcd(e, phi(n)) = 1. The most common choice is e = 65537 (0x10001), which is prime and therefore coprime with phi(n) for any n that is a product of two large primes.

Another follow-up: what is the difference between PKCS#1 v1.5 padding and OAEP? PKCS#1 v1.5 padding is vulnerable to Bleichenbacher's attack (described below). OAEP (Optimal Asymmetric Encryption Padding), defined in RFC 8017, is the modern standard and is not vulnerable to the same attack. Always use OAEP.

The question "what properties must a cryptographic hash function have?" appears in almost every cryptography interview. The answer has three named properties, and you should be able to define each precisely.

Preimage resistance: given a hash value h, it is computationally infeasible to find any input m such that hash(m) = h. This is what makes password hashing work: even if the attacker knows the hash, they cannot recover the password.

Second preimage resistance: given an input m1, it is computationally infeasible to find a different input m2 such that hash(m1) = hash(m2). This is what makes hash-based file integrity checking work: an attacker cannot swap a file for a different one with the same hash.

Collision resistance: it is computationally infeasible to find any two distinct inputs m1 and m2 such that hash(m1) = hash(m2). This is a stronger property than second preimage resistance. Collision resistance implies second preimage resistance, but not vice versa.

SHA-256, defined in FIPS 180-4, satisfies all three properties. MD5 does not: collision attacks are practical (Xie Wang et al. published a collision attack in 2010 that runs in seconds on a modern GPU). SHA-1 is also broken for collisions (Google and CWI Amsterdam demonstrated a practical collision in 2017 with the SHAttered attack). The SHA-256 Hash Generator lets you compute hashes and verify these properties.

A strong answer also mentions the birthday paradox. For a hash function with n-bit output, collision resistance provides n/2 bits of security. SHA-256 has a 256-bit output, so collision attacks require approximately 2^128 operations. Second preimage resistance and preimage resistance provide the full n bits: 2^256 operations each.

This question tests whether you understand encryption modes and authenticated encryption. The answer should cover why GCM has replaced CBC in nearly all new systems.

CBC (Cipher Block Chaining) encrypts each block by XORing it with the previous ciphertext block before encryption. It requires an initialization vector (IV) that is unpredictable but not necessarily secret. CBC provides confidentiality but not integrity. An attacker can modify ciphertext blocks and the recipient will not know. This is why CBC is always paired with a separate MAC (HMAC, for example) in protocols that use it.

GCM (Galois/Counter Mode) is an AEAD (Authenticated Encryption with Associated Data) mode. It encrypts data using AES in counter mode and computes a Galois-field MAC over the ciphertext and any associated data (like HTTP headers or TLS record metadata). One operation provides both confidentiality and integrity. GCM is defined in NIST SP 800-38D.

The practical difference: with CBC, you must implement encrypt-then-MAC separately and get the order right (encrypt first, then MAC the ciphertext, not the plaintext). Getting this wrong leads to padding oracle attacks. With GCM, the authentication is built in. You cannot forget it or implement it in the wrong order.

GCM has one caveat: the nonce (IV) must never be reused with the same key. Reusing a GCM nonce allows an attacker to recover the authentication key and forge messages. This is what broke WPA2's GCMP mode. CBC is more forgiving with IV reuse (it degrades to a known-plaintext attack rather than a complete break), but that does not justify choosing CBC over GCM for new systems. The AES Encrypt/Decrypt tool supports both modes for comparison.

Both signatures and MACs provide message authentication, but they differ in a way that matters legally and architecturally.

A MAC (Message Authentication Code) uses a shared symmetric key. The sender computes MAC(key, message) and appends it. The recipient recomputes it and checks equality. HMAC, defined in RFC 2104, is the standard construction. The HMAC Generator computes HMAC values. MACs are fast and simple, but they have a limitation: any key holder can produce the MAC. If Alice and Bob share a key, and Bob receives a message with a valid MAC, Bob knows it came from Alice or from himself. He cannot prove to a third party which one produced it.

A digital signature uses asymmetric keys. The signer produces signature = sign(private_key, message). Anyone with the public key can verify: verify(public_key, message, signature). Only the private key holder can sign, so the signature is non-repudiable. Bob can show a third party (a judge, for example) that Alice signed the message, and Alice cannot deny it.

The trade-off: signatures are slower and larger than MACs. ECDSA with P-256 produces 64-byte signatures. HMAC-SHA256 produces 32-byte tags and is 100x faster. In TLS 1.3, signatures are used only for server authentication during the handshake. All record-level authentication uses AEAD (GCM), which is symmetric.

Interviewers often ask: can you use a MAC for non-repudiation? No. Non-repudiation requires asymmetric cryptography. A MAC proves the message came from a key holder but cannot prove which key holder produced it.

Naming specific attacks signals real knowledge. Here are the ones that come up most often in interviews.

Padding oracle attack: CBC mode with PKCS#7 padding leaks information about whether padding is valid. By sending modified ciphertexts and observing whether the server returns a padding error or a decryption error, an attacker can recover plaintext byte by byte. This was described by Serge Vaudenay at Eurocrypt 2002. The fix is to use AEAD (GCM) or to verify the MAC before checking padding (encrypt-then-MAC).

Bleichenbacher's attack: RSA with PKCS#1 v1.5 padding is vulnerable to a chosen-ciphertext attack where the attacker sends modified ciphertexts and uses the server's response (whether the padding is valid) to recover plaintext. Daniel Bleichenbacher published this attack in 1998. The fix is OAEP padding, which is not vulnerable. The ROBOT attack showed in 2017 that many implementations still had Bleichenbacher vulnerabilities nearly 20 years later.

Birthday attack: named after the birthday paradox, this attack exploits the collision resistance bound. For a hash with n-bit output, you need only 2^(n/2) attempts to find a collision with high probability. This is why SHA-1 (160-bit output) provides only 80 bits of collision security, which is below the 112-bit minimum NIST recommends. SHA-256 provides 128 bits of collision security, which is adequate.

Timing attack: implementations that branch on secret data (like a byte-by-byte MAC comparison) leak information through execution time. The fix is constant-time comparison, which XORs all bytes and checks whether the result is zero. Most crypto libraries provide a constant-time compare function. The ECDSA Signature tool demonstrates signature verification, which must also be implemented in constant time to avoid leaking the private key through nonce bias.