Introduction
Single-letter frequency analysis is a starting point, but n-gram analysis (bigrams, trigrams, quadgrams) reveals far more about whether a text is natural language or ciphertext. In English, 'TH' is the most common bigram and 'THE' is the most common trigram. This tool counts n-grams of size 2, 3, or 4 in any text, ranks them by frequency, and compares the top results against standard English reference data. Paste your text and the analysis appears instantly.
What this tool does
- Count bigrams (2-grams), trigrams (3-grams), and quadgrams (4-grams) in the input text, with non-letter characters stripped automatically
- Rank all n-grams by frequency, showing count and percentage for each, with the top 20 displayed in a bar chart
- Compare observed n-gram frequencies against standard English reference data for bigrams and trigrams, showing expected vs observed side by side
- Display the total number of unique n-grams and total occurrences in a summary panel
- Handle overlapping n-grams correctly (e.g. 'THE' produces 'TH' and 'HE' as bigrams, 'THE', 'HEX', etc. as trigrams)
- Update results in real time as you type or change the n-gram size
How this tool works
The tool strips all non-A-Z characters from the input and converts to uppercase. For an n-gram size of N, it slides a window of length N across the cleaned text, extracting each substring and incrementing its count in a dictionary. The total number of n-gram occurrences is `textLength - N + 1`.
After counting, the results are sorted by count in descending order. The top 20 are displayed in a bar chart where bar length is proportional to count relative to the maximum. Each entry shows the n-gram, its count, and its percentage of total occurrences.
For bigrams and trigrams, the tool includes reference data from standard English frequency tables. The top 20 English bigrams are TH (3.56%), HE (3.07%), IN (2.43%), ER (2.05%), and so on. The top 20 trigrams are THE (3.51%), AND (1.59%), ING (1.15%), ENT (0.98%), and so on. The reference comparison panel shows each reference n-gram with its expected percentage alongside the observed percentage and count from your text.
Quadgram mode does not include reference data because quadgram frequency tables are large and language-specific. However, the ranked output is still useful for identifying patterns in ciphertext.
How n-gram frequency analysis works
N-gram analysis is a cornerstone of statistical cryptanalysis. Henry Beker and Fred Piper introduced it systematically in their 1982 book 'Cipher Systems: The Protection of Communications', which covers n-gram statistics and their application to breaking substitution and polyalphabetic ciphers.
Abraham Sinkov covered the mathematical foundations in 'Elementary Cryptanalysis: A Mathematical Approach' (1966), published by the Mathematical Association of America. Sinkov's text explains how n-gram frequencies arise from the structure of natural language and how they can be used to distinguish plaintext from ciphertext.
In English, the most frequent bigrams are TH, HE, IN, ER, AN, RE, ND, AT, ON, NT. These appear far more often than random chance would predict because English words have structural patterns: 'th' appears in 'the', 'this', 'that', 'then'; 'he' appears in 'the', 'he', 'when', 'here'. Trigrams are even more distinctive: 'the' alone accounts for about 3.5% of all trigrams in English text.
For cryptanalysis, n-gram frequencies serve two purposes. First, they help identify the correct decryption among candidates. A text with high frequencies of TH, HE, IN, ER, AN is likely English, while one with uniformly distributed bigrams is likely ciphertext or random text. Second, n-gram log-probabilities are used as scoring functions in automated solvers like the Hill Climbing Cipher Solver, which sums the log-probabilities of all bigrams in the candidate decryption.
For single-letter frequency analysis, see the Frequency Analysis tool. For chi-squared scoring, see the Chi-Squared Calculator. For Index of Coincidence calculations, see the Index of Coincidence.
How to use this tool
- Paste your text or ciphertext into the input field. Non-letter characters are stripped automatically
- Select the n-gram size: Bigram (2), Trigram (3), or Quadgram (4) using the buttons in the settings panel
- Review the top 20 n-grams in the bar chart, showing count and percentage for each
- For bigrams and trigrams, compare your observed frequencies against the standard English reference data in the comparison panel
- Check the summary for total unique n-grams and total occurrences
- Use the results to identify language patterns or score candidate decryptions: a text whose top n-grams match the English reference is likely plaintext
Real-world examples
Verifying a decryption with bigram analysis
After decrypting a substitution cipher, paste the result and select Bigram. If the top bigrams are TH, HE, IN, ER, AN, the decryption is likely correct. If the top bigrams are random-looking pairs like QX, ZR, KP, the decryption is wrong and you should try a different key.
Distinguishing plaintext from ciphertext using trigrams
Paste 300 characters of English plaintext and select Trigram. The top trigram will be THE, followed by AND, ING, ENT. Now paste 300 characters of substitution-encrypted ciphertext. The trigram distribution will be flat, with no single trigram dominating. This visual difference is how automated solvers distinguish good decryptions from bad ones.
Analyzing quadgram patterns in a CTF challenge
Select Quadgram and paste a long ciphertext. Look for repeated quadgrams. In a Vigenere cipher with a short key, repeated quadgrams at regular intervals can reveal the key length (similar to Kasiski examination but at a higher resolution). In a substitution cipher, repeated quadgrams correspond to common English words like 'TION', 'NTHE', 'THAT'.
Comparing observed bigrams to English reference
Paste a paragraph of English text and select Bigram. The reference comparison panel shows that TH is expected at 3.56% and your observed percentage should be close to that. If your text is technical writing with many 'SH' and 'CH' bigrams, those may appear higher than the reference, which is based on general English prose.
Comparison with similar methods
| Method | Complexity | Typical use |
|---|---|---|
| Bigram analysis (this tool) | 26^2 = 676 possible bigrams | Substitution cipher verification, language identification |
| Trigram analysis (this tool) | 26^3 = 17,576 possible trigrams | Stronger language identification, CTF analysis |
| Quadgram analysis (this tool) | 26^4 = 456,976 possible quadgrams | High-resolution pattern detection, Kasiski-style analysis |
| Single-letter frequency | 26 letters | Basic substitution cipher solving |
| Bigram log-probability scoring | Sum of log10(P(bigram)) | Hill climbing and simulated annealing solvers |
Limitations or considerations
N-gram analysis requires sufficient text length to produce meaningful statistics. For bigrams, at least 100 characters are needed for reliable results. For trigrams, 300 or more. For quadgrams, 1000 or more. Short texts produce sparse n-gram distributions that are difficult to interpret.
The reference data for bigrams and trigrams is based on general English prose. Specialized texts (medical, legal, code) will have different n-gram distributions. For example, code samples will have high frequencies of 'IN' (from 'int', 'string'), 'OR' (from 'for'), and 'TH' (from 'this', 'that'), but also unusual bigrams like 'PR' (from 'print', 'printf').
The tool does not compute n-gram log-probabilities for scoring. For automated cipher solving with n-gram scoring, use the Hill Climbing Cipher Solver, which uses a built-in bigram log-probability table.
Quadgram mode does not include reference data. The standard English quadgram frequency table is large (456,976 entries) and is typically used as a log-probability lookup table in automated solvers rather than displayed visually.
Frequently asked questions
What is an n-gram and why is it useful in cryptanalysis?
An n-gram is a sequence of N consecutive letters. In English, certain n-grams appear far more often than random chance would predict: 'TH' is the most common bigram, 'THE' the most common trigram. By comparing a text's n-gram frequencies to expected English frequencies, you can determine whether a candidate decryption is likely correct.
How many characters do I need for reliable n-gram analysis?
For bigrams, at least 100 characters. For trigrams, 300 or more. For quadgrams, 1000 or more. Short texts produce sparse distributions where statistical noise dominates. The longer the text, the more reliable the n-gram frequencies.
Why are bigrams better than single-letter frequencies for cryptanalysis?
Single-letter frequencies only tell you about individual letter counts. Bigrams capture letter-pair patterns that are more distinctive of natural language. For example, both 'TH' and 'HT' contain T and H, but 'TH' is far more common in English. This additional structure makes bigram-based scoring more accurate for automated solvers.
How are n-gram log-probabilities used in hill climbing?
The Hill Climbing Cipher Solver scores candidate decryptions by summing the log-probabilities of all bigrams in the text. Common bigrams like TH and HE have high log-probabilities (less negative), while rare bigrams like QZ have low log-probabilities (very negative). A higher total score means the text is more English-like, guiding the solver toward the correct key.
Can I use this tool for languages other than English?
The tool counts n-grams for any text, but the reference comparison data is for English only. For other languages, you can still use the ranked n-gram output to identify patterns, but you will need to provide your own reference data for comparison.
Conclusion
N-gram frequency analysis is a powerful technique for distinguishing natural language from ciphertext and for scoring candidate decryptions in automated solvers. This tool counts bigrams, trigrams, and quadgrams with visual rankings and English reference comparisons. For single-letter frequency analysis, see the Frequency Analysis. For chi-squared scoring, use the Chi-Squared Calculator. For automated substitution cipher cracking with bigram scoring, see the Hill Climbing Cipher Solver.