1.4 billion password breach compilation wordlist





Under this gist.github link you can find torrent link to 7z package with a lot of passowrds:

As a comments we can found few remarks  how to clean and optimize this list.

file statistic

> breachcompilation.txt.7z - Size: 4,4 GB (4373756272 bytes)

un pack file

> sudo apt-get install p7zip-full
> 7z x breachcompilation.txt.7z

file statistic

breachcompilation.txt.7z - Size: 4,4 GB (4373756272 bytes)
breachcompilation.txt 	 - Size: 9,6 GB (9640817648 bytes)

count lines

wc -l breachcompilation.txt ==> 1 012 024 699 breachcompilation.txt

sort with the number of cores you have available

LC_ALL=C sort --parallel=8 -u breachcompilation.txt -o breachcompilation.sorted.txt

remove leading spaces

sed -i -e's/^\s*//' breachcompilation.sorted.txt

file statistic

breachcompilation.txt.7z     - Size: 4,4 GB (4373756272 bytes)
breachcompilation.txt 	     - Size: 9,6 GB (9640817648 bytes)
breachcompilation.sorted.txt - Size: 4,1 GB (4073172683 bytes)

count lines

wc -l breachcompilation.txt        ==> 1 012 024 699 breachcompilation.txt
wc -l breachcompilation.sorted.txt ==>   384 153 427 breachcompilation.sorted.txt