Why do we write “2” and not “two” in this sentence? “The only even prime number is 2.”

Because whenever numbers are used in an essentially mathematical sense, people expect to see figures, not words. There’s just the “don’t begin a sentence with figures” issue to deal with, but that’s not difficult.