Defense against a
SYN attack is typically done through the use of
SYN cookies.
Steve Gibson proposal is a
flawed implementation, independently developed (without knowledge of the existence of SYN cookies
*). It demonstrates nicely why
security protocols should be peer reviewed.
The flaw is Gibson's implementation is that the server ISN is being set to Encrypt{client IP}+client ISN. Since the "secret" here is static, a hostile attacker needs only sniff a server SYN-ACK once to then be able to spoof the third step in the handshake.
In essence, GENESIS makes it impossible to fill up the SYN queue, by instead allowing a hostile attacker to freely fill up the connection pool. Oops. Demonstration:
Alice send SYN, with IP A, CISN N to Bob.
Bob computes {A}+N, sends SYN-ACK with SISN {A}+N, acknowledging CISN N. Bob forgets the connection.
Alice sends CISN N+1, ACK SISN {A}+N.
Bob checks that SISN-(CISN-1), decrypted, is A. If so, Bob assumes a three-way handshake exists.
Lucy detects either Alice's Akc, or Bob's SYN-ACK. While Lucy may not be able to break the key used to encrypt IPs, she can easily determine the value of {A}. Lucy now creates a random packet generator, which, for random value R, sends to Bob an ACK with the following parameters:
Source IP: A
CISN: R+1
SISN: {A}+R
Bob's only choice is to block Alice's IP, thus effectively blocking their communication - this is Steve's solution to this weakness.
*which is pretty impressive, since he claims to be a security researcher.