## Abstract

Several recently proposed ciphers, for example Rijndael and Serpent, are built with layers of small S-boxes interconnected by linear key-dependent layers. Their security relies on the fact, that the classical methods of cryptanalysis (e.g. linear or differential attacks) are based on probabilistic characteristics, which makes their security grow exponentially with the number of rounds Nr. In this paper we study the security of such ciphers under an additional hypothesis: the S-box can be described by an overdefined system of algebraic equations (true with probability 1). We show that this is true for both Serpent (due to a small size of S-boxes) and Rijndael (due to unexpected algebraic properties). We study general methods known for solving overdefined systems of equations, such as XL from Eurocrypt’00, and show their inefficiency. Then we introduce a new method called XSL that uses the sparsity of the equations and their specific structure. The XSL attack uses only relations true with probability 1, and thus the security does not have to grow exponentially in the number of rounds. XSL has a parameter P, and from our estimations is seems that P should be a constant or grow very slowly with the number of rounds. The XSL attack would then be polynomial (or subexponential) in Nr, with a huge constant that is double-exponential in the size of the S-box. The exact complexity of such attacks is not known due to the redundant equations. Though the presented version of the XSL attack always gives always more than the exhaustive search for Rijndael, it seems to (marginally) break 256-bit Serpent. We suggest a new criterion for design of S-boxes in block ciphers: they should not be describable by a system of polynomial equations that is too small or too overdefined.

Original language | English |
---|---|

Title of host publication | Advances in Cryptology - ASIACRYPT 2002 |

Subtitle of host publication | 8th International Conference on the Theory and Application of Cryptology and Information Security Queenstown, New Zealand, December 1–5, 2002 Proceedings |

Place of Publication | Berlin, Germany |

Publisher | Springer, Springer Nature |

Pages | 267-287 |

Number of pages | 21 |

Volume | 2501 |

ISBN (Electronic) | 9783540361787 |

ISBN (Print) | 3540001719, 9783540001713 |

DOIs | |

Publication status | Published - 2002 |

Event | 8th International Conference on the Theory and Application of Cryptology and Information Security, ASIACRYPT 2002 - Queenstown, New Zealand Duration: 1 Dec 2002 → 5 Dec 2002 |

### Publication series

Name | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) |
---|---|

Volume | 2501 |

ISSN (Print) | 03029743 |

ISSN (Electronic) | 16113349 |

### Other

Other | 8th International Conference on the Theory and Application of Cryptology and Information Security, ASIACRYPT 2002 |
---|---|

Country/Territory | New Zealand |

City | Queenstown |

Period | 1/12/02 → 5/12/02 |

## Keywords

- AES
- Block ciphers
- Camellia
- Gr¨obner bases
- MQ problem
- Multivariate cryptanalysis
- Multivariate quadratic equations
- Overdefined systems of multivariate equations
- Rijndael
- Serpent
- Sparse multivariate polynomials
- Square
- XL algorithm