System health and intrusion monitoring using a hierarchy of constraints

This paper presents a new approach to run-time security monitoring that can detect system abnormalities including attacks, faults, or operational errors. The approach, System Health and Intrusion Monitoring (SHIM), employs a hierarchy of constraints to describe correct operation of a system at various levels of abstraction. The constraints capture static behavior, dynamic behavior, and time-critical behavior of a system. A system in execution will be monitored for violation of the constraints, which may indicate potential security problems in the system. SHIM is based on specification-based intrusion detection, but it attempts to provide a systematic framework for developing the specifications/constraints. SHIM does not detect directly the intrusive actions in an attack, but their manifestations as violations of constraints. In this paper, we describe the constraint model and the methodology for developing the constraints. In addition, we present preliminary results on the constraints developed for host programs and network protocols. By bounding the behavior of various system components at different levels of abstraction, SHIM has a high chance of detecting different types of attacks and their variants.