Augmenting branch predictor to secure program execution

Yixin Shi, Gyungho Lee

Research output: Chapter in Book/Report/Conference proceedingConference contribution

14 Citations (Scopus)


Although there are various ways to exploit software vulnerabilities for malicious attacks, the attacks always result in unexpected behavior in program execution, deviating from what the programmer/user intends to do. Program execution blindly follows the execution path specified by control flow transfer instructions with the targets generated at run-time without any validation. An enhancement is therefore proposed to secure program execution by introducing a validation mechanism over control flow transfer instructions at micro-architecture level. The proposed scheme, as a behavior-based protection, treats a triplet of the indirect branch's location, its target address, and the execution path preceding it as a behavior signature of program execution and validates it at run-time. The first two pieces of information can prevent an adversary from overwriting control data and introducing foreign code or impossible targets to redirect an indirect branch. The last one is necessary to defeat the attacks that use a legitimate target but follow an unintended execution path. Interestingly, the branch predictor is found to contain the signature information already and doing a portion of the validation when resolving the branch, thus greatly reducing the validation frequency. An enhancement of branch target buffer (BTB) entry together with a signature table implemented in the form of a Bloom filter in hardware is proposed to incorporate the validation into the processor's pipeline, providing a new defense in the processor architecture to secure program execution.

Original languageEnglish
Title of host publicationProceedings - 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2007
Number of pages10
Publication statusPublished - 2007
Externally publishedYes
Event37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2007 - Edinburgh, United Kingdom
Duration: 2007 Jun 252007 Jun 28

Publication series

NameProceedings of the International Conference on Dependable Systems and Networks


Other37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2007
Country/TerritoryUnited Kingdom


  • Bloom filter
  • Branch predictor
  • Control flow validation
  • Indirect branch
  • Software protection

ASJC Scopus subject areas

  • Software
  • Hardware and Architecture
  • Computer Networks and Communications


Dive into the research topics of 'Augmenting branch predictor to secure program execution'. Together they form a unique fingerprint.

Cite this