Automating Assertion-Failure Triage After Design Changes

By Kenneth Roe PhD, Cofounder and Chief Architect, VerifAIX

One of the biggest challenges in design is debugging assertion failures. A large design often involves iterations at the top level where RTL is refactored to meet area, power or performance requirements in addition to introducing new features. Each iteration introduces failures in the regression suite. This regression suite can be quite large with thousands of test cases. A single change may introduce hundreds of failures. This requires quite a bit of work on the part of verification engineers to correct.

The engineers first need to understand the design and to review counterexamples in a waveform viewer to identify the root cause of the failure. The failure can be classified into three categories: 1) an actual bug in the RTL, 2) an error in the assertion or 3) an illegal input condition. From there, an appropriate fix needs to be created. The fix then needs to be verified. Any change may introduce other bugs or cause new assertions to break. This process can be quite tedious.

The verification that needs to be done after each iteration of a design change is a large portion of the cost in developing IP. Automation of the verification process will both reduce the cost and speed up the delivery of new IP.

The VerifAIX Debug Agent automates this process using AI. In order to properly fix bugs, AI tools need to understand the intent of the design, figure out the root cause and make decisions based on this understanding. This is exactly what the Debug Agent does.

As input the Debug Agent will take your design, testbench and any documentation that exists. It will then run your formal tool to identify failing assertions. It then identifies the root cause of each failure and proposes fixes. The fixed version is then run through the formal tool again. If failures persist, then the Debug Agent will reanalyze and propose new fixes. The process iterates until all assertions are passing or a maximum threshold is reached. In addition to fixing failing assertions, the Debug Agent also cleans up compiler warnings which often indicate bugs in the RTL. Vacuous assertions are also cleaned up.

Finally, after the Debug Agent is done, it produces a report detailing each failure and the change made to fix it. An engineer can quickly review the changes. For assertions the Debug Agent was unable to fix, the engineer may be able to give suggestions on how to fix them. For the fixed assertions, the changes can either be approved or the engineer can send back objections and ask the Debug Agent to iterate more to resolve the objections.

In summary, the VerifAIX Debug Agent can automate a lot of the tedious work that needs to be done to triage regressions after a design change. We still have an engineer in the loop to give final approval. However, the process requires much less engineering time, saving money and making turnaround time much faster.