Airac5 (SNU) vs. Coverity (Stanford)

This page shows how our Airac5 compares against Coverity.com's SIMPLE_BUFFER checker.

• Airac5 detects all the real errors detected by Coverity's.
• Airac5 detects some real errors that Coverity's misses.
  • Airac5 detects more errors (e.g. cdc-acm.c, line 703 and line 625, 626) than Coversity's.
  • Not all those more errors are false alarms.
• Airac5 avoids some false alarms (e.g. ip6_output.c, line 348) generated by Coverity's.

Airac5 ran on a Pentium4 3.2GHz box with 4GB of main memory running GNU/Linux kernel version 2.6.8. The C files tested here are part of the Linux kernel, version 2.6.4.

Test Results

Following options were used for analyzing these software.

• ud=1 : inline each functions once.
• cf : in case of no main(), Airac5 analyzes each file assuming that the main() procedure consists of calls to the functions in the order of their definitions.

Each columns mean

• #Airac5 Alarms : the number of buffer-access expressions that may overrun.
• #SWAT Alarms: the number of SWAT's buffer-overrun alarms
• #Real Bugs: the number of buffer-access expressions that can really overrun.

	Airac V		SWAT
Program(LOC)	#Time(sec)	#Alarms	#Alarms	#Real Bugs
vmax301.c (246)	161.38	1	1	1
xfrm_user.c (1201)	5974.44	9	1	1
usb_midi.c (2206)	2784.22	4	2	2
atkbd.c (944)	526.75	2	2	2
af_inet.c (1273)	9825.47	25	1	2
eata_pio.c (984)	2319.11	1	1	1
cdc_acm.c (849)	199.15	7	1	2
ip6_output.c (1110)	7857.39	0	1	0
mptbase.c (6158)	14928.75	7	1	1
aty128fb.c (2466)	972.09	2	1	1