2012 Benchmarks

In order to allow participants to easily compare the performance of their agents with participants of the 2012 challenge, we benchmarked the eight agents that participated in the classic track of the 2012 Angry Birds AI challenge. We ran them on the first 21 levels of the freely available Poached Eggs levels. These are the default levels when using the Chrome version of Angry Birds. Since none of the agents solved any level again during the competition, only the first successful attempt for each of the 21 levels was considered. Each agent had a time limit of 30 minutes per level. If an agent did not solve a level after 30 minutes, it proceeded to the next level. The following table gives an overview of the performance of the eight agents that participated in the classic track:

Level ABC-AI-UNICAL  ABC-IS-UNICAL  CU Dan FEI2 Naive RedBacks Zonino High score  3 stars
1 27550 30490 27090 28890 0 29330 33340 29080 33340 32000
2 52420 34600 52590 42960 60880 43440 44080 52560 60880 60000
3 33460 41070 42880 32880 21070 41350 23480 40800 42880 41000
4 18690 27990 36960 27710 0 28090 38160 19200 38160 28000
5 36280 62780 52740 55560 66050 54350 67630 63230 67630 64000
6 17870 17500 19270 23630 32550 24510 24530 17900 32550 35000
7 22510 20560 31240 26240 0 23620 0 24020 31240 45000
8 47400 40440 47780 47180 34190 48270 47920 46820 48270 50000
9 35600 42500 51570 43710 0 31440 33410 36720 51570 50000
10 41530 43970 51820 37130 0 53390 58530 51240 58530 55000
11 60420 53250 57900 39010 0 57280 42950 57610 60420 54000
12 63440 63440 46530 46800 0 56980 52810 54930 63440 45000
13 36560 36640 21030 36940 0 26530 32480 32950 36940 47000
14 65790 60110 54180 52470 0 46010 0 55640 65790 70000
15 46760 45800 44740 48200 0 42540 33570 39050 48200 41000
16 55110 50730 54370 43980 0 50890 66180 60140 66180 64000
17 41590 48950 48090 34350 0 39090 49140 55150 55150 53000
18 55170 51240 57320 38860 0 40060 38630 47070 57320 48000
19 27590 33340 23040 38720 0 27950 29510 34960 38720 35000
20 40410 44900 49480 35370 0 37580 38900 51440 51440 50000
21 0 0 71740 0 0 57010 54460 0 71740 75000
Total 826150 850300 942360 780590 214740 859710 809710 870510 1080390 1042000 
Time 2h4m 1h48m 1h30m 2h22m 9h03m 1h12m 2h33m 1h25m    
Table 1: Results of the 8 agents participating in the 2012 classic Angry Birds AI challenge for the 21 Poached Eggs levels

It is interesting to see that all participating agents hold at least one high score for the 21 levels. Agent Redback and Agent CU have the most high scores, and Agent CU has by far the highest overall score. The overall time used by the agents is also listed. The Naive Agent is the fastest and Zonino agent the second fastest. The score required in order to get a three star rating is listed in the last column and is taken from the angrybirdsnest website. In 6 of the 21 levels, none of the agents received three stars. 

Please note: The new 2013 Naive Agent behaves differently than the 2012 Naive Agent listed here, as we improved the vision component and the trajectory planning component. But this does not make the new Naive Agent better than the old one. You can check this for yourself.