2017 Competition Results

  • Print

Eagle's Wing is the new AIBIRDS Champion!!! 

Congratulations to Tian Jian Wang from the University of Waterloo and Zazzle Inc., very well done! 

Eagle's Wing participated for the second time this year. In 2016, the agent was called HeartyTian and beat all others in the 2016 Qualification Round. Some modifications to the agent for the 2016 Finals led to an unfortunate drop in performance. But this year the agent showed it's excellent performance again and beat Team IHSEV (the runner up from 2016) in an exciting Grand Final. All bets were on IHSEV who outperformed all others in the Quarter Final and the Semi Final and who took an early lead in the Grand Final. But IHSEV struggled to solve one of the 8 levels and for most of the Grand Final tried again and again to solve that level. Had it succeeded, IHSEV would have won the Grand Final, the gap between the two agents was less than what Eagle's Wing scored on that level. 

This year we had 10 participants including the Who Is Who in AIBIRDS: in addition to the two finalists, we had team DataLab Birds from the Czech Republic, the winner from 2014 and 2015, Team BamBirds from Germany, the defending Champion. Team PlanA+ from Korea who was he first agent to score more than 1 million points in our benchmarks. Team AngryHex from Austria/Italy/Germany/Turkey which has always been one of the strongest teams in the past competitions. Team S-Birds from India, who participated in every competition so far. Team Condor from Argentina, whose previous agent was third in 2015. We also had two completely new agents this year. One is team Vale Fina 007 from Greece and the other one team AngryBNU from China. Both of them use Reinforcement Learning (Q-Learning). While we do not know many details about how Vale Fina 007 works, AngryBNU gave a Symposium talk accompanied by a full length paper describing their agent. It uses Deep Reinforcement Learning, a technique that is taking most of AI by storm and is highly successful in many AI problems and games, including Go and Atari games. We were all excited to see how they would perform and had high expectations. Both BamBirds and DataLab Birds were newcomers when they won, so the competition promised to be very exciting -- and it was. 

Our competition started with three Quarter Final matches. We divided the 10 agents into groups of 3, 3, and 4 teams, all of them had 30 minutes to play 8 new Angry Birds levels. The best 4 agents of the Quarter Finals qualified for the Semi Final. Some of the levels we used this year were structurally different from past years in that they required a bit of reasoning in order to achieve a high score. They were still solvable by not-so-smart strategies, but required more birds and thus resulted in fewer points. 

Quarter Final 1 was played by IHSEV, S-Birds and Condor. IHSEV won by a considerable margin with 261,600 points, S-Birds came second with 147,120 points and Condor last with 94,600 points. Because IHSEV is based on advanced simulations, it took them a bit longer to solve every level, but they consistently solved one level after the next. All agents still had the chance to move to the Semi Final as the 4 best ones of all Quarter Finals will qualify. 

Quarter Final 2 was played by Angry Hex, Eagle's Wing and Vale Fina 007. AngryHex won with 242,980 points in front of Eagle's Wing with 175,510 points and Vale Fina 007 with 106,930 points. The overall ranking was now IHSEV, in front of AngryHex, Eagle's Wing, S-Birds, Vale Fina 07 and Condor. The latter two were out of the competition, while the first four still had a chance to qualify. 

Quarter Final 3 contained four heavy weights. The past winners DataLab Birds, the defending Champion BamBirds, high-performer PlanA+ and the Deep Reinforcement Learning Agent AngryBNU. It promised to be very exciting and there was the chance that none of the agents currently in front of the leaderboard would qualify for the Semi Final. But it took a very unexpected turn. BamBirds got stuck in a level and it took a while for it to recover and to move on, but ended up with only 89,830 points. DataLab showed lots of nerves and missed some easy shots, scoring only 97,100 points overall. AngryBNU didn't even show up on the leaderboard, they didn't solve a single level and ended up with 0 points. Only PlanA+ showed some decent performance and moved in front of S-Birds towards the end of the 30 minutes with 172,410 points, qualifying for the Semi Final as the only team of Quarter Final 3. The audience was speechless, we all couldn't believe it.  

The Semi Final was very exciting and we had very frequent changes on the leaderboard. The agents played 8 new Angry Birds levels for 30 minutes. All four agents, IHSEV, AngryHex, Eagle's Wing and PlanA+ did very well. At around halftime IHSEV and Eagle's Wing moved ahead of the others and increased their lead, making it harder and harder for the others to catch up. IHSEV took a well deserved victory with Eagle Birds second, with quite a big margin ahead or third placed AngryHex. Because of their consistent and impressive performance, everyone in the audience was convinced that IHSEV would also win the Grand Final.

The Grand Final was equally exciting. Again the agents played 8 new Angry Birds levels for 30 minutes. Whenever one of the two finalists solved a level the leadership changed. But one level that Eagle's Wing solved easily, IHSEV couldn't solve. None of its many attempts to solve the level even came close to solving it. It seemed to be a level where a simulation approach completely fails. Eagle's Wing increased its lead more and more by improving on already solved levels, but was always still within reach, had IHSEV solved the one level. It remained exciting until the very end when the 30 minutes were up. Eagle's Wing was the well deserved winner. 

According to Tian Jian Wang, the sole developer of Eagle's Wing, who is a recent graduate from the University of Waterloo in Canada and who now works for Zazzle Inc, the agent does "a multiple strategy affordance based structural analysis that gives multiple decisions and a manually tuned utility to decide between them. Particular attention is paid to buildings, with unique strategies for different types and shapes of buildings. The utility function is learned through the machine learning method xgboost at first using thousands of shots in the first 42 levels in Poached Eggs, and then rewritten in normal programming for much simpler (i.e., tunable) and faster calculation." It is very impressive and encouraging to see that a single student can develop the winning agent when most other teams consist of large groups of people.

Congratuiations to our three best agents in 2017: Eagle's Wing, IHSEV and AngryHex who share the $1,000 prize money sponsored by IJCAI.  

 

We hope you enjoyed the 2017 AIBIRDS competition and are already looking forward to AIBIRDS 2018 which will be held as part of IJCAI 2018 in Stockholm, Sweden (July 13-19, 2018), less than 400km from the home of the Angry Birds! We hope to see many old and new teams there and new approaches tested out. As we saw this year, even a single student can build a winning agent, so everyone is encouraged to participate. We also hope the Deep Learning community will take up our challenge so that we can see some better performing Deep Reinforcement Learning agents in 2018. 

 

 

Grand Final     Quarter Final 1  
         
1. Eagle's Wing 355,700   1. IHSEV 261,600
2. IHSEV 275,110   2. S-Birds 147,120
      3. Condor 94,600
         
Semi Final        
      Quarter Final 2  
1. IHSEV 415,890      
2. Eagle's Wing 350,900   1. AngryHex 242,980
3. AngryHex 238,040   2. Eagle's Wing 175,510
4. PlanA+ 225,780   3. Vale Fina 007 106,930
         
         
Quarter Final Ranking   Quarter Final 3  
         
1. IHSEV 261,600   1. PlanA+ 172,410
2. AngryHex 242,980   2. DataLab Birds 97,100
3. Eagle's Wing 175,510   3. BamBirds 89,830
4. PlanA+ 172,410   4. AngryBNU 0
5. S-Birds 147,120      
6. Vale Fina 007 106,930      
7. DataLab Birds 97,100      
8. Condor 94,600      
9. BamBirds 89,830      
10. AngryBNU 0