The first success of a new format
David Levy – ICGA President
The new format for the World Computer Chess Championship appears to have been a success insofar as it has avoided the need for this year’s event to be cancelled, which is the only alternative we could have followed. For the WCCC in Leiden we have four very strong entries, as we had hoped when announcing the new format. They are:
- Komodo (the reigning World Computer Chess Champion from our 2016 events)
- Jonny (the 2015 World Computer Chess Champion)
- Deep Shredder (the 2015 World Chess Software Champion)
- Chiron (rated 3207 in the CCRL rating list and 3180 in the SSDF rating list)
The 2017 World Chess Software Championship and the World Computer Speed Chess Championship will both benefit by the inclusion of most or all of these participants.
In the couple of weeks since the announcement of the new format there has been some discussion in an online forum amongst chess programmers, raising some interesting and important questions. The ICGA will be encouraging a full debate amongst chess programmers on these and other matters related to our annual events – that debate will be instigated immediately after the conclusion of the events in Leiden at the beginning of July and will lead, I very much hope, to a consensus as to how to select the challenger for the WCCC match for the years 2019 onwards. In the meantime I would like to use one of the postings from the current online forum, contributed by Harm Geert Muller, as the basis for explaining my own thinking on various of these matters, and to reiterate the key issues which led me to propose the new format.
But first a few items from the ICGA’s (and ICCA’s) history.
As many of you know I have been involved in the organization of computer chess tournaments for rather a long time. I first officiated at the ACM’s North American Computer Chess Championship in 1971, in Chicago, where I first met Professor Ben Mittman and Professor Monty Newborn. The three of us were responsible for organizing most of the computer chess tournaments during the 1970s. On the morning after the final day of the 1973 ACM tournament in Atlanta, the three of us breakfasted together and Ben exclaimed: “Hey. This is a lot of fun. What can we do next?”
My response was to suggest that we emulate FIDE, the World Chess Federation, by organizing regular World Championships. And so it came to pass that the first WCCC took place in Stockholm, at the Birger Jarl Hotel, in the summer of 1974. The British entrepreneur publisher Robert Maxwell donated a medal for the first World Champion, which was won by the Soviet program Kaissa, developed by Mikhail Donskoy, Vladimir Arlazarov and Anatoly Uskov. Two moments from that tournament remain etched permanently in my memory. During the tournament one of the American programmers present was enjoying the World Championship so much that, while on the telephone to his wife from the tournament hall, he was heard to say: “Gee honey. This is great. Better than sex.” (His marriage did not last.) The other notable event for me from Stockholm was the closing dinner, during which President Nixon’s resignation speech was relayed live to all of us in the hotel’s dining room.
Fast forward to 1986. By then the ICCA was almost 20 years old. At our tri-ennial meeting in Cologne, during the WCCC, I was elected ICCA President for the first time. When I came to examine the association’s accounts I discovered that it was technically bankrupt. There was simply insufficient sponsorship money coming in to make up the difference between the ICCA’s expenditure (mainly the printing and mailing of the ICCA Journal and the significant secretarial costs), and the revenue from membership fees. What to do?
Having been involved in FIDE for several years I had seen how its President at that time, Florencio Campomanes, had revived the financial position of FIDE by imposing a kind of tax on countries who hosted FIDE events. The biggest tax was a percentage of the prize fund from the World Championship match. So the ICCA took a leaf from FIDE’s book and decided that, in addition to offering all the local logistical support needed to host a World Computer Chess Championship, the hosting organization would also be required to pay a hosting fee. It was never exactly easy to find hosts for our events but in general we managed it, and the hosting fees, together with a certain amount of sponsorship, have kept the ICCA solvent since then. And when the ICCA morphed into the ICGA at the beginning of this century we had events for some additional games (those in the Computer Olympiad) to act as an incentive for potential hosting organizations.
But in recent years the hosting fees have become much rarer in their frequency, and consequently the ICGA’s reserves have been steadily depleting. As a result of the difficulties experienced in finding other hosts for our events we made a decision that, in years when there is no satisfactory offer from another hosting organization, the ICCA’s events would be held in Leiden. This includes not only the WCCC, the WCSC and the World Computer Chess Speed Championship, but also our annual academic conference on computer games and the Computer Olympiad. Holding our events in Leiden has many advantages, but also one significant disadvantage which was made all too clear to us when, very recently, we conducted a poll amongst chess programmers. As explained in my announcement of the new WCCC format, a significant proportion of chess programmers who might otherwise wish to participate in our events are discouraged by having to come to the same location every year, rather than, as in the “good old days”, travelling to many different countries and thereby making something of a vacation out of their trip.
Another factor in deterring would-be participants has been the cost of travel to our events. So two weeks ago Professor Jaap van den Herik and I were faced with a very difficult situation. We had almost no entries for any of the chess tournaments in Leiden. The obvious conclusion was that we should cancel the chess events, but we both felt that it would be much better, if we could find a way, to perpetuate chess in 2017 and thereafter, despite the cost for the participants of travel to Leiden (or wherever).
Jaap and I agreed that money would be the only way to save the event for this year and to keep it going for the future. The ICGA does not have sufficient reserves to make it prudent to fund the travelling expenses of several programming teams, and in any case we were, at the time of this discussion, only one month away from the start date (July 1st). So we decided to take another leaf out of FIDE’s book and change the format – organizing the WCCC as a match, to be played between the defending World Champion and a challenger to be determined by some sort of tournament, akin to FIDE’s “Candidates” tournament.
Clearly there would not be sufficient time to organize a Candidates tournament for a 2017 WCCC match, so we hit upon a plan to make this year’s WCCC as strong as possible, within the constraints of the ICGA’s limited financial resources. We would attempt to encourage three of the world’s strongest programs to participate in a 4-player tournament together with the reigning World Champion, Komodo, and each of them would receive 1,000 Euro from the ICGA to defray their travel expenses. If we could not attract three other strong programs we could hold a 3-player tournament, and if we could only attract one other strong program we could hold a World Championship match and give each of the two participants 2,000 Euro. Whether we had two, three or four participants, the winner would automatically be invited to defend their title next year.
An important question that we had to consider was how to select the challenger for the 2018 WCCC match. Ideally we would like the challenger to be chosen by some sort of tournament, organized in a way that meets the approval of a majority of chess programmers. Clearly there should be a full and frank discussion amongst chess programmers as to how to achieve this, but equally clearly there was insufficient time for the matter to be debated and decided before next month’s event in Leiden. There was also, in my mind, a nagging doubt that perhaps the new format would not, given the shortness of time, have sufficient incentive for the leading programming teams to wish to participate. After all, one could simply miss this year and come back into the fold for the 2018 event. So to kill both of these birds with one stone I made an executive decision to offer the runner-up in the 2017 WCCC the right to be the challenger for the 2018 WCCC match. In this way we not only gave programming teams a greater incentive to apply for a place in this year’s event, but also it allows plenty of time for the debate to take place about the format of the Candidates’ tournament which will pick the challenger for the 2019 WCCC match.
So that is how we have come to where we stand today. So much for the past history. Now let me turn to the questions raised by Harm Geert Muller. I reproduce his posting here, with my comments interspersed:
I think that in order to have any meaningful discussion on how to conduct a WCCC, we should first decide what goal we have in conducting such a championship at all. Is it supposed to be:
1) for the benefit of the participants, to have them interact and exchange ideas
2) to amuse a world-wide audience of computer-chess afficionados
3) to generate an income for ICGA to help organising the conference
It seems clear that (2) has never been much of a concern; in terms of presentation to the outside world WCCC has always been one of the most appalingly poor events in computer chess. In the not-too-distant past people would learn of what had happened only after the event had concluded. Broadcasting of engine thinking (PVs and scores) has to my knowledge never been attempted, while in watching engine-engine games this is all the fun. This poor publicity is no doubt one of the reasons why outside the small group of regular participants the WCCC is widely regarded as a laughable and pathetic event at best, and a scandalous abomination on average.
I believe we should try to satisfy all three goals, and some additional ones.
We should remember that the chess events are now, and have for some 15 years, been part of the broader activities of the ICGA – the games conference and the Computer Olympiad. So any decision relating to the chess events has some impact on the ICGA’s other activities.
I agree completely with Harm Geert’s criticism of the presentation of our chess events lacking the broadcasting of the analysis and PV of the chess programs, and as part of the discussions which we will be having about the future of the events we need to consider how best to achieve this, given the fact that a program could benefit by ”watching” the displayed analysis and PV of its opponent. Perhaps we should broadcast the games with a delay of a few minutes. I leave it to the community to suggest the best way forward, which we will implement for 2018.
David’s proposals seem to make a very firm step in the direction of (3); I think that reducing WCCC to a two player match firmly squashes virtually all of (1). We should think very hard if this is what we really want. Whether the blatant disregard for a potential world-wide audience will continue can still be decided.
Experience from the world of human chess indicates that the World Championship match is followed, live, by many more chess enthusiasts than follow any grandmaster tournament, even the strongest tournaments and a World Championship tournament. So I believe that the potential of a significantly bigger worldwide audience will be better served by the match format than by a tournament. And even if I am wrong in this belief then surely what I am proposing gives the best of both worlds. We will have some sort of online tournament at the candidates’ stage, wth the display of analysis and PVs, followed by a World Championship match between the tournament winner and the reigning World Champion. A possible tweak to that format would be to hold a two-stage Candidates event – a tournament open to all (subject to eligibility requirements which should be one of the subjects for the programmers’ debate), perhaps followed by a series of knock-out matches amongst the leading 4 or 8 programs, to determine the challenger. Here I should mention in passing the older FIDE system of zonals, interzonals and candidates events. The zonals were regional events, from which the successful players would advance to the interzonal stage – typically an all-play-all tournament with 18 or more players, and then the top 8 would advance to the candidates’ stage which has been held variously as an all-play-all event or a knock-out event.
Of course lack of willingness to participate is a severe problem, that would force abandonment of (1) in practice, even if not in intention. Changing the format as suggested will do very little to improve the situation in this respect. The problem is that new engine developers ‘grow up’ in a general atmosphere where the WCCC is spit on, and participating in it will cause them to get ridiculed in public. This doesn’t inspire a great ambition to participate.
I disagree quite strongly that the suggested change of format as described here will do little to encourage programming teams to participate. It seems to me that the proposed format should make just about everyone happy. If anyone disagrees with this assessment please make your reasons known.
Attending the event is costly in terms of effort and money, and won’t earn them any prestige. While there nowadays are many events that they can participate in at no effort other than submitting their executable, which would earn them a lot of prestige (if they do well). Because these events attract a huge audience of non-programmers, as they are broadcasted over the internet in a professional way, and have virtually all top engines participating. TCEC is a good example of this.
Under the proposed new format the cost element is taken out of the equation for the WCCC match. Each team will receive 2,000 Euro. In addition, I believe that the match format will be more attractive to potential sponsors than is the tournament format. Of course, I might be proved wrong, but it can hardly make matters worse than they are on the sponsorship front. It is essential for the ICGA to generate more revenue in the future than it has been doing in most of the past few years, and we will try our best to achieve that. My reign as ICGA President (and ICCA President before that) is now in its twilight years, and I very much want to hand over to my successor an organisation which is operating on a sound financial footing, so I will try very hard to make this new format work for the benefit of the ICGA, as well as for that of chess programmers and chess enthusiasts.
Apart from the absence of state-of-the-art coverage, it badly hurts WCCC that hardly any of the strongest engines participates. ICGA seems to have always left rule and policy decisions concerning WCCC to the small group of existing participants, and it is understandable that these are not very eager to more actively encourage particpation of strong competitors, especially when commercial interests are at stake. In the long run such a ‘protectionist’ culture will kill the entire event, however, as now seems to be happening.
Availability of the internet has caused the model of engine development to change, and many of the strongest engines are currently open-source projects, where many (sometimes hundreds) of people contributed. The wording of the ICGA rules are not very well suited to provide clarity on how such projects would be treated in the context of WCCC. It is a widespread belief that the rules bar such projects from entering. Probably this is wrong, but no official steps have ever been taken by ICGA to eradicate that belief. Most people are convinced that Stockfish, for example, will never be allowed to participate in WCCC. This will ‘cascade’, because other authors will not take an event that excludes the strongest engines in such an arbitrary way serious, and won’t consider it worth the effort to participate either. There also is the belief that those who have contributed to Stockfish, even if it is only a single line, would be excluded from participation with an engine of their own if Stockfish participates.
I think an overhaul of the WCCC rules, to more unambiguously address the case of large cooperations, is long overdue. The rules should be designed to facilitate participation, rather than discourage it. With regard to the rule that each author could enter one engine only, it could be decided that contributions less than a certain threshold do not count. (Or that he is allowed up to, say, 1.5 engine for all his contributions in total, with no more than 5% to any single additional engine.) The rule that each contributor to a program has to explicitly sign his name on the entry form also causes confusion; people consider this a hopeless endeavor when there are dozens of minor contributors, many of them only known through internet names. It could be made explicit in the rules that public licences such as the GPL, to which those contributors implicitly agreed by contributing, already constitutes sufficient permission to enter in the eyes of the ICGA, and that in the case of minor contributors their exact personalia will not be of any interest, as their contribution is too unlikely to drive up the ‘allowed engine quota’ of a particpant over the limit of 1.5 even if he contributed under an unrecognized pseudonym.
Harm Geert is quite right in saying that the rules for our chess events need to be overhauled. This matter can and should be debated along with the format issues. I shall be asking the ICGA Programmers’ Representative, Mark Lefler, to assist in this debate. In my opinion we need to identify a number of distinct issues and conduct the debate on all of them in parallel, insofar as that is possible. We also need to decide, at the outset, on the “rules” for making the various decisions that are needed.
The GPL explicitly excludes the possibility to put any restrictions on the use of the program to which it applies, so legally anyone has the right enter a GPL’ed program to any tournament he wishes. Now we probably won’t consider it a good thing if any Tom, Dick or Harry could just download a GPL’ed engine to act for it as an operator. It would interfere with goal (1) defined above. OTOH, it is not uncommon that commercial engines just declare a person not involved in the actual programming as a member of their team, and send that person to operate their program. The only difference is official endorsement, which doesn’t make any difference in practice other than as a convenient means to solve conflicts when two different non-contributors would want to enter the same engine. Other methods of resolving such (for now purely hypothetical) problems by ICGA itself are conceivable, though. (Like ‘largest contribution’, with ‘first come, first served’ as a tie breaker.) In general, it is beneficial for events to have more participants, even if some of the participants are not really of the type you aim for. Because they help attract interest and participants of the type you do aim for. Having Stockfish in the tournament, even if operated by a non-contributor, would surely give a big boost to the credibility of the WCCC, which again might convince true programmers to enter their engine too.
These are all valid points but tricky to resolve. For the integrity of the event, as well as that of the ICGA, I feel it is essential to have firm rules regarding participation, so the forthcoming debate needs to consider all of these and similar issues. However, with the new format the problems are, to some extent, diminished. We need to have rules for eligibility into the Candidates tournament, and a somewhat modified version of those rules for the WCCC match. For example, it is impossible to police who is actually operating a particular program in an online tournament, and not trivial to police which program is (or programs are) actually participating as one “player” in an online tournament. An entrant could be sitting with half a dozen top programs running at home, and choosing the moves himself, thereby creating a significantly stronger playing entity. Such cheating could enable an undeserving person from qualifying from an online event and participating in the WCCC match, which is one of the problems that he community needs to consider, debate and rule on.
Finally (for the moment) I would like to comment on some complaints I have heard that that this year’s WCCC event is limited in number to four programs.
I don’t see that any programmers can justifiably complain about the limitation. By the time we made the decision to change the format in this way the original deadline for submitting entries had passed, so how could any program that did not enter by that deadline justifiably complain that they could not enter for the new format event, or that if they did enter they might not be accepted for the new format?
Our decision to change the format was made in good faith, with the aim of perpetuating the World Championships and revitalizing the concept. Thus far we seem to have been successful. Let us see what comes of the forthcoming debate amongst the chess programming community.
June 19th 2017