| I was first invited
to judge for the 2012 Festival. It was rather a surprise to be
given this honour, as my own highest awards have been 4-star
awards. The Competition Manager, David Newman, explained to me
that he was keen to have active film makers amongst the judges
and that the fact that I make a wide range of different sorts of
films would mean that I had some understanding of the problems
faced by film makers.
Since that first year, the system has changed slightly. Here is how it works now. In January David Newman sends out a lengthy briefing letter explaining the assessment system to the judges. The briefing pack includes details of the criteria which the judges are asked to use when awarding star ratings for the films.
The pack also includes a selection of films from the previous year, with their anonymised appraisals. Judges are invited to watch these films, refer to the criteria, make their own assessment and then see what award the original panel gave the film and read the judges' comments.
The Judging Weekend
The judging takes place over a weekend in February at a country hotel near Durham. We gather before supper on the Friday. After the meal the first assessment session takes place in a large room, with all the judges present: 15 First Round judges and three Final Round judges. The Competition Manager, David Newman, shows six films which he thinks more-or-less cover each of the 5 'star' categories. A sheet is handed out showing which three judges will judge that film. It is a different panel of three for each film.
Setting the Standard
David Newman, the IAC and the judges all do everything they can to make the system as fair as possible. During the Friday evening, we are encouraged to give our opinions about all of the films. In my first year, I found this incredibly helpful. I was not used to having such discussions and hearing other people's analysis and opinions of films. At my own club there is relatively little criticism of each other's films. The main opportunity to hear any sort of analysis of a film had been listening to judge's comments at club and regional competitions.
At the end of the discussion on each film, the three panellists agree an award.
The idea is that by the end of the evening all the judges will have in their minds not only the criteria, but also some examples of films at each star level both from the Friday night and from the previous year. When they are faced with difficult decisions over the weekend, they can refer back to these films and use them as a standard.
The rest of the weekend is broken up into 90-minute sessions with breaks for tea or meals in between. The judging panels are changed for every session. (You join two different people for each session.) Judges have a sheet to complete after watching the film. The purpose of this is partly to assist the person who will be writing the appraisal, and partly to give all the panellists the opportunity to consider the film before the discussion begins. They are encouraged to write down their own preliminary view of what the award should be given before the discussion so as to avoid the strongest personality dominating the judging.
Then there is a discussion.
Sometimes the judges are all in agreement about the merits of the film and how many stars it should have. But sometimes there is a slight disagreement. On occasion they are miles apart. For example I recall an instance when I thought a film should have 5-stars, a second judge thought it should have 4-stars and the third judge thought 1- or 2-stars. After discussion, the majority agreed to 4-stars, and so that was the award. But my experience last year was that most of the time the judges were close in their opinions, and it was the difference between say 3-stars and 4, or between 4 and 5 which might arise. Usually there was complete agreement.
The system is not perfect. One of the difficulties is that there is a shortage of time. We only have about six or seven minutes between each film. That means that the note taking, discussion and decision about the star rating all have to be packed into that time. Very difficult. This can sometimes result in the final decision being taken uncomfortably quickly.
Some judges complain that despite all efforts, the 'one with the loudest voice' is too influential.
Some judges complain that the criteria are either not clear enough or are too rigid or are wrong in the sense that they don't accurately describe what those judges think should be the standards for one or more star levels.
Possibly the biggest difficulty is that we are attempting the impossible. Films are works of art. Judging them is inevitably subjective to some degree. No matter how much we try to put aside our personal preferences, I cannot conceive of a system which is purely objective.
The majority of films which are awarded five stars achieve that level unanimously by the panel of First Round judges watching that film. In that case the Final Round judges do not have any power to downgrade the assessment. But if the First Round judges cannot agree, and only two of them say a film should have 5-stars, it is marked as a 'majority' vote and when the Final Round judges watch it, they in effect re-judge it. If they feel that it does not meet the criteria for 5-stars, it is awarded 4-stars.
This is one of the controversial aspects of the system. Some people argue that there is no reason why the Final Round judges should be allowed to overrule the judgment of the First Round judges. A First Round panel's decision being overruled can feel as if their opinion is being disregarded. The opposing view is that the Final Round judges have seen many other 5-star films and will have more of a sense of whether any particular film is on a par with the others.
Being a Final Round Judge
This year David Newman invited me to be a Final Round Judge. I was surprised that he should think I was 'good enough' for that role. He pointed out that he knew my judging standard from the quality of my written appraisals of films as a BIAFF First Round judge for the previous two years, and that he'd also observed me when I joined in the judging panel's discussions for a regional competition for which he was a member of the judging panel.
He explained that the other two Final Round judges had some links with professional film making. Terence Patrick is a very experienced former film editor and TV director with BBC Television News, and Romy van Krieken is a professional film critic. David wanted a plain, ordinary, active amateur film maker to fill the third slot. When I heard who I would have as my companions on the Final Round Panel, I couldn't resist the invitation.
We joined the First Round judges for the first two sessions on the Saturday, and after that we were taken to a room with a big screen and had the privilege of watching all the 5 Star films, any films which had been put forward by the First Round judges for special awards, and some top junior films.
There was no time pressure. We would watch a film and make notes like the other judges, but we could then indulge in a discussion at our own pace. And with fellow judges like these, the discussions were riveting.
Only once we'd watched all the 5-star films did we have to make decisions. The system is that the best 8 films are given Diamond awards. To my delight we all agreed which films should win Diamond awards. No one argued for anything different from the eight which were eventually given those prizes.
The other awards did require more discussion. Each time we would put forward our ideas about the contenders and our thoughts on which should win that particular award and why. We would continue discussion until there was a consensus. We did the best we could to argue our cases and to make sure that we gave full consideration to each other's arguments and that in the end we were all happy with every decision.
The Daily Mail Challenge Trophy
The most difficult decision was which film should win the Daily Mail Trophy. It was most interesting to be part of that process. The top films are really very, very good this year. They all have tremendous strengths. Initially we were not agreed on the winner. My recollection is that we each had a different favourite. Each tried to explain why a particular film deserved the prize and also tried to listen to arguments as to why another film had a greater claim on the big prize. It is really extraordinarily difficult to be faced with two or more films which you thoroughly admire and yet you must award one the top prize.
After much discussion we were all agreed on the final result. We knew it would be controversial, but we felt comfortable with the decision.
The End of the Judging weekend
The judging weekend is exhausting.
Each judge has to be alert for many hours. On Saturday we start at 9.00am and go on until 10.00pm and on Sunday most judges work from 9.00am until 3pm. A few First Round judges and the Final Round judges still have work to do after that, sometimes until late on Sunday night.
All the time, whether I've been a First Round judge or a Final Round judge, I've been conscious of the huge responsibility. Every film has been lovingly made. Every entrant is hoping to do well. Any mistake or unfairness could be very upsetting for a film maker. But despite the pressures, I believe that most judges enjoy it. I know I do. It is wonderful to see so many good films and to be part of so many interesting discussions. The unpleasant task is classifying the films into star categories and knowing that very many people will be disappointed.
Judges' Written Comments
Part of the point of BIAFF - and IAC in general - is to have an educational role. Every film receives written comments from a First Round judge, and the Five Star Films and the top prize winners also receive a comment from a Final Round judge.
The BIAFF briefing letter from David Newman, that I mentioned earlier, gives guidance. Examples of critiques are provided with the films sent out prior to the judging. I know that as an entrant I take the judge's comments seriously and very much hope that the judge will show that he or she has really thought about my film and noticed the good points and the weaknesses. I hope the comments will give me some clues as to why I didn't get a higher star rating and what I might do to make a better film next time.
So I take a lot of trouble over writing the appraisals. This task is easier if the other judges on that panel have provided me with full notes. And it is much easier if the entrant has taken the trouble to send in a spare copy of the film. I am sure I'm not alone in taking a very long time to write each appraisal. First Round judges will have somewhere between 10 and 20 appraisals to write.
There's no magic
The BIAFF judges are all individuals from different walks of life, with different degrees of knowledge about films and film making, who are willing to give up a considerable amount of their time to make the event a success. They pay their own travel expenses. After the judging weekend they have to be able and willing to spend time writing appraisals.
BIAFF judges use the best of their abilities to come to their decisions as fairly as they can. But no one should ever feel diminished by a poor result. A different panel of judges might have given a different star rating. I don't believe that any system can ensure that there will be no difference of opinion about the results of a competition.
I felt that my view was confirmed when I did a 'straw poll' of about 15 people after the Sunday Show at BIAFF 2014. The question I asked was "Which film would you have given the top prize to?" No less than 7 different films were mentioned, and looking on the IAC Forum I see that someone there has mentioned the 8th Diamond Award winner.
The team for BIAFF 2014