From turbo codes to the neocortex – Claude Berrou, Télécom Bretagne
Claude Berrou is the co-inventor of turbo codes, a major development in the history of digital communications now widely used in satellites and high-speed networks. An internationally-respected researcher and recipient of numerous prizes, as well as a member of the prestigious Académie des Sciences, since 2011 Professor Berrou has been working on a new challenge: applying his knowledge of information theory to artificial intelligence and the neocortex.
Turbo codes, a major scientific turning point
By October 1996, when an article entitled “NearOptimum Error Correcting Coding and Decoding: Turbo-Codes” appeared in the prestigious journal IEEE Transactions on Communications, the international community had already known for some three years that a previously-unknown team of Brittany-based researchers were in the process of revolutionizing digital communications.
Since American Claude Shannon published his pioneering work in 1948, it had been widely accepted that in a digital communication channel disrupted by a certain amount of noise, as long as the average level of noise does not exceed a certain threshold and the information is encoded with appropriate redundancy, the receiver can identify the original message without errors. But the efficiency of forward correction codes is limited and researchers had been attempting to reach the theoretical limit for decades, before finally writing it off as an impossibility. There was thus a considerable degree of scepticism in the air at ICC ’93 (International Communications Conference) when Claude Berrou and his colleague and co-author Alain Glavieux announced that they had reached the fabled Shannon Limit, even more so since this discovery was not backed up by reams of equations. Yet the leading figures in the field verified the results, and an integrated circuit developed by Berrou and his team proved to even the most sceptical observers that the results they had announced could be accurately measured. This marked the dawn of the turbo code era, with dozens of teams launching themselves into researching this new field. Over four hundred patents have been registered in the last twenty years.
This discovery was made possible by the expertise and experience of Claude Berrou, who modestly describes himself as “average at everything.” Prof. Berrou has always been a passionate advocate of interdisciplinary collaboration, and a researcher whose motto is “small is beautiful”. Born in Penmarc’h (Finistère) in 1951, he first arrived at the electronics department of the newly-founded École Nationale Supérieure des Télécommunications de Bretagne (now Télécom Bretagne) in 1978, at a time when the whole curriculum was still under construction. He began working on the physics of transistors, microwaves, circuit architecture and metrology, studying the use of algorithms in a huge variety of contexts before moving on to tackle the theory of information encoding. He was appointed head of the integrated circuit design laboratory in 1986, and began working with digital communication specialist Alain Glavieux to study encoding techniques in telecommunications. He soon began to wonder why scientists in this field did not seem to apply a principle which was common currency in electronics: feedback, or reintroducing a portion of the output signal at the input level. This “naïve”, newcomer’s question turned out to be a major breakthrough. Claude Berrou’s massive contribution to science has earned him international recognition from his peers, and has seen him cited over 13,000 times in academic publications (placing him among the top ten most referenced French scientists in the IT field). His achievements have also been feted in France, where he was elected to the Académie des Sciences in 2007. He is a member of two of the Académie’s sections: computer science & mechanics and inter-disciplinary scientific applications. He is also hoping to help to establish a new section devoted entirely to information technologies, a field of great importance which is currently under-represented at the Académie.
His work has brought him various other honours and awards: in 2003, Claude Berrou and Alain Glavieux (who passed away in 2004) were awarded the Institute of Electrical and Electronics Engineers’ (IEEE) Hamming Medal. He also picked up the Marconi Prize[1] in 2005, following in the footsteps of the creators of Google in 2004, and the inventor of the World Wide Web itself in 2002. He was also nominated for the European Inventor of the Year award in 2006. He was actively involved in the foundation of the company Turboconcept, and has registered several patents in the field of coding and decoding, and on the extension of the ‘turbo principle’ to other functions within telecommunications systems.
Turbo code theory
The principle behind turbo codes is deceptively simple, based on an idea which will have occurred to anyone who has ever done a crossword. A turbo code is composed of two elementary codes which deliver the vertical and horizontal definitions. To solve the puzzle, we fill in the boxes one by one and the vertical ‘decoding’ information can help confirm or contradict the horizontal information, and vice versa. The outputs (hypotheses) of each decoding system are thus fed back into the other one in order to help with the process.At the ICC93 conference, this idea immediately piqued the interest of a researcher from NASA. The increase in efficiency made possible by this method can cut energy expenditure in half, thus drastically reducing the number of solar panels required for space probes and saving the agency tens of millions of dollars. The world now has between four and five billion turbo decoders, powering our high-speed WiMAX and 3G and 4G networks. Turbo codes were used in the Mars Express mission launched in June 2003. Find out more about turbo codes.
Artificial intelligence, the next step
Claude Berrou is now working on a new challenge, exploring the field of neuroscience. As was the case with information theory in the 1990s, exploration of the human brain currently remains the preserve of a specialized group of neurobiologists. Professor Berrou wants to shed new light on our mental functions by approaching them from the perspective of information theory, and looking at the brain as a form of distributed decoder. As he explains, “these days innovation often arises when different disciplines collide.” He remains deeply committed to breaking down barriers between different scientific domains.
In 2007 Prof. Berrou first observed that the neocortex, that “propagation medium which allows biological processes to skip from one part of the brain to another,” reveals some significant structural similarities with modern decoders. The analogies and dissimilarities which can be drawn have already proved useful to neuroscientists and information scientists. Prof. Berrou subsequently decided to devote his full attention to neurosciences: “It was at this point that I came to a second observation: mental information is robust and durable. Given the constant pressures on the brain, this information must be redundant.” And so Prof. Berrou, a leading expert on the principle of redundancy, joined forces with a brilliant doctoral student by the name of Vincent Gripon. Within the space of a few months they had identified new codes which serve to represent and memorise information, and which may help explain why mental information is robust and durable, thus opening up exciting new perspectives for artificial intelligence. “This goes way beyond turbo codes,” Prof. Berrou declares enthusiastically. Indeed it turns out that these codes can be transposed onto any graph, a discovery which could revolutionise research into networks of artificial neurones.
Two patents were soon registered, and various articles published: the Neucod (neural coding) project was born. The project was submitted to the European Research Council in 2011, receiving funding of € 1.9 million. Of the 290 projects selected (from 2287 applicants), only 31 were based in France, compared to 68 in the UK. “Finally, this new approach was vindicated,” Claude Berrou proudly recalls. The funding enabled him to take on six full-time researchers on five-year contracts, supplementing the team’s existing scientific manpower.
“The aim is to produce thinking machines, capable of learning millions of pieces of information and cross-referencing them to produce new information,” the professor explains. This artificial neocortex, or ‘cognitor’, could then evolve from ‘infancy’ to become a real ‘intellectual auxiliary’ by gathering information from the internet, which the researchers will then put into context. A firm believer in the theory of technological singularity, Claude Berrou is certain that “by 2050 we will have machines which are more intelligent than humans.”
“The work we are doing could lead to major, long-term partnerships with a number of big industrial companies,” he adds. As the Neucod team continues to form new partnerships with other research teams, the turbo code adventure could be about to repeat itself, with Claude Berrou once again at the forefront of a major scientific breakthrough, this time in the field of artificial intelligence.
Further reading: A theory of mental information processing
Passionately committed to bringing scientific knowledge to the widest possible audience, in september 2012 Claude Berrou’s latest work was published by Odile Jacob Sciences. This book is an accessible introduction to his latest research interest , co-authored by Vincent Gripon and entitled “Petite mathématique du cerveau. Une théorie de l’information mentale“.
Claude Berrou is the co-inventor of turbo codes, a major development in the history of digital communications now widely used in satellites and high-speed networks. An internationally-respected researcher and recipient of numerous prizes, as well as a member of the prestigious Académie des Sciences, since 2011 Professor Berrou has been working on a new challenge: applying his knowledge of information theory to artificial intelligence and the neocortex.
