© Wolcott Henry/National Geographic/GettyAn alien's description of the cosmos might teach us a thing or two about the nature of reality.

If we ever establish contact with intelligent aliens living on a planet around a distant star, we would expect some problems communicating with them. As we are many light years away, our signals would take many years to reach them, so there would be no scope for snappy repartee. There could be an IQ gap and the aliens might be built from quite different chemistry.

Yet there would be much common ground too. They would be made of similar atoms to us. They could trace their origins back to the big bang 13.7 billion years ago, and they would share with us the universe's future. However, the surest common culture would be mathematics.

Mathematics has been the language of science for thousands of years, and it is remarkably successful. In a famous essay, the great physicist Eugene Wigner wrote about the "unreasonable effectiveness of mathematics". Most of us resonate with the perplexity expressed by Wigner, and also with Einstein's dictum that "the most incomprehensible thing about the universe is that it is comprehensible". We marvel at the fact that the universe is not anarchic - that atoms obey the same laws in distant galaxies as in the lab. The aliens would, like us, be astonished by the patterns in our shared cosmos and by the effectiveness of mathematics in describing those patterns.

Mathematics can point the way towards new discoveries in physics too. Most famously, British theorist Paul Dirac used pure mathematics to formulate an equation that led to the idea of antimatter several years before the first antiparticle was found in 1932. So will physicists' luck hold as they aim to probe still deeper levels of structure in the cosmos? Are limits set by the intrinsic capacity of our brains? Can computers offer insights, rather than just crunch numbers? These are some of the questions that exercise me.

The precedents are encouraging. The two big breakthroughs in physics in the 20th century owed much to mathematics. The first was the formulation of quantum theory in the 1920s, of which Dirac was one of the great pioneers. The theory tells us that, on the atomic scale, nature is intrinsically fuzzy. Nonetheless, atoms behave in precise mathematical ways when they emit and absorb light, or link together to make molecules.

The other was Einstein's general relativity. More than 200 years earlier, Isaac Newton showed that the force that makes apples fall is the same as the gravity that holds planets in their orbits. Newton's mathematics is good enough to fly rockets into space and steer probes around planets, but Einstein transcended Newton. His general theory of relativity could cope with very high speeds and strong gravity, offering deeper insight into gravity's nature.

Yet despite his deep physical insights, Einstein was not a top-rate mathematician. The language needed for the great conceptual advances of 20th-century physics was already in place and Einstein was lucky that the geometrical concepts he needed had already been developed by German mathematician Bernhard Riemann a century earlier. The cohort of young quantum theorists led by Erwin Schrödinger, Werner Heisenberg and Dirac were similarly fortunate in being able to apply ready-made mathematics.

The 21st-century counterparts of these great figures - those seeking to mesh general relativity and quantum mechanics in a unified theory - are not so lucky. A unified theory is key unfinished business for science today.

The most favoured theory posits that the particles that make up atoms are all made up of tiny loops, or strings, that vibrate in a space with 10 or 11 dimensions. This string theory involves intensely complex mathematics that certainly cannot be found on the shelf, and the challenges it poses have been a stimulus for mathematics. Ed Witten, the acknowledged intellectual leader of string theory, ranks as a world-class mathematician, and several other leading mathematicians have been attracted by the challenge.

String theory is not the only approach to a unified theory, but it is by far the most intensively studied one. This endeavour is surely good for mathematics, but there is controversy about how good it is for physics. Arguments rage over whether string theory is right, whether it will ever engage with experiment, and even whether it is physics at all. There have even been commercially successful books rubbishin g the idea.

To me, criticisms of string theory as an intellectual enterprise seem to be in poor taste. It is presumptuous to second-guess the judgement of people of acknowledged brilliance who choose to devote their research career to it. However, we should be concerned about the undue concentration of talent in one speculative field.

Finding a unified theory would be the completion of a programme that started with Newton. String theory, if correct, would also vindicate the vision of Einstein and the late American physicist John Wheeler that the world is essentially a geometrical structure.

An interesting possibility, which I think should not be dismissed, is that a "true" fundamental theory exists, but that it may just be too hard for human brains to grasp. A fish may be barely aware of the medium in which it lives and swims; certainly it has no intellectual powers to comprehend that water consists of interlinked atoms of hydrogen and oxygen. The microstructure of empty space could, likewise, be far too complex for unaided human brains to grasp.

String theory involves scales a billion billion times smaller than any we can directly probe. At the other extreme, our cosmological theories suggest that the universe is vastly more extensive than the patch we can observe with our telescopes. It may even be infinite. The domain that astronomers call "the universe" - the space, extending more than 10 billion light years around us and containing billions of galaxies, each with billions of stars, billions of planets (and maybe billions of biospheres) - could be an infinitesimal part of the totality.

There is a definite horizon to direct observations: a spherical shell around us, such that no light from beyond it has had time to reach us since the big bang. However, there is nothing physical about this horizon. If you were in the middle of an ocean, it is conceivable that the water ends just beyond your horizon - except that we know it doesn't. Likewise, there are reasons to suspect that our universe - the aftermath of our big bang - extends hugely further than we can see.

That is not all: our big bang may not be the only one. An idea called eternal inflation developed largely by Andrei Linde at Stanford University in Palo Alto, California, envisages big bangs popping off, endlessly, in an ever-expanding substratum. Or there could be other space-times alongside ours - all embedded in a higher-dimensional space. Ours could be but one universe in a multiverse.

Other branches of mathematics then become relevant. We need a rigorous language to describe the number of possible states that a universe could possess and to compare the probability of different configurations. A clearer concept of infinity itself is also required.

The multiverse confronts us with infinities, multiplied by other infinities - perhaps repeatedly. To bring sense to these concepts, we must deploy the mathematics of transfinite numbers, which date back to Georg Cantor in the 19th century. He showed that there was a rigorous way to discuss infinity and that in a well-defined sense there are infinities of different sizes. Without these exotic concepts, cosmologists will not be able to firm up the concept of the multiverse theory and decide, without paradoxes or ambiguities, what is probable and what is improbable within it.

At its deepest level, physical reality may have a geometric intricacy that would be satisfying to any intelligences on Earth or beyond, just as it would have delighted the Pythagoreans. Provided we could understand it, a unified theory that revealed this would be an intellectual triumph. Calling it a "theory of everything", though, is hubristic and misleading as it would offer no help to 99 per cent of scientists. Chemistry and biology are not held up through ignorance of subnuclear physics; still less are they dependent on the deepest structure of space-time. String theory might unify two great scientific frontiers, the very big and the very small, but there is a third frontier - the very complex. That is perhaps the most challenging of all, and it is the frontier on which most scientists work.

There are nonetheless reasons to hope that simple underlying rules might govern some seemingly complex phenomena. This was intimated in 1970 by the mathematician John Conway who invented the "game of life". Conway wanted to devise a game that would start with a simple pattern and use basic rules to evolve it again and again. He began experimenting with the black and white tiles on a Go board and discovered that by adjusting the simple rules of his game, which determine when a tile turns from black to white and vice versa, and the starting patterns, some arrangements produce incredibly complex results seemingly from nowhere. Some patterns can emerge that appear to have a life of their own as they move round the board.

The real world is similar: simple rules allow complex consequences. While Conway only needed a pencil and paper to devise his game, it takes a computer to fully explore the range of complexity inherent in it.

Computer simulations have given science an immense boost. And there is no reason why computers cannot actually make discoveries, albeit in their own distinctive way. IBM's chess-playing computer Deep Blue didn't work out its strategy like a human player. Instead, it took advantage of its computational speed to explore millions of alternative series of moves and responses before deciding an optimum move. This brute force approach overwhelmed a world champion.

The same approach could be put to good use to solve problems that have us so far eluded us. For example, scientists are currently looking for new superconductors that, rather than requiring low temperatures to conduct electricity as they do now, will work at ordinary room temperatures. This search involves a lot of trial and error, because nobody understands exactly what makes the electrical resistance disappear more readily in some materials than in others. Suppose that a machine came up with a recipe for such a superconductor. While it might have succeeded in the same way that Deep Blue defeated Russian chess champion Garry Kasparov, rather than by having a theory or strategy, it would have achieved something that would deserve a Nobel prize.

Simulations using ever more powerful computers will, in a similar way, help scientists to understand processes that we neither study in our laboratories nor observe directly. In my own subject of astronomy, researchers can already create a virtual universe in a computer and carry out experiments in it, such as calculating how stars form and die.

Some day, perhaps, my biological colleagues will be using them to simulate many processes including the chemical complexities within living cells, how combinations of genes encode the intricate chemistry of a cell, and the morphology of limbs and eyes. Perhaps they will be able to simulate the conditions that led to the first life, and even other forms of life that could, in principle, exist.

However there is a long way to go before real machine intelligence is achieved. A powerful computer can be a world chess champion, but not even the most advanced robot can recognise and move the pieces on a real chessboard as adeptly as a five-year-old child.

Maybe in the far future, though, post-human intelligence will develop hypercomputers with the processing power to simulate living things - even entire worlds. Perhaps advanced beings could even simulate a "universe" that goes far beyond mere patterns on a chequer-board and the best movie special effects. Their simulated universe could be as complex as the one we perceive ourselves to be in. This raises a disconcerting thought: perhaps that is what our universe really is.

It is fascinating to speculate whether hyper-intelligent aliens already exist in some remote part of our cosmos. If so, would their brains "package" reality in a mathematical language that would be comprehensible to us or our descendents?