I was nervous about going to this show. Mark Rothko, after all, is an icon of Modernism. If there is such a thing as a cult artist, then he is probably it. Important people make pilgrimages to his chapel in Houston. They come to pay homage to his murals at the Tate. And they stand in grave silence before his empty expanses with looks on their faces that bespeak lofty thoughts.

What if nothing happened when I paid my own visit to a new show of his work at Tate Modern? What if I found that I didn't feel anything? The catalogue certainly wasn't promising. I flipped my way past a series of rectangles showing not very much, followed by a whole lot more rectangles showing nothing at all. And though this is hardly surprising, for Rothko is always betrayed by reproduction - how can a meagre little A4 image capture the vast mystic feelings his canvases set out to convey? - the austerity of the work that he did during the last decade of his life, on which this show exclusively focuses, looked pretty daunting.

There's no doubt about it, Rothko can be a challenge. It is not that his works belong to the “my child could do that” school because your child couldn't do it - he couldn't even reach. Rather, Rothko evokes all that could be criticised as most pretentious, most clannish, most pseudish about his spectators. They stand there gravely perusing something that to the outsider probably looks more like a patch of half-stripped wallpaper than a picture and then declare themselves profoundly moved. And many outsiders will start to wonder if they are being duped, if this Modernist emperor actually has no clothes on and his fans are just the blind followers of some aesthetic faith.

Times Archive, 1961: Mark Rothko, painter of walls of light

'The progression of a painter's work will be towards clarity; towards the elmination of all obstacles'

• Obituary: Mark Rothko

Related Links

• How Rothko became an Anglophile

• Feature: The Rothko files

• Rothko sells for record $73 million

Then a suspicion that you might be missing something makes you feel a bit worried. suddenly it's like being back at school, watching the child at the next desk busily scribbling their exam answer while you are still sucking your pencil and wondering where to start.

But forget all those pre-show nerves and book your ticket. I wouldn't dither in the opening gallery to look at architectural maquettes that discuss schemes for hanging. I would double back later to visit the side-chapel in which Four Darks in Red has been displayed. Rather, head straight for the awe-stirring experience at the heart of this show. A single vast gallery is the focal point. Here, unfurling all around you in a cloistral gloom, is a series of 15 of Rothko's great Seagram murals. These are the canvases at the heart of his myth.

You probably know the story. It ended on a cold February morning in 1970 when the artist's body was found lying on the floor of his huge Manhattan studio, amid a congealing colour field of his own spilt blood. He had swallowed a massive dose of barbiturates, apparently, and slicing deep into his inner arm at the elbow had taken his own life. He was 66.

On the same day, in London, the Tate (after years of discussion) finally took delivery of a series of enormous murals which, more than ten years earlier, Rothko had created as part of his famously uncompleted commission for the walls of a swanky new restaurant in Mies van der Rohe's glittering Seagram skyscraper in New York.

Rothko had found himself outmoded in the years that had passed between making these murals and presenting them to the Tate. Abstract Expressionism was seen as passé by Pop's fast-paced world. A new generation, ignorant of the European humanist tradition that had nurtured this difficult Russian immigrant, dismissed his vast canvases as “apocalyptic wallpaper”. His mission was over, ostensibly - though not to him.

But with his death, his work suddenly came back into the spotlight. Rothko was no longer a cranky old legacy of a bygone era; he was the curator of a mystery that unfurled before reverential eyes. This latest Tate show returns us to this moment of revelation but, instead of suggesting that we go back over the years that led up to it, it suggests that we look at Rothko's comparatively neglected late works in its light. So you can forget those lovely fields of luminous colour “breathed” on to the canvas; those expanses of glowing yellows and oranges and reds. This show focuses on Rothko's last lonely decade. It could hardly be called decorative. The brightness has faded from his vision. The altarpiece has been folded up. We are looking at the Lenten grisaille of a man whose life was disintegrating amid broken relationships and depression and drugs.

To stand amid the Seagram murals - eight of the Tate's massive canvases have, for the first time, been reunited with nine others of similar dimensions - which unspool like a frieze around the space, is to stand in a sombre, almost sepulchral world. Rothko said that he wanted his works to create their own “place”. And they do. This gallery is pervaded by a quiet, almost lowering mood. To me, it feels like stepping into a cathedral. It has a melancholy, almost sacramental, magnificence which comes partly from the scale of the installation and partly from the way that the colours almost open like windows, then close like veils drawn across a void. The eye slowly loses itself in its own thoughts.

For the rest of the decade - as you will see as you follow the ramifying course of this exhibition - Rothko continued to explore the possibilities of the series. “If a thing is worth doing once, it is worth doing over and over again,” he said. The spectator watches him paring down his vision, stripping his aesthetic back to its basics. It feels almost as if he is dismantling his previous canvases, bringing everything back to black and white, as he reaches for some essential truth.

The Black-Form paintings of 1964, for instance, are abruptly striking. Four dark oblongs surround you on the four walls of the gallery. Rothko wants to return to the very beginnings of creation, perhaps - to the darkness that brooded over the face of the deep, or the first art work which, according to the legend of Pliny the Elder, was painted by a young Corinthian woman who, bidding farewell to her seafaring lover, noticed his shadow cast upon the wall and, reaching into the fire for charcoal, filled it in with black.

To see black merely as an absence is to blind the eye to perception, Rothko seems to suggest. As you gaze into the canvases you see that their surfaces are modulated. Different patterns and intensities and tones emerge. They almost glow with inner light. It's like moving through the night. Sometimes the shadows seem almost to open around you; sometimes they draw so close that your eyelashes scrape against the blackness. Rothko tries to make darkness visible in these works.

Not all of the works are exciting, but they all tell us something. As a series of (rather uninteresting) moonscape-like canvases shows, Rothko is exploring the ways in which the slightest inflections of tone, the subtlest modulations in scale, the smallest changes in proportion, the faintest adjustments of balance, can affect our perception of what we see. His canvases are not messages from their creator. They can't be decoded. Rothko refused explanations. And no wonder. Few works are as open - and as closed - to explanation as these late abstracts. Attempts at description turn to tangles on your tongue. “Silence is so accurate” as the artist said. Perhaps we should see ourselves less as spectators than as participants.

That is why you should not worry about “getting” Rothko. You may at first be aware only of your own body, awkwardly stranded in a difficult place. But this acute consciousness is part of a heightened awareness. By tuning your perceptual processes they help you to see what lies beyond the surface world.

You don't necessarily need Rothko to show you this. But how often in our frenetic modern world do we take the time to consider anything with such careful, scrupulous, meditative intensity? Rothko creates the occasion. His works are a bit like the visual equivalents of Zen koans.

And what if you feel nothing in front of his canvases? Well, that's not nothing because to feel nothing is one of the strongest feelings you can have. Rothko is at Tate Modern (020-7887 8888, www.tate.org.uk) from Friday until Feb 1

Editor's choice

• Guns N’Roses: Chinese Democracy
• The 50 Biggest Movies of 2009
• Quantum of Solace