Leonard Cohen’s habit of going into every concert with a prayer on his lips is hardly surprising. He believes that someone took a shot at him long ago at a concert in France that was attended by rowdy Maoists. As he reflected later: “They’re tough critics, the Maoists.”

The world has turned and so have the tables. Last week the 73-year-old Canadian troubadour knocked out his audience and enthralled arena with a dazzling recital of his classic songs. “Magnificent,”critics at London’s O2 summed up the London Evening Standard. Even the monumental eyebrows of Alistair Darling, the chancellor, were reported to be dancing in the sell-out performance.

Discarding his mantles of “poet laureate of pessimism” and “godfather of gloom”, Cohen wore a fedora and grey suit as he ran nimbly on to the stage to rekindle fervour for the love songs of his youth and the witty, sardonic style of later years. His rasping voice, honed by Marlboro cigarettes, and his evident delight in his adoring audience reinforced a recent triumph at the Glastonbury festival.

Cohen’s songs have permeated the cultural fabric of two generations evoking love, desire, suffering and the perils of a world gone mad with an intensity and skill that some claim have been matched only by Bob Dylan.

Dozens of his songs have been covered by artists who revere him, from U2 and the late Johnny Cash to Rufus Wainwright, who declared him to be “the greatest living poet on earth”. Jeff Buckley’s version of Cohen’s signature hit Hallelujah went to No 1 on the US iTunes chart in March.

Cohen’s personal reasons for embarking on his first British tour in 15 years give the occasion added poignancy. Emerging from five years in a Buddhist monastery, his plans for retirement had to be revised in 2005 when he alleged that he had been swindled out of $5m (£2.5m) by Kelley Lynch, his former manager and lover. After 30 years of recording and performing, he was left with just $150,000. In 2006 he was awarded $9m in a civil lawsuit but Cohen may never see the cash. “I had to go to work,” he said. “I have no money left.”

Female fans still swoon, but Cohen has said goodbye to his days as a ladies’ man who romanced Rebecca De Mornay, the actress, Nico, the Velvet Underground vamp, and Janis Joplin, the singer-songwriter. “I had a great appetite for the company of women,” he once said. “And for the sexual expression of friendship.” Yet he considers that his looks may have improved: “That’s what sitting on your ass does to your face.”

Two of his song titles mark him out as a luckless man who fell in love with other people’s women. So Long Marianne is about his affair with Marianne Jensen, with whom he lived on the Greek island of Hydra in the 1960s when her marriage to Axel Jensen, the Norwegian novelist, broke down. His song Suzanne was widely assumed to refer to Suzanne Elrod, the Los Angeles artist with whom he had two children, Adam and Lorca, in the 1970s. In fact, it was Suzanne Verdal, the wife of one of his friends in Montreal, Canada, who brought him tea and oranges in the song.

These days Cohen lives for half the year in a middle-class suburb of Los Angeles, which he shares with his daughter Lorca and several dogs. Savouring the fragrance of incense (“as close as I can get to smoking”), he starts the day drawing and painting – skills that have earned him recognition. His beautiful girlfriend and collaborator, Anjani Thomas, a Hawaii-born singer, lives down the street. Cohen, who has said that “cowardice” and “fear” always prevented him from marrying, spends the rest of the year in his native Montreal.

According to Sharon Robinson, another collaborator who is a backing singer on the tour: “Leonard is a quiet, gracious person. And although everyone thinks he is so serious, he has a fantastic sense of humour.” This is borne out by Christopher Goodwin, who interviewed Cohen in The Sunday Times last year: “We spent most of our time together in stitches with laughter.”

It is another dimension of the doleful singer who used to be mocked as “Laughing Len” and whose lyrics made him seem such an insular and depressing “bedsit bard” that he wondered himself if his CDs should not come with razor blades.

Although Cohen insisted that his depression was overestimated, he admitted that for much of his life he was dogged by “a kind of mist, a kind of distress over everything”, which miraculously evaporated in old age. “I feel tremendously relieved that I’m not worried about my happiness,” he said recently.

As a child he had a lot on his mind. Born in Montreal on September 21, 1934 to a prosperous Jewish family, he was nine when his father Nathan, a clothing store owner of Polish descent, died. His mother Masha was a Russian Jew from Lithuania, from whom he inherited an affinity for song and poetry. Soon news arrived of the fate of other relatives in the Holocaust. His sister, Esther, lives in Montreal.

From a young age he was immersed in the language and rituals of the synagogue: “I was touched as a child by the music and the kind of charged speech.” These provided a context for his early ambition to become a serious writer. He pursued the dream through Herzliah high school, McGill University, where he was president of the debating society, and Columbia University in New York.

Having learnt to play the guitar as a teen-ager, he formed a band called the Buckskin Boys. His writing did not prosper: his first novel was not as well received as his two books of poetry and it was clear that he could not make a living at it.

He sought inspiration elsewhere, staying at the north London maisonette of a friend, Stella Pullman, where “I had certain duties, like getting the coal in and lighting the fire”.

So he turned to rock’n’roll. Although paralysed by stage fright at his first big performance at an antiVietnam concert in New York in 1967, thanks to the singer Judy Collins coaxing him back to the microphone he eventually won over the crowd. Not long afterward, he was signed to Columbia Records and his debut album, Songs of Leonard Cohen, earned him cult status and friendship with the likes of Dylan, Joan Baez and Jimi Hendrix. For most of the 1960s he commuted between Hydra and the Chelsea hotel, Manhattan’s bohemian residence, where he entertained his friends and hung out with Andy Warhol’s circle.

He produced a spate of remarkable albums, including Songs from a Room, which features the often recorded Bird on the Wire, Songs of Love and Hate and New Skin for the Old Ceremony. In 1973 Cohen signed up with the Israeli air force, performing at army bases during the Yom Kippur war.

By the middle of the decade disco was making his songs of tortured love seem dated. The kiss of death came when he collaborated with Phil Spector, the eccentric “wall of sound” producer, on the album Death of a Ladies’ Man, which Cohen considered “grotesque”.

On a personal level, Cohen found the producer to be “very pleasant”. He recalled: “It was when other people were around in the recording studio that he seemed to move into his Mr Hyde period.” At one point Spector put a gun to Cohen’s head, declaring: “Leonard, I love you.” The singer replied: “I hope you really do, Phil.” A comeback album in 1988, I’m Your Man, restored him to a wider audience.

In 1994 Cohen sought refuge from the world in a monastery on Mount Baldy, near Los Angeles. Having followed Zazen Buddhism for three decades, he joined Roshi, his friend and teacher, and became a monk in 1996. It was an arduous regime of early morning awakenings and menial duties. “I was interested in surrendering to that kind of routine,” he said. He was not a good monk: “Much of the time Roshi and I were two buddies drinking.”

In 1999 his depression lifted and Cohen decided to return to the world and begin a new phase of song making – until he realised he was broke. Now honours shower upon him: in March he was inducted into the US Rock and Roll Hall of Fame and last month he became a grand officer of the national order of Quebec.

He has advised young songwriters that if they stick with a composition long enough, it will yield: “You might have to stay with it for years and years.” With Cohen approaching perfection, his fans can only hope that he will continue to follow his own counsel.

Explore Music

• Proms

• Festivals

• CD Reviews

• Live Reviews

Editor's choice

• Are Glasvegas the new Oasis?
• Proms Podcast with Stephen Hough
• Vote for musical perfection