Not so long ago, Rupert Everett was sitting outside Bar Italia in Frith Street in Soho when the conga line of Hare Krishnas that frequents this part of London appeared from around a corner, jingling their bells like “transcendental morris dancers”. In days gone by, Everett would sometimes recognise a face en route to nirvana: “club world crashes” of his acquaintance often repaired to the Krishnas’ cafeteria, the better, as he puts it, to swap their methamphetamine for cucumber raita. But on this afternoon, he saw no one familiar until, just as the line was about to disappear again, he suddenly caught sight of a producer he’d last seen at the London office of a Hollywood studio.

“Rupert!” exclaimed this man, tambourine in hand. And then: “Hare, hare, hare!” – words that could hardly have been more doleful in context. The fellow in question, a straight white guy of a certain age, had been fired by the studio and, at a complete loss as to what to do next, had duly taken his place in this apricot-tinged, dhoti-wearing human caravan, eager to help broadcast its message of peace, love and saag aloo. Briefly, the two of them talked of a script of Everett’s – it had been rejected by the same studio – and then the line moved off again, until it was only a “swaying smudge” heading towards Chinatown.

Everett’s account of this encounter appears at the beginning of his new book, and it’s full of kindness, even tenderness (“off he ran, backpack bouncing, tiny ankles in large trainers…”). At this point, you realise, his feeling for failure is a writerly gift, throwing a navy shadow over even his funniest and most scabrous lines, with the result that the reader may not know whether to laugh or to cry. But then, as anyone who has read his three memoirs will know, he’s hardly a stranger to disappointment, its cruelty as familiar to the actor and director by now as wig glue and first-night nerves.

He had the idea for his new book in the long moments after the Hare Krishnas passed by. Feeling somewhat on the scrapheap himself, it occurred to him that he might as well turn a few of the ideas he’d pitched down the years – ideas that never got the green light – into short stories. But what is an “American no”? The collection is named after a term, invented by a pal, for the emotionally evasive but nonetheless brutal way Hollywood types have with those who are in search of a commission. In the flesh, they love-bomb you, telling you how “psyched” they are. Once you’re safely elsewhere, however, you’ll never hear from them again – the American no. As titles go, it’s quite perfect. An uncanny bleakness rises from this book, one that brings to mind not only Scott Fitzgerald and Shirley Conran’s Lace, but also the empty, agonising feeling of having been ghosted by a guy you really liked.

It comprises seven stories and a script (the last of which, extraordinarily, is for a TV series based on Proust’s In Search of Lost Time). One has to do with a Paris funeral gone badly wrong, another with a Russian countess, a third with the deathbed confession of a woman who upped and left for India in the 1850s. But the crazed masterpiece of the collection is a long story – almost a novella – called Cuddles and Associates, about a group of struggling actors in 80s Hollywood and what they’ll do (anything!) to survive. If Netflix doesn’t turn it into six episodes, there’s no justice. Think Succession, only with agents and the threat of straight-to-video hanging over everything like LA smog.

In the end, though, I didn’t read The American No for these stories so much as for the bursts of pure Everett, revealed as he introduces each one. Here he is talking to John Schlesinger about what went on in a certain hotel room during the filming of Midnight Cowboy (“I was in heaven”); and here he is fighting for the rights to Graham Greene’s Travels With My Aunt (“each time I inquire of the hard-headed agent I get the same rebuff, while she distributes Greene’s work with abandon for interesting reinterpretations – I’m being polite – like Brighton Rock”).

My love for Everett’s books is hardly a secret (I’m quoted praising him, I notice, on the jacket of this new book and I’m wondering if I can somehow parlay this into an intense new friendship). But still, let me say it again. He is a brilliant writer: opulently gossipy as few are these days, but also truthful, witty, wise and stoical. As soon as I have an opportunity, I’ll be quoting him on why older actors “always” overact, but for now, all I can say is that he’s wasted on those fools who turn him down for parts; who fail to return his calls or to read his scripts. If his talent is unwieldy, there’s also tons of it going spare.