The 78-year-old singer may have entered his sixth decade in the music business but he has always been concerned that his vocals are "too damn sensitive" and not particularly "manly".
He said: "I worry that it's too damn sensitive. It's too fragile.
"What's a man doing being so ... feminine, so perfect, gilding the lily on all those notes. Is that manly?"
The former Simon & Garfunkel star - who met his former musical partner Paul Simon when they were both at school - also admitted that he never dreamed he could grow up and sing for a living when he was child.
In an interview with Mojo magazine, he said: "I never knew how to take this joy of singing in the alleyways at seven, nine, 12 years old and see it vocationally. I was never able to go there. I was sure that I'd have to ultimately find a profession; I'm going to be a teacher or something.
"To imagine that I could get away with singing, and that I could get royalties for it and be able to eat, raise a family on that money, was way behind my thinking as a kid. I never felt I could get away with singing as a life."
Garfunkel first met Paul Simon when they were just 11 years old and he recalled how his pal made singing much more "interesting" for him.
The 'Bridge over Troubled Water' singer said: "We lived three blocks away from each other in Queens. There was a bunch of us, five per cent of the neighbourhood knew about this subversive music, ad when we tried to sing it, I had a good singing voice, right from an early age, and Paul began to hook up to it, and then I had a harmonist. And then that was a tremendously fun game, what you could do with harmony.
"I knew I could sing, but if Paul is available to tilt the singing in to all these harmonies, it's more interesting. And then he played guitar magnificently, never forget that.
"I was very attracted to how good Paul plays acoustic guitar. He's wonderful - rhythm and subtleties and accents. Charlestons and syncopations fall into place."
