He also had the stigma of having been a star player. Perhaps the game had come too easy to him. He was a six-time All-Star for the Yankees, and they retired his jersey number, right alongside those of Babe Ruth, Lou Gehrig and many others.
Maybe Donnie Baseball wouldn't understand just how difficult the game is, and maybe he wouldn't be patient with players who weren't as good as he'd been.
Mattingly's quiet demeanor also didn't do him any favors. Many managerial candidates go out of their way to make themselves available to reporters and let them know how much they know.
All Mattingly did was take care of his business as hitting coach for Joe Torre, pouring himself into the job in a way that spoke volumes about how he'd approach managing the Dodgers.
"He was there for me in my worst times in 2010," Dodgers center fielder Matt Kemp said. "We talked on the phone. We talked at the ballpark. He knows what to say, knows what button to push."
Almost two years later, every question about Mattingly as a Major League manager has been answered and then some. His Dodgers have overcome a wave of injuries to remain in contention, and when Clayton Kershaw finished a weekend three-game sweep of the Giants with a five-hit, 4-0 shutout, they were back tied for first in the National League West at 56-47.
Actually, most of the questions were answered during Mattingly's first season. Down the stretch in 2011, he got the Dodgers to play with purpose and conviction even though the club was out of the race and there was ownership uncertainty swirling from every direction.
With pretty much nothing to play for, the Dodgers still went 45-28 after July 7th and won 10 of their last 11 series. Along the way, Mattingly proved himself with the toughest constituency he'll ever had -- his players.
They praise him for his attention to detail, intensity and organizational skills. But in the end, it's his essential decency that has won them over.
Mattingly is comfortable in his own skin and treats people the way we'd all like to be treated. Fans love to debate lineups and strategy, but managers can hire people to help with that stuff.
If a manager can't relate to his players on a human level, if he can't convince them to play hard every single day and to put team goals above individual ones, he has no chance of succeeding.
"He's the ultimate player's manager," Dodgers catcher A.J. Ellis said. "I never played for anybody quite like him. He opens every Spring Training, and the first thing he promises us is he'll never forget how hard this game was to play. I've had coaches in the past who kind of forget that. It's a tough, tough game."
All Mattingly asks in return is a consistent effort. For that, he has their back. Even when Kemp went down, Mattingly let his players know that they still had enough talent to win.
"Going back to Spring Training, he really put us in the mindset that we were coming to the ballpark every day with the attitude we were going to win," pitcher Chad Billingsley said. "We're not going to give any outs away. We're going to grind you until the last pitch. As you can see, we've had some two-out rallies throughout the season. Those are huge. We grind at-bats out, grind the game out. He reenforces it each day. It gets pounded into our heads that we've got to do everything we can do to win."
There's also a sense of calm around him, a demeanor that communicates confidence and poise. Players get a kick out of him yelling at umpires because that's a side of him they seldom see.
"I think it's important that I'm myself more than anything else," Mattingly said. "For the most part, what you see from me is the way I am. I'm not always calm on the inside, but it's pretty much length-of-the-season type stuff. I know this is a long grind-it-out, day-in-day-out thing. I can't go getting all emotional or going crazy when you lose a few. When you lose extended games, it bothers you more and more. But you still have to understand you have another game tomorrow. If you lose a tough game at midnight and have to be back in eight hours, it's not going to do you any good to get all emotional about it."
For his part, Mattingly pretty much resembles a guy who has been doing it forever. And enjoying it.
"I love it," he said. "It's a challenge. It's the same for me as it is for these guys. That's why I think I like it. It's that battle every day of being ready, being prepared, day-in, day-out, not getting down. It's the same as for a player without any real control. I can't go make a play."
Richard Justice is a columnist for MLB.com. Read his blog, Justice4U. This story was not subject to the approval of Major League Baseball or its clubs.