It's primary season in Massachusetts, and Democrats and Republicans are courting the ideological fringe. On the Republican side, Charlie Baker is making his second run for governor and is the strong favorite in the race. But his primary rival Mark Fisher says Baker is too liberal to be the GOP’s standard-bearer — and he’s not alone. For now, Baker is shrugging off the criticism, but it could pose problems down the road.

When Gov. Deval Patrick proposed housing children who’d immigrated illegally from Central America here in Massachusetts, the conservative backlash was fierce — including a rally outside the State House in July. Republican gubernatorial hopeful Mark Fisher was there.

"We are currently supporting over 200,000 illegal immigrants in this state alone," Fisher said to a chorus of boos at the rally. "It costs us two billion dollars … We will fight you tooth and nail if you continue to steal from us!"

But Baker’s Republican rival Charlie Baker wasn’t. According to rally organizer and WRKO talk-show host Jeff Kuhner, that’s a sign of a bigger problem.

"My conservative listeners believe that Charlie Baker fundamentally is a RINO," Kuhner said. "He's a squishy Republican in name only. He’s an establishment Republican; he doesn’t have any core Republican convictions."

Kuhner says that on a slew of issues — illegal immigration, taxes, dysfunction at the Department of Children and Families — Baker is trying to please people on every end of the political spectrum. He thinks that’s the wrong way to campaign.

"When I look at Charlie Baker, I see a man who’s pandering," Kuhner said. "I see a man who wants to be all things to all people. And I think fundamentally it’s going to damage his credibility. The Democrats will not back him, and I believe much of the base is not going to turn out."

It’s a message Fisher is happy to embrace. Almost no one in Massachusetts politics thinks Fisher — a self-described Tea Party member — will beat Baker in September. But Fisher predicts he’ll benefit from conservative discontent.

"We haven’t had a choice in a primary between someone like Charlie and someone like me," Fisher said. "This is the first time in a long time. So those 300,000 voters who didn’t get off their couch and vote for Gomez, they’re going to vote for me."

That’s a reference to Gabriel Gomez — the moderate Republican who lost a 2013 U.S. Senate race to Ed Markey. To say Baker seems unconcerned would be an understatement.

"I’m going to continue to do as I’ve said all along, and that's chase 100 percent of the vote," Baker said. "Remember, my mom's a Democrat, my dad's a Republican, so I grew up listening to both sides at our dinner table every night."

As he talks to voters, Baker says, their concerns transcend ideology.

"The vast majority of people in Massachusetts, more than anything else, they want to be able to find a job if they don’t have one," he said. "They want to be able to keep the job they do have and pay their bills. They want to be able to live in a safe, thriving community where they believe tomorrow’s better than today. And I don’t think those are Democrat or Republican or independent issues."

That’s a message that should serve Baker well in the general election. At that point, Baker’s biggest challenge might be getting out the conservative vote. Rob Eno, editor of the blog Red Mass Group, is a Baker backer. He says Baker’s critics don’t give him enough credit.

"He’s not a conservative," Eno said. "He’s a center-right politician. But I know that Charlie Baker’s for charter schools. I know that Charlie Baker on illegal immigration is someone that believes in legal immigration, and that people who come here illegally have benefits that shouldn’t be afforded them."

But Eno also says that if conservatives are lukewarm on Baker’s candidacy, the candidate himself is partly to blame.

"In 2010, Scott Brown ran on a consistent conservative message," Eno said. "There was a clear distinction between Scott Brown and Martha Coakley. If I was running [Baker's] campaign I would come up with three or four issues where they can make a clear distinction between Martha Coakley, or Steve Grossman, or — probably not Don Berwick, maybe not Don Berwick — and just run on that! Have that be the focus of their campaign!"

Then again, with a Democratic field that’s light on charisma—and a popular Democratic incumbent still in the Corner Office—acting like a moderate Dem might be Baker’s best path to victory. If not, Massachusetts conservatives will be happy to say: We told you so.

Watch the Greater Boston segment: