WASHINGTON (AP) — A proposal to strengthen IRS enforcement to crack down on tax scofflaws and help fund a nearly $1 trillion bipartisan infrastructure spending bill is officially off the table, Republican Sen. Rob Portman said Sunday.

Portman, who is involved in negotiating the bill, cited “pushback” from fellow Republican lawmakers who dislike the idea of expanding the reach of the IRS, which they have accused over the years of unfairly targeting conservatives. He said another reason the IRS provision was shelved is that Democrats are including a more robust enforcement plan in a separate $3.5 trillion infrastructure bill that they intend to pass through the Senate using special budget rules and without Republican votes.

“That created quite a problem because the general agreement is that this is the bipartisan, negotiated infrastructure package and that we will stick with that,” the Ohio senator said on CNN’s “State of the Union.”

Portman’s announcement that the IRS provision had been removed underscores the difficulty facing the bipartisan group of Republican and Democratic senators in finding mutually agreeable ways to pay for billions of dollars of new spending their White House-backed plan calls for.

Portman said meetings were planned Sunday to discuss alternatives to the IRS provision, which had been estimated to bring in an estimated $100 billion over 10 years. The proposal to go after taxpayers who skip out on income taxes initially had potential bipartisan appeal, but outside groups came forward to lambaste it as a way to enable the IRS to snoop around Americans’ personal finances.

IRS Commissioner Charles Rettig said at a congressional hearing in April that the national gap between federal taxes owed and actually collected is about $1 trillion annually, more than double what official government estimates have previously indicated.

Senate Majority Leader Chuck Schumer is pressuring lawmakers to reach agreement this week on the pair of massive domestic spending measures, signaling Democrats’ desire to push ahead aggressively on President Joe Biden’s multitrillion-dollar agenda.

Schumer, D-N.Y., said last week that he is scheduling a procedural vote for Wednesday to begin debate on the still-evolving bipartisan infrastructure bill. Senators from both parties, bargaining for weeks, have struggled to reach final agreement on a $1 trillion package of highway, water systems and other public works projects.

Portman on Sunday called that an “arbitrary deadline” and one that was premature given that senators haven’t even agreed on details of the “complex” bill yet.

“We want to get it right. It’s not too much to ask that we have time to do that,” Portman said. “We ought not to have an arbitrary deadline forcing this process.”

Schumer said he also wanted Democratic senators to reach agreement among themselves by Wednesday on the specific details of a separate 10-year budget blueprint that envisions $3.5 trillion in spending for climate change, education, an expansion of Medicare and more.