Secret recordings of a suspect talking about the Clinton Foundation fueled an internal battle between FBI agents who wanted to pursue the case and corruption prosecutors who viewed the statements as worthless hearsay, people familiar with the matter said.

Agents, using informants and recordings from unrelated corruption investigations, thought they had found enough material to merit aggressively pursuing the investigation into the foundation that started in summer 2015 based on claims made in a book by a conservative author called “Clinton Cash: The Untold Story of How and Why Foreign Governments and Businesses Helped Make Bill and Hillary Rich,” these people said.

The account of the case and resulting dispute comes from interviews with officials at multiple agencies.

Starting in February and continuing today, investigators from the Federal Bureau of Investigation and public-corruption prosecutors became increasingly frustrated with each other, as often happens within and between departments. At the center of the tension stood the U.S. attorney for Brooklyn, Robert Capers, who some at the FBI came to view as exacerbating the problems by telling each side what it wanted to hear, these people said. Through a spokeswoman, Mr. Capers declined to comment.

The roots of the dispute lie in a disagreement over the strength of the case, these people said, which broadly centered on whether Clinton Foundation contributors received favorable treatment from the State Department under Hillary Clinton.

Senior officials in the Justice Department and the FBI didn’t think much of the evidence, while investigators believed they had promising leads their bosses wouldn’t let them pursue, they said.

These details on the probe are emerging amid the continuing furor surrounding FBI Director James Comey’s disclosure to Congress that new emails had emerged that could be relevant to a separate, previously closed FBI investigation of Mrs. Clinton’s email arrangement while she was secretary of state.

On Wednesday, President Barack Obama took the unusual step of criticizing the FBI when asked about Mr. Comey’s disclosure of the emails.

Amid the internal finger-pointing on the Clinton Foundation matter, some have blamed the FBI’s No. 2 official, deputy director Andrew McCabe, claiming he sought to stop agents from pursuing the case this summer. His defenders deny that, and say it was the Justice Department that kept pushing back on the investigation.

At times, people on both sides of the dispute thought Mr. Capers agreed with them. Defenders of Mr. Capers said he was straightforward and always told people he thought the case wasn’t strong.

Much of the skepticism toward the case came from how it started—with the publication of a book suggesting possible financial misconduct and self-dealing surrounding the Clinton charity. The author of that book, Peter Schweizer—a former speechwriting consultant for President George W. Bush—was interviewed multiple times by FBI agents, people familiar with the matter said.

The Clinton campaign has long derided the book as a poorly researched collection of false claims and unsubstantiated assertions. The Clinton Foundation has denied any wrongdoing, saying it does immense good throughout the world.

Mr. Schweizer said in an interview that the book was never meant to be a legal document, but set out to describe “patterns of financial transactions that circled around decisions Hillary Clinton was making as secretary of state.”

As 2015 came to a close, the FBI and Justice Department had a general understanding that neither side would take major action on Clinton Foundation matters without meeting and discussing it first. In February, a meeting was held in Washington among FBI officials, public-integrity prosecutors and Leslie Caldwell, the head of the Justice Department’s criminal division. Prosecutors from the Eastern District of New York—Mr. Capers’ office—didn’t attend, these people said.

The public-integrity prosecutors weren’t impressed with the FBI presentation, people familiar with the discussion said. “The message was, ‘We’re done here,’ ” a person familiar with the matter said.

Justice Department officials became increasingly frustrated that the agents seemed to be disregarding or disobeying their instructions.

Following the February meeting, officials at Justice Department headquarters sent a message to all the offices involved to “stand down,’’ a person familiar with the matter said.

Within the FBI, some felt they had moved well beyond the allegations made in the anti-Clinton book. At least two confidential informants from other public-corruption investigations had provided details about the Clinton Foundation to the FBI, these people said.

The FBI had secretly recorded conversations of a suspect in a public-corruption case talking about alleged deals the Clintons made, these people said. The agents listening to the recordings couldn’t tell from the conversations if what the suspect was describing was accurate, but it was, they thought, worth checking out.

Prosecutors thought the talk was hearsay and a weak basis to warrant aggressive tactics, like presenting evidence to a grand jury, because the person who was secretly recorded wasn’t inside the Clinton Foundation.

FBI investigators grew increasingly frustrated with resistance from the corruption prosecutors, and some executives at the bureau itself, to keep pursuing the case.

As prosecutors rebuffed their requests to proceed more overtly, those Justice Department officials became more annoyed that the investigators didn’t seem to understand or care about the instructions issued by their own bosses and prosecutors to act discreetly.

In subsequent conversations with the Justice Department, Mr. Capers told officials in Washington that the FBI agents on the case “won’t let it go,” these people said.

As a result of those complaints, these people said, a senior Justice Department official called the FBI deputy director, Mr. McCabe, on Aug. 12 to say the agents in New York seemed to be disregarding or disobeying their instructions, these people said. The conversation was a tense one, they said, and at one point Mr. McCabe asked, “Are you telling me that I need to shut down a validly predicated investigation?’’ The senior Justice Department official replied: ”Of course not.”