auricio Savarese And Beth Harris, The Associated Press Published Thursday, August 18, 2016 1:09PM EDT RIO DE JANEIRO -- A Brazilian police official told The Associated Press that American swimmer Ryan Lochte fabricated a story about being robbed at gunpoint in Rio de Janeiro. The official, who has direct knowledge of the investigation, spoke on the condition of anonymity Thursday because he was not authorized to speak about an ongoing probe. He said that around 6 a.m. on Sunday, Lochte, along with fellow swimmers Jack Conger, Gunnar Bentz and Jimmy Feigen, stopped at a gas station in Barra da Tijuca, a suburb of Rio where many Olympic venues are located. One of the swimmers tried but failed to open the door of an outside bathroom. A few of the swimmers then pushed on the door and broke it. A security guard appeared and confronted them, the official said. The official says the guard was armed with a pistol, but he never took it out or pointed it at the swimmers. According to the official, the gas station manager then arrived. Using a customer to translate, the manager asked the swimmers to pay for the broken door. After a discussion, they did pay him an unknown amount of money and then left. The official says that swimmers Conger and Bentz, who were pulled off a plane going back to the United States late Wednesday, told police that the robbery story had been fabricated. Lochte first lied about the robbery to his mother, Ileana Lochte, who spoke with reporters, the police official said. That led to news coverage of the incident and prompted police attention. A police news conference was scheduled for Thursday afternoon. Lochte's lawyer, agent and father did not immediately return messages seeking comment from the AP. USOC spokesman Patrick Sandusky said late Thursday morning that Lochte's teammates were scheduling a time and place to meet with authorities. "All are represented by counsel and being appropriately supported by the USOC and the U.S. Consulate in Rio," Sandusky said. The swimmers could face punishment -- probation, suspension, a fine or expulsion -- under USA Swimming's code of conduct, which prohibits dishonesty or fraud. It was not immediately clear if the organization planned to act. Mario Andrada, a spokesman for the Rio organizers, said he was relieved that the story had turned around. "Let's give these kids a break. Sometimes you take actions that you later regret," Andrada said. "Lochte is one of the best swimmers of all-time. They had fun, they made a mistake, it's part of life, life goes on, let's go." The situation escalated Wednesday when Conger and Bentz were pulled from a flight at the Rio de Janeiro airport amid uncertainty over what truly took place during the late-night outing after Olympic swimming wrapped up at the Rio Games. Court officials had called for the four swimmers' passports to be seized, but Lochte already had returned to the United States before authorities could enforce the decision. Representatives from the U.S. consulate arrived at the airport shortly after the swimmers were stopped from leaving the country Wednesday night. Brazilian authorities continued pressing the American swimmers over the ever-changing account of the robbery. Police said the swimmers were unable to provide key details in early interviews and they found little evidence to support the robbery claim. The swimmers said they had been intoxicated and could not remember what type and colour of taxi they rode, where the incident happened or what time the events occurred, police said. The police official said officers grew suspicious when they reviewed security video of the swimmers returning to the athletes village and saw them wearing watches. The office of Judge Keyla Blanc, who ordered the passports seized, said there were discrepancies in the statements by the swimmers. Lochte had said he was with Conger, Bentz and Feigen when they were robbed at gunpoint in a taxi by men with a police badge as they returned to the athletes village from a party several hours after the final Olympic swimming events on Saturday. NBC reported Wednesday night that Lochte backed off some of his earlier claims, saying the taxi wasn't pulled over by men with a badge but rather the athletes were robbed after stopping at a gas station. Lochte also said the assailant pointed a gun at him rather than putting it to his head. Authorities said that after the incident, the swimmers did not call police; officers began investigating after they saw media reports in which Lochte's mother spoke about the incident. Lochte's attorney, Jeff Ostrow, previously said there is no question the robbery occurred. "Why would anybody fabricate anything?" Steve Lochte said. "It's just ridiculous." Lochte told USA Today that he and his teammates didn't initially tell U.S. Olympic officials about the robbery because "we were afraid we'd get in trouble." Steve Bentz, father of Gunnar Bentz, when reached by phone late Wednesday night, said: "I really don't want to say anything," and hung up. The British Olympic team said Thursday one of its athletes also was robbed in Rio. The British Olympic Association was responding to a Guardian report that the unnamed athlete was held up at gunpoint after a night out. The BOA declined to say if the incident was reported to police. While he's medaled often in the Olympic pool, Lochte's accomplishments have long been overshadowed by teammate Michael Phelps -- the most decorated Olympian in history. Lochte won a gold in Rio in a relay race alongside Phelps.