The man who beat Joe Frazier — and lost everything because of it

On May 20, 1964, a 295-pound boxer beat 20-year-old Joe Frazier in the final of the U.S. Olympic heavyweight trials at the New York World's Fair grounds in Flushing. He had beaten Frazier before. He earned the ticket to Tokyo fair and square.

He never went. And the man he beat became one of the greatest heavyweights who ever lived.

Buster Mathis stood 6'3" and weighed 295 pounds — about 100 pounds heavier than Frazier — but calling him slow would have been the wrong read. 1 Sports Illustrated described him in terms that were both mocking and reluctantly admiring: "He wobbles. He quivers. He rolls. He shakes. He is a dripping mass of flesh, a monument to fat." Then the same reporter added: he "floats like a baby elephant and stings like a bee." 1

The trials fight went three rounds, the standard amateur distance. Frazier tried to work inside and attack Mathis's body. The punches were "smothered by flab." Mathis kept him at distance with a left hook — each one thrown with a loud grunt, "uuuuunnnnhhh!" — and won a clear points decision. 1 It was not the first time Mathis had done it: this was the second time he had beaten Frazier as an amateur — the only two losses of Frazier's amateur career, both to the same man. 2

Mathis had the Olympic spot. Frazier went home as the alternate.

On September 19, 1964 — three weeks before the Tokyo opening ceremony — it was announced that Buster Mathis had broken a bone in his hand and could not compete. 3 Exactly what broke and when is genuinely disputed: different sources say thumb, finger, or knuckle; some say it happened during training in Tokyo, while The Olympians notes it may have occurred in the May 20 fight with Frazier himself — four months earlier, possibly already broken when Mathis walked off with the Olympic berth. 1

Nobody knows for certain. What is certain: Mathis was out, and the alternate was in.

Frazier, already in Tokyo as the reserve, stepped straight into the heavyweight bracket. He stopped his first two opponents in the opening round. In the semi-final he knocked out Vadim Yemelyanov of the Soviet Union in the second — and broke his own left thumb in the process. 3 He told no one. He had it taped. He kept fighting.

The heavyweight gold medal bout was October 23, 1964, at the Korakuen Ice Palace in Tokyo. Frazier's opponent: Hans Huber, a 30-year-old bus driver from Bavaria who moved cautiously and kept retreating throughout. 2 Without his left hook — the hand was broken, and the weapon that had defined his style for years was largely unavailable — Frazier relied almost entirely on his right.

He won, but barely. Three judges scored for Frazier, two for Huber. The decisive punch came in the last minute of the final round: Frazier finally unleashed his left hook, connected with Huber's ear, and Huber "wobbled and swayed" before clinching his way to the bell. 2 A 3-2 split decision. Frazier was the only American boxer to win gold at the 1964 Tokyo Olympics. 3

Afterward, he was asked whether Buster Mathis was still on his mind.

Then Frazier flew home to Philadelphia, and nobody was at the airport. He went back to work in a slaughterhouse while his thumb healed. 3

Four years later, on March 4, 1968, both fighters arrived at Madison Square Garden undefeated. Mathis was 23-0 with 17 knockouts. Frazier was 19-0 with 17 knockouts. The prize was the vacant NYSAC World Heavyweight Title, stripped from Muhammad Ali after he refused military induction. 4

The amateur pattern reversed. Over three rounds, Mathis could box and keep distance; over fifteen, Frazier's body attack accumulated. Mathis led on points through six rounds. Then Frazier's pressure wore him down, and in the 11th, a left hook dropped him. Referee Arthur Mercante — one of boxing's most respected officials of the era — stopped it at 2:33. 1 Mathis had never been floored before. This was the first time.

Frazier went on to defeat Jimmy Ellis for the undisputed championship, then handed Muhammad Ali his first professional loss in "The Fight of the Century" in 1971. 4 He is consistently ranked among the greatest heavyweights ever to fight.

Mathis won two more fights after the Frazier loss, beat George Chuvalo (a notably iron-chinned contender) by decision in 1969 for a Ring Magazine top-10 ranking, then lost to Jerry Quarry, Muhammad Ali, and Ron Lyle in succession. 5 He retired in 1972 with a record of 30-4. His weight climbed to 550 pounds after boxing. He died of heart failure in Grand Rapids, Michigan, on September 6, 1995, at 52. 5 Two months later, his son Buster Mathis Jr. fought Mike Tyson.

The Olympians put it plainly: "Hard-luck Buster Mathis could not win. He lost his biggest shot at the title against Frazier in 1968. And in 1964, he beat Frazier in the Olympic Trials, and still lost his shot at Olympic glory. If not for Frazier, he coulda been a champ. Twice." 1

The obvious lesson here — timing is everything, fate is cruel, one injury changes a career — is real but incomplete. What makes this story genuinely strange is the symmetry: Mathis beat the man who would become champion. The champion also showed up to the Olympics with a broken bone and hid it. Both men were simultaneously injured, one visibly, one secretly. Frazier's broken thumb was not diagnosed before the gold medal bout; his coaches and corner did not know. The difference between a gold medal and a bronze, between legend and footnote, came down to who told someone about their hand.

For trivia purposes: Frazier is the only man to go to the Olympics as a replacement, win gold with a broken thumb, and later become undisputed heavyweight champion of the world. There is no one else in that category. The man he replaced, who put him in that position, is the reason any of it happened — and almost nobody knows his name.