As well as starring in the first two shark thrillers, Scheider received two Oscar nominations during his career.
He was up for best supporting actor for The French Connection in 1972 and best actor for 1979's All That Jazz.
He died in hospital in Little Rock, Arkansas. He had been treated there for multiple myeloma, a cancer of the plasma cell, for the past two years.
Scheider's Jaws co-star Richard Dreyfuss said: "He was a wonderful guy. He was what I call a knockaround actor.
"A knockaround actor to me is a compliment that means a professional that lives the life of a professional actor and doesn't yell and scream at the fates and does his job and does it as well as he can."
Jaws, released in 1975, was directed by Steven Spielberg and was the first film to make $100m (£51m) at the box office.
It is widely regarded as the film that ushered in the blockbuster era.
Scheider's other film credits included Klute, Marathon Man and Sorcerer.
In the 1990s, he played the captain of a giant, high-tech submarine in futuristic undersea adventure TV series SeaQuest DSV.