Davis was a member of a local group called the Six Soul Survivors around 1966 and later in another group called the Endless Chain. In 1968 he was a writer for Malaco Records, based in Jackson, Mississippi. Davis signed to Bang Records in 1969, and in 1970, released a cover version of The Jarmels' hit song "A Little Bit of Soap", reaching No. 52 on the Billboard pop charts. His first album, A Little Bit of Paul Davis, was released in 1970. In 1974, he recorded his third album, Ride 'Em Cowboy, and the title track, his first top 40 single, peaked at No. 23 on 18 January 1975. The same song also became a Top 40 Country hit for Juice Newton in 1984.