Davis was a member of a local group called the Six Soul Survivors around 1966 and later in another group called the Endless Chain. In 1968 he was a writer for Malaco Records, based in Jackson, Mississippi. Davis signed to Bang Records in 1969, and in 1970, released a cover version of The Jarmels' hit song "A Little Bit of Soap", reaching No. 52 on the Billboard pop charts. His first album, A Little Bit of Paul Davis, was released in 1970. In 1974, he recorded his third album, Ride 'Em Cowboy, and the title track, his first top 40 single, peaked at No. 23 on 18 January 1975. The same song also became a Top 40 Country hit for Juice Newton in 1984. Davis had his first American Top 10 single with the ballad "I Go Crazy," which after 28 weeks on the Hot 100 peaked at No. 7 on 18 March - 1 April 1978. "I Go Crazy" spent 40 weeks on the Billboard Hot 100, which at the time set the single-song record for most weeks, consecutive or not, on the chart in the rock era. The follow-up, "Sweet Life", also did well, peaking at No. 4 on the Adult Contemporary chart and No. 17 on the Pop chart.