According to reports, 61-year-old Magufuli, a COVID-19 sceptic, had not been seen in weeks amid concerns he had contracted the virus.
Vice president Samia Suluhu Hassan announced Magufuli had died in a Dar es Salaam hospital from heart complications, the BBC reported.
"It is with deep regret that I inform you that today... we lost our brave leader, the president of the Republic of Tanzania, John Pombe Magufuli," she said.
Peak Resources, which holds the Ngualla rare earths project in Tanzania, said it was greatly saddened by the news of Magufuli's death.
"We would like to recognise the substantial contribution of President Magufuli has made to the development of the United Republic of Tanzania and express out condolences to his family, the government, and the people of Tanzania," the company said.
Magufuli, who had been in power since 2015, was named as one of mining's 20 most influential people in 2017 by MNN sister publication Mining Journal.
Nicknamed ‘the bulldozer', he made headlines in 2017 by taking on Barrick Gold and subsidiary Acacia Mining.
He also oversaw the introduction of Tanzania's new mining code, which included the introduction of a mandatory 16% free-carry ownership of projects and a 1% inspection fee on the value of mineral exports.
Barrick president and chief executive Mark Bristow said the company mourned with the people of Tanzania at the loss of Magufuli.
Bristow described the late president as a visionary statesman who saw the value of a thriving mining sector to his country's economy and partnered with Barrick in a joint venture, Twiga Minerals Corporation, to manage the company's mines in Tanzania and to share the economic benefits they generated equally between Barrick and its stakeholders in that country.
Bristow said Twiga would stand as a monument to the late president's foresight and should serve as a model for future partnerships between governments and mining companies in Africa.
Hassan will take over as president, the first female leader in the country's history.