Footage shows rescuers trying to use ladders to help people escape
The fire started before dawn on Sunday in a building housing seasonal workers. In all, three buildings were affected.
Seventy firefighters battled to bring the blaze under control. The cause has not yet been confirmed.
Footage from the scene shows fire crews using ladders to try to rescue people from inside the three-storey building.
One video, posted on social media, shows at least one person jumping from an upper floor window in an attempt to escape the blaze.
Local media reports say the building houses about 60 resort workers, including foreigners.
Three of the four seriously injured were taken to hospital by helicopter, local officials said.
They added that some of those caught up in the tragedy were probably sleeping when the fire broke out. It took emergency crews several hours to control the blaze.
The accommodation at the centre of the fire was situated above a Chanel retail store.
Frédéric Loiseau, a local government official, told French broadcaster BFMTV that the identities of those killed had not yet been established.
"In the case of those seriously injured, it's impossible to specify whether their injuries came from a fall. Because the fire was on the third floor, some people jumped to save their lives," he said.
"There were carbon monoxide emissions, and there were burns."
He added that local prosecutors and police investigators, who had cordoned off the area, would now launch an investigation into what started the fire.
Christophe Castaner, France's Interior minister, tweeted his condolences to the victims' loved ones and paid tribute to the firefighters.
Comment: See also: Huge gas explosion at university in Lyon, France