Malian government soldiers have struggled to push the rebels back
Two army helicopters were scrambled in response, a local official told AFP news agency by phone.
Gao, with a population of 87,000, more than twice the size of Kidal, hosts one of the biggest garrisons in the north.
Separatist rebels seeking to carve out a desert homeland began a rebellion in the west African state in January.
'People running'
"We can hear heavy fire coming in the direction of the main military camp," a Reuters reporter said.
"People here are running all over the place and all the shops are closing."
Mahamane Diakite, an aide to the governor of Gao, told AFP: "We can hear heavy weapons fire. We have also seen two helicopters taking off to shoot. Rebels have entered the town."
Mali's government forces have struggled to drive back the rebels. Mid-ranking officers in the military overthrew the government this month, saying the army needed more equipment to fight.
Coup leader Capt Amadou Sanogo has asked for foreign help to tackle the rebels but his coup has been condemned, with neighbouring states threatening sanctions.
Three members of the military leadership have gone to neighbouring Burkina Faso for talks with President Blaise Compaore, who is mediating in the crisis.
The regional body, the Economic Community of West African States (Ecowas), has said it will close land borders, freeze Mali's assets and impose a financial blockade if the army does not stand aside before Monday.
They have also placed a peacekeeping force on standby.
The Tuareg fought side by side with Islamist fighters to take over Kidal, the BBC's Thomas Fessy reports from Dakar.
However, it is not clear how they will share their success, our correspondent says.
Rebels from the National Movement for the Liberation of Azawad (MNLA) want an independent north while a smaller, Islamist group called the Ansar Edine wants to impose Sharia law.
Azawad is the Tuareg name for their home region in the Sahara Desert.
Tuaregs have launched several rebellions over the years, complaining that the government in Bamako ignores them.
