"The terrorists, numbering over 2,000, appeared from various directions on Thursday and engaged the soldiers in Kirenowa town and adjoining communities in Marte," said Imamu Habeeb, a local community leader.
"They fought with soldiers over the night and the fight continued today (Friday), forcing hundreds of soldiers to flee," he added from Borno state capital Maiduguri.
Local fighter Shehu Dan Baiwa said more than 2,000 fighters had been armed with bombs and tanks. "They used the weapons without restraint and succeeded in killing several people," he was quoted by AFP as saying.
This is the third time Boko Haram has seized control of Marte in restive Borno state, a key battleground of their six-year insurgency, which has killed more than 13,000 and left 1.5 million homeless.
The city is among several retaken in recent weeks by Nigeria's military, which has launched an offensive against the militants as part of a regional operation supported by Chad, Cameroon and Niger.
But Boko Haram have been fighting back, and Nigerian troops were also forced to retreat from Boko Haram's Sambisa Forest stronghold this week after a landmine blast killed one soldier and three vigilantes.
A senior local politician confirmed, on condition of anonymity, that the insurgents had retaken Marte.
"We lost many (people) because some of our people that fled to Chad and Cameroon have return after Nigerian troops recovered the town recently," he added.
A senior military official confirmed the attack on Marte, but refused to say whether Boko Haram had retaken the town, describing the army's retreat as "strategic".