The fire is believed to have spread quickly due to lots of raw material being stored inside the factory, which manufactured handbags.
However, no details were immediately available on the cause of the fire.
Atul Garg, an official with Delhi Fire Services said: "Till now we have rescued more than 50 people, most of them were affected due to smoke."
Nearly 30 fire engines were rushed to the spot.
The factory was operating in a congested residential area.
Amit Shah, India's home minister, said in a tweet: "Have instructed concerned authorities to provide all possible assistance on urgent basis."
The fire was doused and a team of National Disaster Response Force was searching the factory to find out if any more people were trapped.
Officials will now conduct a probe into the cause of the fire.
Prime Minister Narendra Modi described the fire as "extremely horrific."
"My thoughts are with those who lost their loved ones. Wishing the injured a quick recovery," he tweeted. Authorities are providing all possible assistance at the site of the tragedy, he said.
