At least 10 million pilgrims are set to do so by the end of the day.
The event, every 12 years, is billed as the biggest gathering on Earth. More than 100 million people are expected to attend the 55-day festival.
Hindus believe a festival dip will cleanse sins and help bring salvation.
In 2001, more than 40 million people gathered on the main bathing day of the festival, breaking a record for the biggest human gathering.
|What is a Maha Kumbh Mela?|
The Kumbh Mela is a mass pilgrimage in which Hindus gather in specific locations along the holy rivers Ganges, Yamuna and the mythical river Saraswati.
There are three different kinds of kumbh: an ardh (or half) kumbh is held every six years at two set locations; a purna (full) kumbh is held every 12 years at four set locations.
The 2013 gathering is a Maha Kumbh and that only ever happens after 12 purna kumbhs every 144 years - and it is always held at Allahabad.
Astrology determines most aspects of the festival, including its exact date and length. Where the festival is held also depends on the position of Jupiter, the sun and earth.
The festival formally started at dawn on Monday. All roads leading to the Kumbh Mela grounds are packed with hundreds of thousands of pilgrims.
There was a chill in the air as holy men sprinted into the waters in Allahabad, but the day dawned warmer than in recent weeks when a cold snap hit northern India.
Police estimated that by the late afternoon about 7.5 million people had bathed.
For festival-goers, one of the most memorable spectacles of the day was when the Naga sadhus, or ascetics, sprinted into the river reciting religious chants, many clad only in marigold garlands.
The naked ash-smeared men arrived in a colourful procession and waded into the chilly waters of Sangam - the point at which the rivers converge.
The Kumbh Mela has its origins in Hindu mythology - many believe that when gods and demons fought over a pitcher of nectar, a few drops fell in the cities of Allahabad, Nasik, Ujjain and Haridwar - the four places where the Kumbh festival has been held for centuries.
Although the gathering is held every 12 years, this year's festival is what is known as a Maha Kumbh, which only occurs every 144 years and is always held at Allahabad. It will last for 55 days, a period of time determined by an astrological calculation.
There are six particularly auspicious days to bathe - the biggest bathing day will be on 10 February when approximately 35 million people are expected to take to the waters.
Teams are managing crowds on the river bank - as soon as pilgrims finishing bathing, they are encouraged to move away and make space for other bathers.
Shankari Devi from the Indian city of Udaipur arrived on Sunday and spent the night in the open but her conclusion this morning was that "we had a good bath so all the troubles were worth it".
"I have washed off my sins," Mandita Panna, a resident of Nepal and another early bather, said.
Ashok Kumar Singh also took to the waters but he expressed concerns about its cleanliness.
"Yes I had a good bath... but it is us who dirty the Ganges, we are responsible. Most of the pilgrims were throwing their marigold garlands into the water and that pollutes the water."
Allahabad has been preparing for the festival for months and a vast tented city has grown up around the river.
Fourteen temporary hospitals have been set up with 243 doctors deployed round-the-clock, and more than 40,000 toilets have been built for the pilgrims.
Police checkpoints have been set up on all roads leading to Allahabad and about 30,000 policemen and security officials have been deployed to provide security during the festival. At Sangam, some policemen on horseback have been posted.
Tens of thousands of men, women and children have set up camp on the white sands of the river front.
On Sunday night, smoke could be seen rising from hundreds of small fires which people had built to cook dinner or keep warm.
The sadhus have been leading processions accompanied by elephants, camels, horses, chariots and music bands in recent days.
The festival has prompted health concerns, however, with campaigners warning that the river waters are heavily polluted.
Most pilgrims drink a few drops of the Ganges water and many fill bottles to take home with them.
Authorities say they have taken steps to address the concerns.
Last week, companies along the banks of the Ganges and Yamuna were warned against discharging any pollutants into the waters.
Reservoirs upstream have been ordered to discharge fresh water into the rivers ahead of the six big bathing days, and the festival authorities have declared the Kumbh Mela area a plastic-free zone.
The Kumbh Mela, which is costing the authorities 11.5bn rupees ($210m; £130m) to organise, is expected to generate business worth at least 120bn rupees, according to a report by India's Associated Chambers of Commerce and Industry (Assocham).
The report says that the festival is also expected to draw over a million foreign tourists.
Pandey, Geeta. 2013. “India's Hindu Kumbh Mela festival begins in Allahabad”. BBC News. Posted: January 14, 2013. Available online: http://www.bbc.co.uk/news/world-asia-india-21006259