The couple welcomed Neve, their first child, on Thursday, announcing the news on Instagram.
They announced today the baby's name: Neve Te Aroha Ardern Gayford.
In 1990, Bhutto was the first head of government to give birth while in office.
Ms Ardern said she and her partner had opted for the alternative spelling Neve rather than the traditional Niamh, to avoid confusion.
Senegal, Japan: underdogs aiming for a round of 16 spot
Japan: Kawashima; Sakai, Yoshida, Shoji, Nagamoto; Hasebe, Shibasaki; Haraguchi, Kagawa, Inui, Osako. Senegal: N'Diaye, Wague, Sane, Koulibaly, Sabaly, Sarr, N'Diaye, Gueye, Niang, Mane, Diouf.
The 37-year-old leader was admitted to hospital early Thursday morning and a statement released by her office confirmed that the PM was in labour.
She is the second female world leader who has given birth while in office, and the first to take a maternity leave. "And just figure things out as we go".
"Te Aroha was our way of reflecting the amount of love this baby has been shown before she arrived and all of the names we were gifted along the way".
She told media she was "sleep-deprived but doing well" and thanked the midwives, nurses, doctors, security and communications staff at Auckland hospital before addressing the nation.
Support also flooded in from around the world, with Britain's Queen Elizabeth, whom Ardern met in London in April, sending an emailed message of congratulations and Canadian Prime Minister Justin Trudeau tweeting at Ardern: "Pro tip, Jacinda: Briefing notes make excellent bed time stories".
Mexico Proves It's No Fluke at World Cup
Winning the group with three straight victories will be a good sign, but getting past the next round is the ultimate goal . South Korea's president Moon Jae-in is set to be attendance, which Shin feels will be "very encouraging for our players".
"The baby was looking up at Jacinda and it looked like she was in awe of her and I couldn't get over how alert it was just after being born", she said.
While the day-to-day prime-ministerial duties will be handled by the polarising 73-year-old deputy PM Winston Peters, Ms Ardern will still be on call for any major decisions.
She will take six weeks of leave before returning to work.
Clarke Gayford however was quick to relay what it was like watching Jacinda hold Neve Te Aroha Ardern Gayford for the first time.
Former New Zealand Prime Minister Helen Clark said the birth and the way the country had greeted it would be seen as inspirational by advocates for gender equality and women's empowerment.
Uruguay coach defends Luis Suarez by saying superstars are not ‘robots’
Uruguay have lost just one of their last nine World Cup group games, While in last six games they have won five out of them. Suarez stopped the goal of Ghana in the last moments due to which the red card was shown by the refree to him.
"So I hope for little girls and boys that actually there's a future where they can make choices about how they raise their family and what kind of career they have that is just based on what they want and it makes them happy".