Those expecting things to turn in United's favour were wrong as Rooney remained a threat, forcing a sprawling save out of Sutherland before rasping a left-footed piledriver past the keeper to restore Bury's lead with only 21 minutes remaining.
Thankfully, Joyce's side were swiftly back on level terms as Tyler Blackett, pushed into midfield for the second half, embarked on a powerful run that led to Will Keane turning his marker and supplying a terrific finish past Jensen for his second goal in two games since returning from injury.
After Pereira saw a free-kick fisted away by Jensen, more good work by Blackett, who capitalised on hesitancy by Jennings, set up Will Keane again for a side-foot into the inviting target. It looked like the visitors had done enough but substitute Dudley raced through to get his shot away as Thorpe challenged to beat Sutherland and make it 3-3.
So a shoot-out was required and Will Keane, Pereira, James and Thorpe all scored from the spot with only Cole missing due to a Jensen save. Shaun Harrad and Euan Holden missed the target for Bury to enable United to win 4-2 on penalties to take the three points, with the Shakers still picking up a fully-deserved point. The Reds face Bolton in the competition on Monday night.
United: Sutherland; Varela, M.Keane, Thorpe, Blackett; A.Pereira, Ekangamene, Cole, James; Wilson (Grimshaw 78), W.Keane. Subs not used: Barmby, Henderson, McNair, Love.