Second half goals from Dennis Oli, Adam Miller and a Simeon Jackson penalty turned the game on its head after Marc Richards had given Vale the lead.
Vale were given a major boost with top scorer Richards returning from suspension to replace Kyle Perry in attack, while Gills boss Mark Stimson opted for the same eleven that triumphed at Morecambe.
And it was the visitors, buoyed by Saturday's 1-0 win, who almost took a third minute lead when Andy Barcham was denied by Joe Anyon with Luke Prosser blocking Gary Mulligan's follow up.
However, Vale responded well with Simon Royce having to tip Louis Dodds' rasping drive over the crossbar after 13 minutes.
In the 22nd minute Oli almost gave Gillingham the lead, but the former Grays striker was unlucky to see his shot clip the bar with Anyon a mere spectator.
Both sides continued to press but it was Vale who went in front in the 29th minute as Richards notched his 10th goal of the season from Steve Thompson's lay off.
On a bitterly cold night, the goal lifted spirits amongst the home contingent and seemingly put Vale in control of the game heading into the second half.
But just four minutes after the interval Oli latched onto a loose ball before firing high into the net from 18 yards.
And after a spell of pressure the Valiants surrendered the lead on 69 minutes as Miller hooked home from John Nutter's deep cross.
The turnaround was complete three minutes later when referee James Linington penalised Anthony Griffith for hand ball in the area and Jackson made no mistake from the spot.