The Ghana midfielder scored Hearts' first-half opener against in-form Motherwell at Fir Park before setting up Andrius Velicka for a late second deep into stoppage time.
Motherwell handed a debut to Stephen Hughes following his arrival on transfer deadline day.
The former Leicester City midfielder had been widely expected to join Hearts only to seal an alternative move to his hometown club at the 11th hour.
Hearts though were also able to hand a debut to another new signing after naming former Sheffield United striker Christian Nade in their own starting line-up.
It was the visitors who settled first and Andrius Ksanavicius felt he should have earned an immediate penalty when he appeared to be bundled over in the box by David Clarkson.
Calum Elliot was next to threaten for the Edinburgh club when he met Robbie Neilson's cross but saw his diving header from 12 yards parried away by goalkeeper Graeme Smith.
Hearts were knocking on the door at that stage and duly took the lead on 24 minutes through Kingston's first decisive moment.
Kingston toyed with Jim Paterson just outside the penalty area before picking his spot and arrowing a low drive into the corner of the net to break the deadlock.
Motherwell finally found some attacking rhythm as half-time approached and only a fine block from Ibrahim Tall prevented Keith Lasley from restoring parity with a shot from 15 yards soon after.
Well began the second half at a frantic pace but a series of fine defensive blocks prevented Clarkson and Ross McCormack from finding the back of the net from close range.
Steve Banks in the visiting goal repelled a low shot from Steven McGarry as the home side built up a head of steam and Chris Porter was next to try his luck but found the veteran shot-stopper equal to his stabbed effort as the minutes ticked away.
But with Motherwell pushing forward in search of the equaliser, they fell to a classic sucker punch in time added on.
Kingston sent Velicka free and he chipped the advancing Smith to wrap up three much-needed points for the Edinburgh club.