Whenever I get in the car, the dash shows "Driver 2 detected".

What determines this? The key itself? Or the Key fob?

I have 4 set of keys... and 1 key fob.

 

key fob. there's a label on the backside near the key ring loop that indicates it but it can be worn off.

if you put the memory to driver 1, press unlock with your fob, it'll switch to driver 2. Key doesnt matter.