You can go ahead and actually track these files (sans the sensitive info), but then use:
git update-index --assume-unchanged <file>
on each file. Then you can go ahead and add the sensitive info to each file, but Git will not see the file as changed, and not try to commit (and thus push) that sensitive info.
To get Git to update the info again, you'd use:
git update-index --no-assume-unchanged <file>
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…