Plugboards affect information put into the book's metadata. Save-to-disk/send-to-device templates affect where the book was stored.
What are you looking at to see if something has changed? If you are looking at file names, then you won't see any difference. If you are looking at metadata, you should see a difference. Calibre can update metadata only on books that are not DRM-infested, but as you are using Baen Free books, that shouldn't be an issue.
If you run calibre in debug mode and send to the device, buried in the output will be something like
Code:
Device using plugboard epub FOLDER_DEVICE [[u'{title_sort}bar', u'title_sort']]
Setting metadata in: ...
where you see your plugboard and KINDLE2. The next line(s) should give the title(s) of the book(s) affected.