Personally, I'm not sure you'd get much in the way of evil protagonist + evil wins full-length novels outside of suspense horror-borderline thrillers like the Hannibal Lecter books or
American Psycho. Even the
Dexter novels the TV show is based upon put him as "semi-cuddly evil we like vs not-so-cuddly evil we don't like".
That said, if Flashman-type self-serving callous bastards who usually manage to get out more-or-less on top appeal, then you might want to try Peter David's semi-satirical Sir Apropos of Nothing series. I've only read the 1st, so I don't know if he reforms in some way afterwards, but he was pretty unrepentant in that book while managing to screw a bunch of people over (admittedly, a number of them quite deserved it).
Also, if you're again willing to look outside classic-type fantasy, here's a few more candidates:
Grey and greyer morality predominate in several historical urban fantasies of the old-school "unexpected magic in modern life" type (not the current 1st person snarker paranormal PI in a world of overt magickal creatures).
I highly recommend Ian Tregillis' WWII set super-science vs eldritch magick Milkweed trilogy, the 1st book of which I found very enjoyable. Unfortunately, the other parts of the trilogy aren't out yet and you've stated you don't want to do unfinished series, but I suggest reading the online tie-in story @ Tor.com (no real spoilers and will give you a definite taste of why I think this almost fits what you want, keeping in mind that the character in question is one of the main not-quite-viewpoint, but very prominent ambiguously intentioned motivators who part-dominates the narrative):
What Doctor Gottlieb Saw.
If you find this story appeals to you and you'd like to read more like it, then I also highly recommend Tim Powers'
Declare, which is basically what you'd get if you had a John Le Carré novel and mashed it up with some Biblical/Lovecraftian mysticism and turned it into an arms race. Again, good and evil become increasingly irrelevant terms as the people who pretend to themselves that they're working on the right side find themselves sliding increasingly into the self-justifying abyss, as is the case in many espionage thrillers.
The following are spoiler-texted in case you don't want to know too much about the story in advance, but I assure you these are worth reading if you haven't come across them already.
For a flat-out
take on this trope, I suggest Charles Stross' excellently creepy novella
A Colder War, available to
read free online and also in Stross' collection Toast, which he offers for free download in a variety of e-book formats
off his website.
For a nifty
take on this trope, you might want to try Neil Gaiman's marvelously subversive
A Study in Emerald if you haven't already, which is also a truly brilliant Holmes pastiche, and offered free for download (PDF, but worth it)
off his website.
On the sf side, you may also want to try some of Peter Watts' stories and novels, since he often has a rather dark and cynical take* on things and tends to go for "your victory is hollow and tastes like ashes in your mouth".
In any case, here are two of his Hugo-winning or nominated stories free to read online which I think will fit in fairly well with what you want.
The Things is probably the closest candidate to doubleplus ungood protag not defeated in the end. It's also a riff on a classic sf story and movie based upon.
The Island is more the grey-and-greyer what-you-thought-was-right-may-not-turn-out-to-be kind of story. It's very good and you can get it from his website or in an e-book version in our very own Patricia Clark Memorial E-Book Library
collected with some of his other stories (I recommend reading "Fractals", which I consider a rather good look at the fractal nature of human hatred, and "Bulk Food" is a very funny dark satire, especially if you keep in mind that Watts is a marine biologist who probably had to deal with things much like that when he was still living on BC's "Sunshine Coast").
If you like these, you may want to try his Rifters Trilogy, which he offers for free download under a Creative Commons license
off his website. It's very dark in places, there is an evil viewpoint protagonist in some parts of it, and no one really "wins"; they just get to rejoice that maybe they're not quite as screwed as they thought they would be.
Anyway, hope these aren't too far off-topic from what you have in mind.
* This may or may not be due to having been beaten up by the US border guards and contracting flesh-eating bacteria. Mind you, he was kind of like that even before the border guards and having to have a chunk of his leg cut out.