I don't think it's so much that he annoyed the Americans, but rather he was infuriated by the fact the American law gave no protection to the works of British authors at the time, and American publishers made large sums of money from reprinting his enormously popular books, for which he received not a penny. One can understand his viewpoint

.