[Turtle91]

I think letter-spacing adds that padding after each letter (not just between letters as is commonly understood) and that would be left or right of the letter based on the reading direction (rtl/ltr) of the language. The only time that is really noticeable is in this particular scenario where you are trying to center something.

When trying to control the spacing in a divider I've found that using word-spacing is more predictable than letter-spacing. You just need to add a space between each character to delineate the 'word'.
<div>*_*_*</div> ( _ means space )


However....
I would recommend using the horizontal rule <hr/> for chapter dividers, not <div>s. The accessibility standard requires the hr so that reading systems can tell when there is a break. It is really easy to style the hr... Yes I know that sounds weird but it is very simple to style the tag to show images and characters not just line height/width/thickness.

Here are a couple examples of using an image as well as characters and using word spacing for comparison:

CSS:
div {text-align:center; word-spacing:1.5em}

hr {border:none; margin:1em 0; text-align:center; word-spacing:1.5em}

/* using an image fleuron*/
hr.ChDiv1 {
  height:2em;
  background: transparent url("../Images/img_ChDiv.svg") no-repeat center;
  background-size: auto 100%;  
}
/* using symbols*/
hr.ChDiv2::after {content:'◆◆◆◆◆'}

hr.ChDiv3::after {content:'* * *'}

hr.ChDiv4::after {content:'♥♥♥'}

HTML:
  <p>Lorem ipsum dolor sit amet, consectetur adipiscing elit.</p>

  <hr class="ChDiv1"/>

  <p>Suspendisse fermentum nec mauris sit amet laoreet.</p>

  <hr class="ChDiv2"/>

  <p>Phasellus nec lorem dignissim, sodales lorem ac, luctus nulla.</p>

  <hr class="ChDiv3"/>

  <p>Donec viverra ut sem at ornare. Fusce neque augue, dignissim.</p>

  <hr class="ChDiv4"/>

  <p>Sed vehicula massa id vulputate mollis.</p>

  <div>♥ ♥ ♥</div>

edit: added colors to make it pretty