When Whitman first wrote this poem, slavery was a fact of the American culture. There are two places where the poet celebrates the humanity and nobility of the African American. First is his meeting with the runaway slave:
"The runaway slave came to my house and stopt outside,
I heard his motions crackling the twigs of the woodpile,
Through the swung half-door of the kitchen I saw him limpsy and
And went where he sat on a log and led him in and assured him,
And brought water and fill'd a tub for his sweated body and bruis'd
And gave him a room that enter'd from my own, and gave him some
coarse clean clothes,
And remember perfectly well his revolving eyes and his awkwardness,
And remember putting piasters on the galls of his neck and ankles;
He staid with me a week before he was recuperated and pass'd north,
I had him sit next me at table, my fire-lock lean'd in the corner."
We see Whitman's compassion for this man. The last line quoted marks the poet out as a contrast to those who hunt the slave.
Not long after that Whitman portrays the strength and nobility of a black worker.
"The negro holds firmly the reins of his four horses, the block swags
underneath on its tied-over chain,
The negro that drives the long dray of the stone-yard, steady and
tall he stands pois'd on one leg on the string-piece,
His blue shirt exposes his ample neck and breast and loosens over
His glance is calm and commanding, he tosses the slouch of his hat
away from his forehead,
The sun falls on his crispy hair and mustache, falls on the black of
his polish'd and perfect limbs.
I behold the picturesque giant and love him, and I do not stop
I go with the team also."
The key I think is that in making a "Song of Myself" Whitman is incorporating all of humanity in his vision. Only thus can one be truly oneself.
As another poet said
"No man is an island entire of itself; every man
is a piece of the continent, a part of the main;"