Here is a picture of the channel after being guassian blurred, the selection then incerted and cleared.
(for some reason I couldnt save the channel as a jpeg after having made it a new document from the original letter alpha channel)
Here is a picture of the final letter, after the lighting effects filter has been applied to this channel. The jagged edges are clearly visible, and they are inside the selection not out. Inverted the selection and deleting does not help on this layer.

hope this helps.