I created this little tutorial for
new Photoshop users, who want to know how to smooth the edges of a shape.
Mark has written a closely related tutorial (
http://www.photoshopgurus.info/inter...-corners.shtml ), but I?m going to focus in this technique on the use of
Layer masks.
It?s important to know first that every selection has an anti-alias option that can be found on the tool bar once a selection tool is selected. In most cases you will keep this option selected. This same option is also available for the
Paint Bucket Tool
Now look at my attachment; We are going to make the first shape (1) as smooth as the second one (2).
In my example I have the shape on a transparent
Layer 1 (3) that has rough edges.[list][*]
Create a layer mask
I select the object by holding CTRL (CMD on the Mac) and clicking on this layer with the left mouse button. (all non-transparent pixels will be selected, in this case our circle).
Now that we have this shape selected, we click on the
Add Layer Mask button, indicated by a red circle in example no. 3.
We now have created a mask in the shape of our circle. Also notice in example 4, the double lines around the layer mask; these indicate that this layer has been selected. If this layer is not selected, just use the left mouse button to click on it.
[*]
Gaussian blur the layer mask
Now we are going to slightly blur this layer (5).
First we make sure that nothing is selected, so we go to
Select / Deselect (CTRL+D or CMD+D on the Mac). This step is not needed in some cases.
Now we are going to make the mask visible, by holding the
ALT key (Option key on the Mac) and clicking on the mask with the left mouse button (This whole step is optional, but it will make it easier for you to see what you?re doing).
Go to
Filter / Blur / Gaussian Blur?
Now move the slider all the way to the left and then slowly start moving it in the opposite direction, until you can see the rough edges disappear in the blur.
Don?t apply too much, or the final object will be much smaller than the one you had. Make sure that all edges are invisible and that you have a smooth blur.
[*]
Use a levels adjustment to sharpen the mask
And now we?re going to do some magic. With the layer mask still selected(!), we?re going to
Image / Adjustments / Levels?.
When you open this image you will notice little 3 triangles. These triangles can be moved inwards with the mouse button.
Notice the position of the center triangle. Now we move the left triangle to this central position (don?t you worry if the other triangle moves too). Now we move the right triangle until towards the other triangles until you have something like in screenshot #6.
You will notice in the preview window how suddenly the blur disappears and a sharp edge appears?magic! Use too much and the rough edges will show up again, use too less and your edge will be blurry. Just play a bit with all three sliders until you like what you see, but always make sure that all 3 triangles are closer to the right than the left side of the levels screen.
It also might help to switch the preview window off/on to see what you?re doing.
If you?re done you can click
Ok and click on your object layer and you will notice the same smoothness that the mask had. You have now 2 choices; leave it like this or get rid of the mask. If you want to get rid of the mask, you need to merge this layer with a new layer.
If you have any more questions please let me know.
G-man
[/list:u]
How to smooth edges 
Linear Mode
