The selection you made in the Layers palette should be active in the new channel you just created, so go ahead and fill it with a color. Any color will do, as this color selection won’t have any impact on your final product. After you fill it up with a color, deselect it by pressing Ctrl + D or by clicking on Select, Deselect. Now, go to Filter, Blur, Gaussian Blur. Blur the image to a considerable amount. In my example here, I have set the blurred amount to 22. The more you blur, the smaller the end result will be. After blurring it, pull up the Image Levels dialogue box by pressing Ctrl + L or by going to Image, Adjustments, Levels. Now, this next part is easy. This time around, you don’t need to play with the sliders to see what kind of result it gives you. Simply bring all three of the sliders toward the middle.
This essentially means that you need to bring the two sliders that you see on the extreme left and right, toward the middle slider. As the sliders accumulate in the middle, you’ll see the blur effect going away, with nice rounded off edges appearing on your rectangle. Seems like magic, doesn’t it? Press OK to close the Levels dialogue box. Now, we need to take this rounded off rectangle back to our original layer.
Do a Ctrl + click on the channel that we created (alpha channel) to load the rounded rectangle selection. Then switch back to the Layers palette. You should see the selection you just loaded from the channel active on the layer you related, so go ahead and fill it up with the color you want. Bada bing! You are done. In just a few steps, you have a neat little rectangle with rounded edges!
You can also create a rounded edges rectangle effect by going to Select, Modify, Smooth, but as you would find out, it doesn’t really give a neat sharp effect in the end. I would suggest that you go the Select, Modify, Smooth route only when the rounded off edges that you desire are smaller in nature. So there you go. A basic, yet important, Photoshop trick!
~ Yogesh Bakshi