WordPress by default adds an extra 10px to the width of the container that wraps images with captions. This is because the default styling for captions has a thin border that surrounds the entire image.

In a recent design I decided I wanted to caption images in a slightly different way and just have the caption below the image without the surrounding border like so.

Its a cat!

To remove the additional 10px just add the following code to your function.php.

function remove_caption_extra_width($width) {
   return $width - 10;
}
add_filter('img_caption_shortcode_width', 'remove_caption_extra_width');