@nitinkhanna thanks for the recommendations. I had looked at the EWWW plugin, but seems like it uses their CDN? I'm quiet happy with my already configured S3/Cloudfront cdn so really just need the optimization. Or did I miss something in the description?
@miklb "Images can be optimized using tools on your own server for free (jpegtran, optipng, pngout, pngquant, gifsicle, cwebp)" and then uploaded to Cloudfront/S3 using the plugin. You don't need to use their CDN.