The purpose of a CDN is to better cache common resources between different web sites. For example, if you’re using a Roboto font from Google CDN on your web site, just like many other web sites do, the user who previously visited other sites with such font will load your web site much faster and will spend less traffic, because he already has this font from CDN in their cache. It also means that you save money on hosting.

If you remove CDN from the equation, you punish yourself and your users. That’s a very dumb idea. Especially when CDNs are free to use.