Your time is extremely valuable, and execution of your product is everything. When it comes to building out key pieces of infrastructure, the choice of whether to build it, buy it, or use open source is very personal, and depends on what the business really needs.
You do things yourself when you need fine grained control of a resource, or require a deep understanding of a mission critical function. Can anyone do it better than you can? Even if they can, do you have enough control in the direction to get what you need? Does the time it takes to learn and execute in a given domain warrant the value it provides? Or could that time be spent on something else that would move the needle?
Dead giveaways are things the user sees. Design and product development can’t be outsourced, nor can the actual execution of your product. Those stay in house, no matter what. Your ability to find your voice and interact with your customers is the thing that makes your product unique. If you cannot execute on your own product, it’s a strong indication you’re not doing the right thing.
However, depending on your needs, there gets to be a long list of things that aren’t your product, but are still critical to running your business. Campaign and transactional email are great examples. Executing on mass email is extremely difficult because of defenses put in place by major ISPs. You need an IP with a good reputation, reverse DNS, a host that will allow massive amount of email to leave it’s datacenter, and the list goes on and on.
There are also implicit supporting requirements of running a service that you need to have but shouldn’t ever try to build. Project tracking, monitoring, graphing, and alerting are all examples of components that have executed extremely well by others, and can simply be turned on or deployed with minimal effort on the part of your team.
In the end, what’s critical is spending your energy on the things that actually create impact. Knowing the intersection of tradeoffs and priorities is what will keep your team on the path that gets the job done.