nikhil.io

In addition, engineers have commoditized many technical solutions that used to be challenging in the past 15 years. Scaling used to be a tough challenge, not any more for many companies. In fact, part of my daily job is to prevent passionate engineers from reinventing wheels in the name of achieving scalability. It’s not because we don’t need to solve scalability problems, but because the infrastructure is good enough for most of companies. Building and operating so called “big data platform” used to be hard, not that hard any more. Building machine learning pipeline used to be hard, not that hard any more for many companies. Of course, it’s still challenging to build a highly flexible and automated machine learning pipeline with full support of closed feedback loop, but many companies can get by without that level of maturity.

– via Hacker News (emphasis mine)