Scaling in startups — Lessons learned
A month ago, i hosted a WomenWhoCode meetup at our Qventus HQ in Mountain View, California. As part of the meetup, i gave a talk about “Scaling in startups — Lessons learned”. A lot of folks had approached me about this and encouraged me to speak.
The enthusiasm in the meetup was tremendous, and folks wanted to copy of the presentation. I decided to publish it on medium, as a initiative to share my experiences. In the hope it might be useful to anyone going through a similar journey :-)
A bit of a background. Here’s what i mean by scale, as its very subjective:
- Team of 4 Team of 50 & growing
- Single hospital system Multiple hospital systems(25+)
- West coast only hospital to hospitals across US
- Data in few GB to data in TB
When startups grow, you have a unique opportunity to witness the various challenges and solutions at every step a startup faces/develops. The ones listed below are purely from a engineering standpoint. The list below is by no means complete :-) (You know that!)
Lesson #1
- UTC !!!
- Always always store datetime in UTC, not PST or EST
- If every datetime is stored in UTC, getting relative times is that much easier
- Or better yet, timestamps !
- You will not lose your mind in conversions :-)
Lesson #2
- Know thy limits of your underlying DB
- Our system is expected to process complex joins and complex queries in general
- MySQL will not fare well for peak performance
- Start working towards the path of upgrading to redshift / dynamodb / mongodb etc
Lesson #3
- As you scale, system reliability becomes core to your success or failure in growing
- Invest in solid error monitoring stack
- Splunk / ELK (Elastic search, logstash, Kibana) / AWS metric monitors to name a few
- Alerts in every third party integration
- We also ended up building our own ML model monitoring tool using Cassandra / Kafka / Spark
Lesson #4
- Follow the policy of — start with minimal access
- Every member of the team should have appropriate roles assigned
- AWS IAM is your savior !
- Roles / Policies / Groups -> Define them appropriately
Lesson #5
- Training/onboarding is an important aspect of a new joinee
- Do not compromise on that time !
- A well trained team member is bound to be more efficient & confident
As i mentioned, the list above is by no means complete. This talk just gave me an opportunity to sit down and think about what are things we could have done better. If you would like to know / learn more, contact me, and would be happy to chat.
Another thing to note is, sometimes, you may have the advantage of scaling with ample time in hand. Quite often, you don’t :-).
