At the Daho.am conference last June, we had the honour to have Christian Hardenberg giving a talk on the 10 things he has learned as Rocket Internet CTO. The talk was so insightful that we decided to transcribe the best parts of Christian’s talk below:
1. Page speed matters:
conversion rate is directly correlated to page speed.
Use not only Page Speed but also New Relic to have deep insights. We aggregate the page speed metrics across all our companies (400+). Reason for slow pages is sometimes due to local slow internet (like in Philippine).
- Defer loading external scripts
- Prioritize content above the fold
- Split CSS in critical and non-critical part
- Reduce DOM size
- Page type specific JS/CSS
- JS/CSS prefetching
- Stream html header early
2. Test Driven Optimization
Optimisation is Dangerous.
1. define your objectives
2. Build realistic Test Data Sets
3. Run jMeter Load Tests
The enemy is generally where you think it, in one of our company Memcache was one of the culprit because of the large amount of data read and write on each request.
3. Test Driven Security
Don’t fix it before you test it. Your worst enemy should be inside: we hired hackers and then they show the team their finding. ‘Showing’ breaches is a good way to make engineer aware of the importance of security.
Security learnings:
- DDOS Attacks : Only solution is external providers (CloudFlare, MyraClowd, Neustar, Prolexic)
- WordPress / Jenkins – Separate from Production environment because very vulnerable
- Password: 2-factor authentication
- Public Github Repos: Automated monitoring
- Credentials Storage in Puppet ; HIERA-EYAML Encryption
- Password hashes – Sanitized views on DB
- Offboarding – Google Apps as Identity Provider
4. Hardware is stable, Networks are not.
How we improved Stability?
1. Minimize time to recovery
2. Detect Network Bottlenecks with Testing
3. Detect Issues early with sensors
4. 24/7 Monitoring and health checks
5. Technologies that work for us
- Rabbit
- Kafka
- Solr: Super scalable
- MongoDB
- PHP
- Moving from PHP to Golang on some projects
- REDIS
- memcached
- AWS
- PostgreSQL
- ElasticSearch
6. Monolith First vs Micro Services
Disadvantages of going with Microservices:
- Overhead is signifcant (but decreasing)
- Early on the requirements are never clear. Refactoring Micro Services is hard
- Hard to do microservices in a KISS Strategy.
Alternative:
- Split along very obvious boundaries with thin interfaces (TMS, Shop, Accounting)
- Merge systems with non-obvious boundaries (Shop, Customer, Order) but avoid tight coupling
- Start unbundling once your team grows about 30 dev.
7. Timeboxes create Efficiency
Time Boxed Rocket Master Launch Process. Change scope rather than timeline. Better keep the launch date fixed as it create pressure to get things done. We prefer SCRUM rather than Kanban because it’s time-boxed.
8. Efficiency is fun
What do we do to make things going so fast? We’re very productive and efficient
Efficiency = Standardization + Inconsistency + Pragmatism + Learning
- Standardization: Sky Rocket Tech Framework.
- Eventual Inconsistency: Same as with database, if you want consistency you have a lot of overhead. 90% works well, accept the 10% inconsistent and fix along the way
- Pragmatism: “Do we really need it?”: KISS + YAGNI, Bias for Action – Don’t ask just do. Combine with a culture of learning.
- Correct Mistakes Fast: Measure and Benchmark everything. Accept new Data and Act. When we move to AWS, it was much more expensive, crashed a lot, and slower. We rolled back. After a while we re-evaluated and we’re not on AWS for all new companies we launch.
9. Simple is beautiful
Leave the campground cleaner than you found it.
Every code you touch, clean it a little bit.
- Ongoing Refactoring
- Pull Requests with Code Review
- Pair Programming
- Unit Testing
- Scrutinizer
- Quality Culture
Being productive is fun. Find technical debt and fix it to not loose the good people.
10. The Mobile Moment
More computers in the pockets than on desktops. Will talk more about this topic next year.
Watch Christian’s video:
