Week 09 - 2015

Week 09 - 2015

#work

Today is about handling events real time. I've been talking to different people about events handling and the outcome of these conversations is that you get a different opinion based on who you're talking to. It's hard to get some consensus on what to choose. Imho it's too easy to say that A is better than B without considering all aspects of the solution.

Questions that you should ask yourself before jumping on one technology:

  • What will the startup cost be?
  • How long will it take to get started with the first MVP?
  • What will the server cost be to keep it running with your traffic?
  • Is there any license cost?

Remember to consider both internal/external man-hours, cost and time line. Be honest with yourself. Listen to people who have done this before and check different sources. Never trust just one blog post.

Twitter is handling millions of events every second. It would be interesting to know who many servers and ops people it takes to run their platform.

Handling five billion sessions a day – in real time // blog.twitter.com - Thanks to@joawan for the tip.

LinkedId is also handling a couple of events every second. The say they have 8 million incoming messages a second and 32 million outgoing messages a second.

Kafka at LinkedIn: Current and Future // engineering.linkedin.com - Thanks to@eide for the tip.

Both Twitter and LinkedIn are using more or less self managed services for their real time data pipeline. Is this the best solution?

Recently Ebay entered the game with their own solution called Pulsar. They've launched it in production and it is currently handling all user behavior events.

Pulsar can be used to collect and process user and business events in real time, providing key insights and enabling systems to react to user activities within seconds.
-ebaytechblog.com

Announcing Pulsar: Real-time Analytics at Scale // ebaytechblog.com

Amazon has been in the game since December 2013 with their managed service Kinesis.

Amazon Kinesis is a fully managed, cloud-based service for real-time processing of large, distributed data streams.
-Amazon.com

Why Amazon created AWS Kinesis, its live data processing service

#personal

When you make a mistake remember that it's not the end of the world. There is always someone who is in deeper shit than you.

Knightmare: A DevOps Cautionary Tale // dougseven.com

And when everything seems a bit dark you should always take care of yourself and drink more coffee.

Do you love coffee? You should probably be drinking even more // qz.com

#fun

This week ends with a couple of fun links.

Language Perl (bottled by Acme::EyeDrops) // 99-bottles-of-beer.net

If programming languages were countries, which country would each language represent? // quora.com

Have a nice weekend!