Sunday, 16 March 2014

Notes & Learnings from Q Con London 2014 - Day 3

Gunter Dueck - The World after Cloud Computing & Big Data

Gunter is a funny and intelligent man with a delivery style I would compare to that of a stand-up comedian.  Some of his content was on dangerous ground, but he also made some very interesting points.


Gunter showed a diagram similar to the above which illustrates the choices we face when creating an IT solution, which I'm inclined to agree with.

He also showed another diagram which I'll re-create in list form.  

  1. Creative.
  2. Skilled.
  3. Rote work.
  4. Robotic work.
Gunter made the point that work starts off at the top of this list and gradually works it's way down until eventually it's fully automated.  

What I took away... Make sure your work is as close to the top of the list as possible.


Akmal B Chaudri - Next Gen Hadoop: Gather around the campfire and I will tell you a good YARN

This talk was aimed at Hadoop novices which was perfect for me and also the reason as to why I didn't "get" the joke in the title.  Akmal is from Hortonworks which is a company (similar to Cloudera) that offer professional services to support the Hadoop framework.

Big Data 
It now takes only two days for the world to store as much data as it stored from the beginning of time until 2003.  The three Vs of big Data:

  • Variety - unstructured and not fit for a relational DB.
  • Volume - also keeping for a longer duration.
  • Velocity - generated at very high speed.
All of this data, in it's many different forms (Sentiment, Clickstream, Sensor/Machine, Geographic, Server Logs, Text etc etc) is useful to us.

Hadoop
Hadoop is a framework for solving a data-intensive process.  It's fast for large jobs but not small ones.  Hadoop can be integrated with many of your existing data stores and doesn't have to replace anything to add value.

Hadoop 2.0 consists of the following modules:

  • Hadoop Common
  • HDFS - The Hadoop Distributed File system.
  • YARN - A framework for job scheduling and cluster resource management.
  • Map Reduce - For processing large data sets in parallel.
Pig - An extension of Hadoop that simplifies ability to query large data sets on HDFS.
PigLatin - A way to query the data.  Good for people performing ETL.
Hive & HiveQL - Like SQL so less of a learning curve.  This gives a view of the data that is like a relational DB. 

HDFS Name Node - Keeps track of where data is.
HDFS Data Node -  Stores he data.

The exact same data can be stored on multiple nodes for redundancy.  This means loosing one node will not loose your data.

What I took away... Hadoop is not a single framework to learn, it's many related frameworks that seem to be changing rapidly.

Joakim Recht - The Mean & Lean Pipeline

Joakim talked about how Tradeshift has changed as a company and how it's build pipeline changed with it.  He summarized the company's early days (when it was a startup) as:

  • Not enough time.
  • Not enough people.
  • Not enough money.
  • Too many crazy ideas.
  • Too many requirements.

Joakim said that the perfect build pipeline should have the following characteristics:

  • Stops bad code going live.
  • Fast.
  • Ensures consistency
  • Fully automated
  • Creates nice screens and buttons to click.
  • Allows you to talk about it on conferences

...however, setting all of this up takes time and effort!

The development/infrastructure/build-pipeline looked like this in 2010:


  • Java backend.
  • All on Amazon.
  • Drupal front end
  • Subversion - no branches - everything on HEAD.
  • Hudson to run tests.
  • Deploys to prod - semi automatic.
  • Little docs but Lots of tests (which define behaviour)
Moved from Subversion to git.

In an effort to improve UI tests, they migrated away from "fragile" Selenium to Geb and Spock (Groovy based BDD framework)

By Mid 2011 the number of teams and grown which made keeping track of build status on branches was hard.  They custom made a nice build screen which collected build info.  The code for this is not in an open-sourceable state!

New Office in San Francisco
Starting the new office could have gone better for Tradeshift.  I say this because the "new office" slide featured an animated gif of someone putting a knife in a toaster and then being showered by burning hot components in the ensuing explosion.  Having distributed teams working on the same code base sounded like a big cultural change which demanded technical changes to their build pipeline.  More consistency and rules were enforced with Jenkins.  Along with this came mandatory code reviews on github (some requiring a specific group of reviewers, e.g. DB changes).

Other recent issues can be found in this blog post.

What I took away... The better your build pipeline, the more you have to invest in it up-front.  As teams increase in size or become more distributed, the better the build pipeline has to be.


Glen Ford - Lean Under Pressure

Glen (chief architect at Zeebox) clarified the meaning of lean and pressure:

Pressure "The use of persuasion, influence or intimidation to make someone do something."

Lean (in software) -

  • Eliminate waste
  • amplify learning
  • decide as late as possible
  • deliver as fast as possible
  • empower the team

All of the factors above can create tensions within the team as that is where the compromises on the above have to be made.

Tensions within a Startup (I personally think this can be extended to any work) - What you think you need vs What you can get away with.

The Story

  1. Two founders have an idea.
  2. They create a proof of concept.
  3. Investment gained.
  4. To create the idea, they hire (brings challenges).
  5. To deliver as quick as possible, they split the big team up (divide and conquer). 
  6. Inconsistencies creep in with "the idea".
  7. They then slimmed down (removed some management).
  8. Set a goal which was unattainable - Managers like doing this generally (according to Glen), but in a startup there is a notion that you work very hard.
  9. The unattainable goal inevitably results in failure.
  10. Team demoralised - some burnt out.

They go back to the drawing board and try again - this time with better results.


  1. Teams re-organised - cross functional product teams to deliver vertical slices of the idea.
  2. Fostered a culture of urgency but NOT panic (attainable goals).
  3. Increased learning and knowledge sharing with things like lightning talks.
Other things which helped:

  • You build it, you run it (no ops team).
  • Encourages delivering small improvements - Understanding operational costs and being open with them.
  • Garden architecture - However, you must accept the need to prune and shape as you go along.


Final point: Don't be disheartened.  Learn and adapt.  Enjoy the journey as it never ends.

What I took away... Tensions in a team are inevitable as it's where the many compromises have to be made. Cross functional product teams seem to work everywhere.


Volker Pacher & Sam Phillips - How Shutl delivers even faster using the Neo4J, the Graph Database

A very interesting talk on the architecture of Shutl.  Their business provides a point to point delivery service which is an alternative to the standard Hub & Spoke" model (i.e. big distribution centres where packages are sent to and dispatched from) which is offered by the big companies such as Royal Mail.

Generating a quite for a user involves generating many 

Russell Miles - PaaS - Epic search for Truth

Russell explained that a Platform as a Service can promise (and deliver) a huge number of services for you to take advantage of.  PaaS can offer the following:-

  • Database
  • Integration
  • User Management
  • Development workflow
  • Build services
  • Health Monitoring
  • Multitenency
  • etc etc etc
All of this looks and sounds so great that... "Anyone on the golf course will buy it!"

The risk of PaaS
Libraries will do what you want but frameworks will say "work my way".  A PaaS is a framework which inevitably reduces your flexibility.  A PaaS needs to be as changeable as the software you are running on it. 

How to stay flexible with PaaS?
  • Ensure that the barrier to entry for your PaaS is low.
  • FaaS - Fail as a Service - This means apply Darwinian evolution.  Experiment with multiple PaaS as opposed to just using one for everything.

What I took away... PaaS offers many benefits, but beware of the hidden coupling to your software.

Sunday, 9 March 2014

Notes & Learnings from Q Con London 2014 - Day 2

Tim Lister - Forty Years of Teams

Or... "Forty years of playing well with others".  A really uplifting talk by Tim introduced us to day two, where we looked at Tim's very interesting career.  The alternative title was given because "one person cannot do anything substantial" and even if you could it's more satisfying saying "we did this!"

His IT career started by chance, when he found himself living with Fred Wang at University.  Fred's father, An Wang, was founder of Wang Laboratories which meant him and Fred had access to more computing power than his entire University.  They used this to analyse data from their fruit fly breeding as part of his Biology degree.

Tim was also lucky enough to be mentored by computer scientist Michael Jackson who was a "software poet" and produced code so beautiful and lucid, it was as if it were written by aliens.

The term "Best practices" can undermine the intellectual work.  I interpret this point as not blindly following rules regardless of circumstances e.g. "all code released to production must be performance tested".

Tim (with his friend Tom Marco) wrote the very popular book Peopleware now in it's third edition.  On the subject of writing "I think I am a sharper thinker when I write and then re-read it".  I couldn't agree more.

What I took away...Writing helps you gather your thoughts.  If you don't get innate pleasure from IT, move on.

Michael Nygard - Manoeuvrable Architecture

I love aviation so was pleased to see Michael relate aircraft design which win dog fights to an architecture that succeeds in business.

  • A plane that can fly fastest and highest will not necessarily win a dog fight - According to military strategist John Boyd.
  • The plane most likely to win is the one that can change it's speed and direction the quickest.
  • In military strategy, you need to set the tempo of engagement so your enemy adjusts to you.
  • In IT, this means you need to be able to scale up and down fast.  
  • The F16 was almost given a ladder which would have hampered its ability to accelerate/decelerate -   Make sure your projects don't fail due to incremental compromises.
One of Boyd's key concepts is the OODA loop... Observe, Orient, Decide, Act.  The quicker you can do this, the higher the chance of success.

The concepts Michael discussed:

Plurality - It's ok to have multiple services in multiple styles with multiple versions existing at the same time.  Darwinian evolution can be applied to determine what survives.

Use URIs with abandon - This has many advantages.  For example, the terms and conditions on any website frequently change.  If a user registers on a given date, the T's and C's URI can be stored against that user keeping a record of what they agreed to.  Also, the URI stores the reference and complete context (as opposed to a mere id).

I spoke to Michael at the end of his talk as I was keen to get a book recommendation on some of his subjects, his response was "I really should write one".

Dan North - Deliberate advice from an accidental career

Contributor to 97 Things Every Programmer Should Know and creator of first ever BDD framework JBehave.  Much like the day's first talk, Dan reflected on his career so far.  His talk was divided into the following points.

  • Today we are going to learn about DB restores - I missed the start (was talking to Michael, see above) but gathered that Dan once shut down a production DB.  What was memorable about this was his manager's reaction... calmly saying "today is the day we learn about database restores".
  • "It's a fantastic feeling to have someone believe in you" - This sounds corny but I can definitely point to a few things people have said to me that have had a huge impact, the same goes for Dan.
  • They're not pairing, they're just helping each other... at their own desks.  Some developers will resist collaboration (pair programming), but when people are prepared to meet each other half way, collaboration can flow.
  • The stacktrace - Code for the person who supports it... always!  A colleague tried to interpret the root cause of a stacktrace and found it hard despite being told by Dan that it was work in progress.  
  • The Product - Think about the value you add to the product when you code.  Ugly code in live could be giving the business value, beautiful code in development offers none.  
  • The Leader - "You'll have to teach me" - A culture of not being afraid to admit you don't know something is very healthy.  Put the organisation ahead of your ego.
  • Don't hire yourself - We are not rational people and we are biased to like people that are similar to ourselves.  Hiring is one of the most important things in business so read Peopleware (see above).
  • You are what you do, not what you say.  This is especially relevant when under pressure.

Trisha Gee - HTML5, Angular.js, Groovy, Java, MongoDB all together - what could possibly go wrong?

Trisha was able to create all tiers of a web application in-front of a few hundred IT professionals.  She was brave to take this on and skilled to pull it off.

What I took away...Get your IDE keyboard shortcuts sorted!  Trisha was so quick at using hers I could hardly keep up at times. AngularJS looks awesome.

Adrian Cockcroft - Migrating to Microservices

Probably one of the best talks of QCon.  I just wished I could press the pause button as keeping up was hard at times.  Warning - there is a lot of relevant info missing from this.

A brief timeline of Netflix's architecture:
  • 2009 - People said Netflix were crazy.
  • 2010 - What Netflix is doing won't work.
  • 2011 - It only works for unicorns like Netflix (This explains Adrian's twitter avatar)
  • 2012 - OK but we can't do that.
  • 2013 - We're on our way using Netflix OSS code.

A few principles Adrian shared with us:

  • Speed to marketplace is paramount - Netflix learns fast.
  • High trust low process facilitates this.
  • Don't create your own, when something already created will do the job.  Don't create your own voltage regulator for your servers...
  • Favour open source - There is no procurement overhead.
  • Reverse Conway's law - Decide on the architecture you want and then create/structure the teams.
Microservices:

  • Additive model - don't edit or drop an existing service, just create a new version of it.
  • Roll out your changes gradually with small sections of users.  Netflix use the EU as their testing ground!  


Further reading:
Adrian recommended the book Lean Enterprise which is due to be released in May of this year.  To convert from your old architecture read Refactoring Databases.  To look at how some of these new data stores really perform, have a look here.

Bodil Stokke - What Every Hipster Should Know About Functional Reactive Programming

Bodil created an interactive game using the RxJS set of libraries during the talk... very impressive.  Source code available here.  

Linda Risling - Organizational Change Myths

The author of Fearless Change spoke to us about the following myths....

  • "Smart people are rational" - They are definitely not as rational as you'd think.  See Thinking fast and slow
  • "Good always triumphs evil" - Just because your idea is good, doesn't mean it will succeed.  You have to fight for your idea.  Linda recommends giving people food in meetings to convince them.
  • "If I just had enough power I could make people change" - If you had the power, all you can hope for is getting compliance, this is not change.  For change, people need to be convinced.
  • "Skeptics, cynics and resistors must be stupid or bad" - Treat these people as valued partners in the change effort, not enemies.
  • "I'm a smart person, I don't need help from others.  After all it's my idea." - The change must not be all about you.
A lot more of this good stuff available in Linda's book Fearless Change.

Thursday, 6 March 2014

Notes & Learnings from Q Con London 2014 - Day 1

I was lucky enough to go to Q Con London 2014.  Here is an update of my experience from day 1.

Damian Conway - Life, The Universe and Everything

Damian is an amazing speaker and made his already fun, interesting, geeky subject, even more fun interesting, and geeky with his great presentational skills. He showed us Perl code (and later Klingonscript) that not only implemented the game of life but also (kind of) disproved Maxwell's Demon. We were also shown an example of a Turing complete machine which was implemented using game of life made by Paul Rendell.  Video also on youtube.

This talk reminded me of another cellular automaton I encountered many years ago whilst studying Genetic Algorithms at University.  I am pleased to say that the creator (David Eck) has converted the original Java Applet version of his program Eaters, that so impressed me, into javacript available here.

What I took away...Coding can and should be fun.

Daniel Schauenberg - Development, Deployment & Collaboration at Etsy

Etsy have around 150 engineers working on a monoloithic php application which forms etsy.com.  Although this sounds like an old fashioned architecture, their delivery pipeline is very impressive.  They manage 50 deploys a day.  Other points:-
  • They favour small changes.
  • 8 people can deploy different sets of changes in one build (that's there magic number).
  • The deploy process the "Deployinator" has two buttons only - no ambiguity, everything automated.
  • They use config to switch features on after the code is safely in production (i.e. soft live releases).
  • Developer VMs can run the entire stack and closely mimic production.
  • They make use of Linux Containers (LXC) On day 1 of working for etsy you deploy a change - This seems like a great idea to me.
  • Tons of dashboards which are monitored after releases.

I asked Daniel at what point etsy decide to roll a build back and stop debugging an issue, he replied that there is no rollback (the closest thing to a rollback is revert commits and create a new build).  This doesn't seem like a good idea to me. I think each build should be rollbackable.  Debugging an issue calmly after you have restored service with a rollback is surely better than a forward only stratgey.

What I took away...Continuous delivery can be achieved without a perfect architecture if you have the right processes and culture in place.  Etsy should be applauded.

Uwe Friedrichsen - Fault Tolerance & Recovery

Uwe explained a lot of common fault tolerance patterns, including...
  • Timeouts.
  • Circuit breaker.
  • Shedding load.
Uwe also explained the Netflix library Hystrix.  

What I took away... There are a lot of good patterns out there for fault tolerance, a lot of them are discussed in Release It!

Graham Steel - How I learned to stop worrying and trust crypto again

Graham started his talk by saying that "the paranoids were right" - NSA has been intervening with crypto standards.  However, all is not lost as properly implemented crypto systems can be relied on.

Here's what makes a good crypto API:-


  • Open and subject to review - More likely to have bug fixes and patches applied.
  • Supports modern cryptographic primitives.
  • Good key management - This can be the Achilles heel of a crypto API.
  • Mistake resistant - Some think people should be free to make mistakes.  I don't!
  • Interoperable.
Examples of crypto APIs.... My notes are sparse here...

  • Big success, widely used.
  • Old - Current version 2.4 was released in 2004.
  • Implementation is closed, but the standard is open.
  • The API is closed (under Oracle's control), however some providers, like Bouncy Castle are open source.
  • Has around fifty "Fix me" comments in it!
  • Contains the NSA backdoor random number generator which you are free to use or not.
  • Work in progress to provide encryption in clientside javascript - Lots of challenges here!
Lessons
  • If you roll your own cryptography implementation, you are guaranteed to have a floor in it.  
  • TLS has been around for years and they found issues just 3 days prior to this talk!
  • Applied Cryptography is a dangerous book as it encourages you to roll your own cryptography.
What I took away...  A new understanding of how much I don't know about security and cryptography.

Dave Farley - Continuous Delivery

According to Dave... "Our highest priority is to satisfy the customer through early and continuous deliveries of valuable software."


  • Finished means in production.
  • Break down silos, Business, Testers, Developers, Operations, all need to collaborate more to make it work.

Feedback loops - starting with smallest and quickest:
  1. Unit Test - Code.
  2. Executable - Build.
  3. Idea - Release.
Principles:
  • Keep everything in version control - I agree.  Any tool that has a UI-only is to be avoided in my opinion.
  • Automate everything.
  • Build quality in.
  • If something is painful, do it more often.  If you release every 6 months because it's painful, the lesson is release every 2 months, then month, then week etc etc.  Don't start releasing every year, it will get worse.
  • Manual testers are far better at exploratory creative testing than repetitive regression.

Accidental benefits:
  • When debugging a long standing defect, if version control and deploy are fast and reliable, you can determine which build introduced it.  Binary chop through a list of builds until you have narrowed it down,
  • Automating everything can introduce auditing for free.
What I took away... Dave Farley knows this subject inside and out and his, Continuous Delivery is highly recommended.


Paul Simmonds - Identity is the new currency

Data is going from: 
  1. Internal.
  2. De-perimeterized.
  3. External Collaborations.
  4. Secured Cloud.
Interested or working with security on the cloud... you must read: