I was recently invited by Amazon to present at their Lean Cloud Event in London, to share some insights and experiences around our engineering journey so far. Contributing back to the community is something the whole engineering team here is passionate about so we were delighted to be part of the event.
The whole event was very enjoyable and informative. I was blown away by the amount of energy that was evident. It was inspiring to chat to so many people who are building really exciting start-ups and who were interested in our technology, methodologies and experiences.
If you’re curious about our AWS experience you can view the slides on SlideShare. Based on some great follow up questions, I figured I’d take this opportunity to expand upon some aspects of our business that folks found interesting.
The Technology behind SkillPages
We get asked quite often about the technology behind SkillPages – we are cloud based platform, based on AWS, and we run a variety of best of breed software on top of this. Essentially, we prefer to focus on building a really great platform that our members find useful – the infrastructure we leave to those that do it best. We like to consider ourselves technology agnostic and prefer to use the best technology to solve the problem. Our architecture is such that any of these elements can be easily replaced as technology advances and better solutions are found.
At the moment we are doing a lot of work with provisioned IOPS and High IOPS SSD backed instances, which is showing great promise on our Cassandra clusters. Interestingly enough, Cassandra is not our sole noSQL element; we are also leveraging DynamoDB to handle elements such as atomic counters. The key point is that AWS is a very cost effective platform if properly managed and there are a whole load of tools available to help with this, for example Bill Tagging.
At SkillPages we are relentless about leveraging our data to improve how we serve our members. Our motto is ‘measure everything’ – anything else is just opinion. We have a fantastic infrastructure in place right now, using EMR to crunch a vast amount of data and onward it to our BI toolsets for analysis and visualization. Although it looks simple enough on a slide, it did take a few iterations to get this just right!
In the presentation I reference ‘Boyd’s Law of Iteration’. For those not familiar with the history, I recommend reading a bit more behind Boyd’s theory - it’s a fascinating story based around two military aircrafts. Boyd gives rise to the principle ‘Speed of iteration always beats quality of iteration’. Some may consider this controversial, but a better way to consider this when growing your business is ‘Where you are today does not matter so much, compared to where you are going tomorrow’. Based on this we have found that iterating fast and measuring the data is something that works really well for us.
In terms of iterating, we always ensure to make provisions for paying down technical debt accrued along the way. We, along with every other company (including yours!), incur technical debt – it is inevitable. Anybody that says otherwise is either in denial (!) or not innovating fast enough.
Our Greatest Asset
There were a few questions around our team, specifically What size is the team? Our engineering team is 19 strong – a very talented bunch of people. Everybody codes, tests and deploys. This led into some really interesting questions around How we hire? and Where do we find people with AWS experience? In truth, we don’t necessarily, the main criteria is that we hire smart people who know how to solve problems and get things done. For us, our team is our greatest asset.
I hope you found this useful, if you have any specific questions don’t hesitate to get in contact.