I want to take you back about twelve months, so that I can set the scene for you. My team is still dealing with the fallout from a merger that forced together two very different tech stacks and teams that don’t know each other. On top of that we are still in the middle of what turns out to be a two-year mega project that has sucked time and energy from almost everyone.
In this blog post I will describe how we set up automatic deployment of our rabbitmq-client library to Sonatype’s Nexus Repository Manager.
Continue reading “Automatic deployment of opensource projects to Maven Central”
At Betfair our read services are struck with billions of requests per day, they are not evenly distributed either. These requests will arrive in huge spikes of traffic during key sporting events, putting our customer facing services under huge pressure during sustained time periods throughout the day. We develop our systems to cater for this demand, keeping true to our latency SLA’s all the while operating without downtime. Unlike comparable trading platforms used in the financial world, we don’t have the option of closing trading at 5pm – sporting events occur around the clock, every day.
When we talk about read services, we are referring to anything that is presented, in real-time to customers – either through the API or via our online channels. Notably, our price read services. They were the first to move to the streaming model. If you are not familiar with financial trading, price read services present ‘ticks’ on a market to our customers – billions of them. Ticks are price/volume pairings for a given selection on a market. See below.
Betfair is the largest on-line betting exchange in the world and has seen tremendous growth in the past few years. One of the fundamental problems that comes with growth is that the technology that drives the exchange has to scale in line with not just current growth rates but also the forecast growth over next X months/years.
What also complicates the matter is that growth is a function of many variables, including but not limited to number of users, jurisdictions, market offerings and the list goes on. So the goal post is constantly moving for us. Whilst it is a massive challenge, in some ways it is also a nice problem to have. As noted in a previous post, the current exchange back-end architecture, whilst still proving its worth everyday, puts some limitations on rapid scale outs and on our ability to deliver new capabilities to maintain the edge in a highly competitive market. We have been slowly migrating the entire back-end architecture to be event driven, and we recently achieved another milestone in this path. A new stream of core exchange activity, that provides the lowest level of transactional activity ticks that occur in the exchange, in near real time, was deployed live after some painstaking work over a few months. This is a significant step forward for us as it essentially opened up options for us to migrate a lot of our internal service applications to source the exchange activity data from the stream, instead of relying on some legacy sources which were highly reliant on a very heavily used database. More significantly, it also allows us to build similar capabilities exposed to our end users. The detailed benefits of this, can probably be discussed in another entry, but this is no doubt a step change for us. Continue reading “Exchange Streaming – The journey so far”
So, we have decided to resuscitate our technology blog. But this time we’re bigger. You may know that both paddy power and betfair have joined forces to create one of the world’s largest online gambling businesses, with more than 7,000 staff. With this comes complexity and many technical challenges, which will give us plenty to share.
The aim of this site is to share our expertise, knowledge and our technology stack with you technology enthusiasts.
In the coming weeks will be posting interesting stories, videos and pictures around how we overcome some of our biggest problems at paddy power betfair.