Bol.com is the largest online retailer in the Netherlands and Belgium and is still growing at a staggering rate. For gathering business intelligence (BI) insights we have a team of twelve BI engineers. The growth of data, users and complexity have forced our team of to rethink our traditional Oracle based data warehouse structure. Currently we are moving towards a more hybrid BI solution based on Microservices at a BIG data scale, using Hadoop, HBASE, Oracle, etc. In this presentation Debarshi and Mark will discuss the unique challenges we have faced and are still facing with petabytes of data along with design decisions, tooling landscape and architectural choices. We have created and explored various toolings and concepts for solving our problems. We discuss concepts pertaining to Data quality at big data scale, large scale batch scheduling, job monitoring, ETL processing, reporting, continuous deployment and team autonomy for BI at scale. Essentially, all the hacks to get our systems towards awesomeness. The presentation will guide audience through transformation of legacy (vintage) BI towards more scalable BI setup as well as mental and techincal block that came along the way.
Bio van Debarshi Basak & Mark Nijenhuis Debarshi started his career in year 2010 in India. Pursued Master at TU Delft in Computer science. Works as Data plumber at Bol.com, Plumbing data for business intelligence.
Mark has 17 years of experience in oracle, data warehousing and all kind of data related stuff. Working at bol.com since 2010. Experience with Service Dis-oriented Architecture.
