Big Data and Agile: The Perfect Marriage

Gino Marckx

Head of Agile Competency Center, EPAM Canada
Video

Big Data is 'in' these days. It’s one of those emerging technologies that many organizations feel the need to climb on board for. But just like with anything else, it’s important to understand how Big Data can help your business innovate and grow. Talking about Big Data without mentioning data science is really not possible. It’s the data science that allows organizations to make sense of the data that we are able to collect these days: what information is hidden in the terabytes of data representing click streams, social media interactions, sensory data points, all kinds of structured and unstructured data.

Using the scientific method, hypotheses around the information that is hidden in the data are proved or rejected and based on the outcome of validating the hypothesis, new hypotheses arise or ways of using that information emerge. Because of the uncertain nature of big data projects, it is crucial to be able to rely on a delivery method that allows the project to consume changing requirements or quickly change directions altogether. 

Agile provides such an adaptive approach to delivery.

The principles and practices that are collected under the Agile umbrella all focus on validating assumptions as early as possible in the delivery lifecycle, significantly reducing the risk exposure as the project continues. Every software engineering project piles up the assumptions one after the other. The entire planning process is based on the assumption that everything will go as expected, even the padding that is added during the planning is based on assumptions that the amount of padding applied is sufficient. The functionality that is implemented is based on assumptions that the functionality will indeed provide the expected business value.

The design and architecture is based on a whole bunch of assumptions and ultimately every line of code that is written is based on the assumption that the line of code does not contain any bugs. By delivering the work in small increments of working - even production ready - software, those assumptions are all validated early on. All code, design, architecture and requirements and validated every time a new increment is delivered, even the plan is validated as teams get real and accurate data around the progress of the project. But the early validation is not the only benefit that Agile brings, it also allows projects to learn from the feedback, take in new or changing requirements and quickly change direction when necessary, without changing the process at all.

Both the fast feedback and the ability to change direction are attributes that support the scientific method and help big data projects succeed. On the other hand, big data analysis can help agile projects to validate the assumptions about requirements even earlier. So the benefits really flow in both directions.

Watch our webinar to learn more why Big Data and Agile is a happy marriage of science and methodology when done right.

Hello. How Can We Help You?


Our Offices

  • Canada

    • Ottawa

      343 Preston Street,
      ON K1S 1N4, Ottawa
      Canada

      Map
    • Toronto

      5 Park Home Avenue,
      Suite 400,
      ON M2N 6L4, North York,
      Toronto
      Canada

      Map
      P: +1-416-591-4004
      F: +1-416-595-1551
    Learn more
  • Mexico

    • Guadalajara

      Periférico Sur #8110,
      Col. El Mante
      45609 Tlaquepaque, Jalisco
      Mexico

      Map
      P: +52-33-462-400-98
    Learn more
  • United States

    • Newtown, PA

      41 University Drive,
      Suite 202,
      Newtown, PA 18940
      USA

      Map
      P: +1-267-759-9000
      F: +1-267-759-8989
    • Bellevue, WA

      110 110th Ave. NE,
      Suite 310
      Bellevue, WA 98004
      USA

      Map
    • Cambridge, MA

      One Mifflin Place,
      Cambridge, MA 02138
      USA

      Map
      P: +1-267-759-9000
      F: +1-267-759-8989
    • Conshohocken, PA

      101 East 8th Ave,
      Suite 201,
      Conshohocken, PA 19428
      USA

      Map
      P: +1-484-382-1300
    • Los Angeles, CA

      11601 Wilshire Blvd,
      Suite 350,
      Los Angeles, CA 90025
      USA

      Map
    • Mountain View, CA

      465 Fairchild Dr,
      Building B,
      Suite 221,
      Mountain View, CA 94043
      USA

      Map
    • New York, NY

      24 West 25th Street,
      5th Floor,
      New York, NY 10010
      USA

      Map
      P: +1-267-759-9000
      F: +1-267-759-8989
    • San Francisco, CA

      222 Kearny Street,
      Suite 308,
      San Francisco, CA 94108
      USA

      Map
    • Washington D.C.

      7901 Jones Branch Drive,
      Suite 400,
      McLean, VA 22102
      USA

      Map
    Learn more
  • Austria

    • Vienna

      Nottendorfer Gasse 11,
      1030 Wien
      Austria 

      Map
    Learn more
  • Bulgaria

    • Sofia

      69 Bulgaria Blvd.,
      Infinity Tower,
      1404 Sofia
      Bulgaria

      Map
      P: +359-700-20-273
    Learn more
  • Czech Republic

    • Prague

      City Tower building,
      Hvezdova 2b,
      Prague 4
      Czech Republic

       

      Map
      P: +420 22 888 28 23
    Learn more
  • Germany

    • Frankfurt am Main

      Franklinstrasse 56, 
      60486 Frankfurt am Main,
      Germany

      Map
      P: +49-69-31019090
    Learn more
  • Hungary

    • Budapest

      Corvin Offices I.
      Futó street 47-53,
      Budapest, H-1082
      Hungary

      Map
      P: +36-1-327-7400
    • Debrecen

      Bethlen Street 3-9,
      Debrecen, 4026
      Hungary

      Map
      P: +36-52-999-485 / 45050
    • Szeged

      Felső Tisza-Part 25,
      Szeged, 6723
      Hungary

      Map
      P: +36-62-808-013
      F: +36-62-550-655
    Learn more
  • Ireland

    • Dublin

      Alexandra House,
      The Sweepstakes,
      Ballsbridge, Dublin 4,
      D04 C7H2
      Ireland

      Map
      P: +353-(0)-1-631-9280
    Learn more
  • Netherlands

    • Delft

      Delftechpark 37j
      2628 XJ Delft
      Netherlands

      Map
      P: +31 20 241 6134
    • Schiphol

      The Base B
      Evert van de Beekstraat 104
      1118CN Schiphol
      Netherlands

      Map
      P: +31 20 241 6134
    Learn more
  • Poland

    • Gdańsk

      Grunwaldzka 472D,
      Olivia Six,
      80-309 Gdańsk
      Poland

      Map
    • Katowice

      Chorzowska Str. 148
      40-101 Katowice
      Poland

      Map
      P: +48-12-222-02-02
    • Krakow

      Opolska Str. 114
      31-323 Kraków
      Poland

      Map
      P: +48-12-222-02-02
    • Warsaw

      Al. Jana Pawła II 23,
      Atrium International,
      00-854 Warszawa
      Poland

      Map
    • Wroclaw

      Ul. Piotra Skargi 1,
      Budynek A,
      50-082 Wrocław
      Poland

      Map
    Learn more
  • Sweden

    • Göteborg

      Lilla Nygatan 2,
      411-09 Göteborg
      Sweden

      Map
      P: +46-31-146-550
    • Stockholm

      Kungsgatan 50,
      111 35, Stockholm
      Sweden

      Map
    Learn more
  • Switzerland

    • Glattpark

      Boulevard Lilienthal 2,
      8152 Glattpark (Opfikon)
      Switzerland

      Map
      P: +41-43-500-21-49
    Learn more
  • United Kingdom

    • London

      114 Middlesex Street,
      London, E1 7HY
      United Kingdom

      Map
      P: +44-203-514-0027
    • Manchester

      Tower 12,
      18-22 Bridge Street,
      Manchester, M3 3BZ
      United Kingdom

      Map
      P: +44-203-514-0027
    Learn more
  • Australia

    • Sydney

      Suite 2.14
      Level 2
      Zhen Building
      33 Lexington Drive
      Bella Vista NSW 2153

      Map
      P: +61-2-8310-8272
    Learn more
  • China

    • Guangzhou

      Unit B01, 23/F,
      Yuexiuxinduhui Building,
      No. 236, 6th Zhongshan Road,
      Yuexiu District, Guangzhou,
      China 510180

      Map
    • 广州

      中国广州市越秀区
      中山六路236号
      越秀新都会大厦中座 23楼 B01室
      邮编510180

      地图
    • Shanghai

      Room A11, 6th Floor, Block C,
      666 East Beijing Road,
      Huangpu District, Shanghai,
      China 200001

      Map
      P: +86-21-53080606 +86-21-53085450
    • 上海

      中国上海市黄浦区
      北京东路 666号
      科技京城C区 6楼 A11室
      邮编200001

      地图
      P: +86-21-53080606 +86-21-53085450
    • Shenzhen

      3/F, Block 5, Vision Shenzhen Business Park,
      9th Gaoxin South Road, 
      Shenzhen Hi-tech Industrial Park,
      Nanshan District, Shenzhen,
      Guangdong, China 518057

      Map
      P: +86-755-36899008
    • 深圳

      中国广东省深圳市
      南山区高新南九道
      威新软件园5号楼3楼
      邮编518057

      地图
      P: +86-755-36899008
    • Suzhou

      Building 12, Creative Industrial Park,
      328 Xinghu Street,
      Suzhou Industrial Park,
      Suzhou, China 215123

      Map
    • 苏州

      中国江苏省苏州市
      苏州工业园区星湖街328号
      创意产业园内12号楼
      邮编215123

      地图
    Learn more
  • Hong Kong

    • Hong Kong

      198 Wellington Street,
      17F, Central
      Hong Kong

      Map
      P: +852-5808-6018
    Learn more
  • India

    • Hyderabad

      10, 11 & 12th Floors,
      Salarpuria Sattva Knowledge City,
      Plot No. 2, Phase - 1,
      Survey No. 83/1,
      Raidurgam Village,
      Serilingampally Mandal,
      Hyderabad, Telangana - 500081
      India

      Map
      P: +91-40-47979900
    • Pune

      SmartWork Business Center Pvt Ltd,
      Suite 8, Level 1,
      West Wing, Nyati Unitree,
      Samrat Ashok Road,
      Yerwada, Pune - 411006,
      Maharashtra
      India

      Map
      P: +91-20-4913-6025
    Learn more
  • Singapore

    • Singapore

      5 Shenton Way
      UIC Building, #10-01,
      Singapore (068808)

      Map
      P: +65-6911-6888
    Learn more
  • United Arab Emirates

    • Dubai

      EPAM Systems FZ-LLC
      Dubai Branch,
      Building 16, Office 241,
      Dubai Internet City,
      PO Box 501929, Dubai
      United Arab Emirates

      Map
      P: +971-4-568-3569
    Learn more
  • Armenia

    • Yerevan

      15 Khorenatsi Street,
      Elite Plaza Business Center,
      0010 Yerevan
      Armenia

      Map
      P: +374-10-60-00-65
    Learn more
  • Belarus

    • Brest

      6A Masherov Avenue,
      224030 Brest
      Belarus

      Map
      P: +375-162-52-5268
      F: +375-162-50-9888
    • Gomel

      80 Rechitsky Avenue,
      246012 Gomel
      Belarus

      Map
      P: +375-17-389-0100, ext. 54079
      F: +375-232-70-50-31
    • Grodno

      87B Gorkogo Street,
      230005 Grodno
      Belarus

      Map
      P: +375-17-389-0100, ext. 69011
    • Minsk

      1/1 Academician Kuprevich Street,
      Suite 110,
      220141 Minsk
      Belarus

      Map
      P: +375-17-389-0100
      F: +375-17-268-6699
    • Mogilev

      19 Cosmonaut Street,
      Suite 510,
      212009 Mogilev
      Belarus

      Map
      P: +375-17-389-0100, ext. 1001
    • Vitebsk

      11-a Stroitelei Avenue,
      Suite 311,
      210032 Vitebsk
      Belarus

      Map
      P: +375-17-389-0100, ext. 54433
    Learn more
  • Kazakhstan

    • Astana

      8 Auezova Street,
      Office B,
      010000 Astana
      Kazakhstan

      Map
      P: +7-7172-475-970
      F: +7-7172-688-774
    • Karaganda

      58/3 Ermekova Street,
      100009 Karaganda
      Kazakhstan

      Map
      P: +7-7212-93-01-01 +7-7212-93-01-00
    Learn more
  • Russia

    • Moscow

      9th Radialnaya Street,
      Building 2,
      115404 Moscow
      Russia

      Map
      P: +7-495-730-6362
      F: +7-495-730-6361
    • Izhevsk

      150 V. Sivkova Street,
      426000 Izhevsk
      Russia

      Map
      P: +7-3412-271882 +7-3412-271337
    • Nizhny Novgorod

      30 Poltavskaya Street
      603089 Nizhny Novgorod
      Russia

      Map
    • Ryazan

      16 Gogolya Street,
      390035 Ryazan
      Russia

      Map
      P: +7-4912-93-57-33
    • Saint Petersburg

      22/2 Zastavskaya Street,
      MegaPark,
      196084 Saint Petersburg
      Russia

      Map
      P: +7-812-611-1094
      F: +7-812-611-1094
    • Samara

      21 Michurina Street,
      443110 Samara
      Russia

      Map
      P: +7-846-200-0970
    • Saratov

      37 Tankistov Street,
      410019 Saratov
      Russia

      Map
      P: +7-8452-692-981
    • Sergiev Posad

      7 Akademika Silina Street,
      141313 Sergiev Posad
      Russia

      Map
      P: +7-496-547-11-39
      F: +7-496-547-11-39
    • Togliatti

      31E Yubileynaya Street,
      445037 Togliatti
      Russia

      Map
      P: +7-495-730-6360, ext. 47650
    • Tver

      8 Kolodkina Street,
      170002 Tver
      Russia

      Map
      P: +7-4822-630-070 +7-4822-630-071 +7-4822-630-072
      F: +7-4822-630-073
    Learn more
  • Ukraine

    • Dnipro

      17A Volodymyra Monomaha Street,
      49000 Dnipro
      Ukraine

      Map
      P: +380-56-790-5651
    • Kharkiv

      33G 23 Serpnya Street,
      61072 Kharkiv
      Ukraine

      Map
      P: +380-57-728-06-22
    • Kyiv: Registered office

      28 Fizkultury Street,
      03150 Kyiv
      Ukraine

      Map
      P: +380-44-390-5457
      F: +380-44-390-0861
    • Kyiv: Visitors office

      14B Kudryashova Street
      03035 Kyiv, Ukraine

      Map
      P: +380-44-390-5457
      F: +380-44-390-0861
    • Lviv

      45 O.Stepanivny Street,
      79018 Lviv
      Ukraine

      Map
      P: +380-32-242-4642
      F: +380-44-390-5458
    • Vinnytsia

      51 Ovodova Street,
      21000 Vinnytsia
      Ukraine

      Map
      P: +380-432-551-294 +380-432-551-275
      F: +380-432-551-293
    Learn more