Big Data calls for the rise of the citizen developer

John Skovron has over twenty years of experience working with systems that run mega scale grid computing solutions. These are the systems used primarily at financial trading shops that are able to compute against loosely coupled, high variety and distributed data that most of the world wouldn’t ever consider as possible. When he slows down to talk about Big Data, his background in doing the impossible is a perfect fit for today’s struggles with ever-increasing amounts of data.

Citizen developers

John is passionate about the rise of the citizen developer, domain experts that have a hacker mentality, are quantitatively focused but don’t necessarily have the parallel programming, advanced statistics or data visualization skills that are found in the pure data scientist role. The biggest reason for this need comes from the serious shortage of the people who have all of the skills necessary to perform the work required of Big Data, mobile, cloud  and other ‘modern’ problems. Gartner defines the citizen developer as, “…an end user who creates new business applications for consumption by others using development and runtime environments sanctioned by corporate IT.” Most surprisingly, Gartner predicted in the past that 25% of new application development by next year will be done by citizen developers.

The citizen developer has a great deal of potential in a world where there simply aren’t enough data scientists and the challenges of getting value quickly from data won’t allow for long development cycles inside structured projects. Some would argue that the citizen developer is such an unstructured role that it doesn’t matter the formal education of its practitioners…a citizen developer can be self-taught either on their own or with the new breed of online teaching tools like code.org or coursera.

Impatient with IT

What truly enables the citizen developer with Big Data is the wide variety of platforms that allow the heavy lifting to be done behind the scenes and allow anyone to work like a data scientist. Great examples would be Datameer, Alpine Data Labs and TIBCO Spotfire, all applications that are extremely business-user friendly and don’t require teams of data scientists. You’ll find lots of citizen developers in marketing departments but also spread across other business functions that have grown impatient with IT”s cost and timelines.

Citizen developers offer IT departments a stark choice…do they enable or fight the trend for more business-savvy people to touch software code and the organization’s data?

Come hear John talk at Interop New York this week, where he’ll be presenting at the Big Data Workshop on the Big Data ecosystem that extends beyond just Hadoop clusters. We hope to see you there.