Clarke Analytics

The Vision of the Smart Energy Grid – are we there yet, are we there yet?

As I look forward to my second year attending the IMR/I2E2 Energy Symposium in Cong, I take a look back at my past interactions with the energy sector. While not my primary business sector of interest, I have had opportunity to work in the sector over the years. Firstly, while with PwC, I was on the initial team that helped the ESB successfully de-regulate the electricity market back in early 2000 with the set- up of EirGrid. The market actually was able to go live, and stave off EU fines for the Irish Government, based on electricity generating and supply nomination file specifications that I was responsible for creating. A pressure cooker project if ever there was one! A few years later, while with EMC, I worked on validating global solutions in the smart metering space with Silver Spring technologies and our Greenplum Appliance.

Both the market de-regulation and spread of automated metering infrastructure across the world have been driven on by the promise of more efficient use of the worlds energy generating assets and more cost-effective and resilient service for end customers.

The figure below is one of my favorites in depicting this utopian end to end connected eco-system of entities and infrastructure. A cursory research into the state of the current energy market in the EU notes the huge costs of connected infrastructure implementations and some feedback from business folks in the energy space on their frustration with the lack of ROI (Return On Investment) in the large data infrastructures that have been put in place to act as the backbone for analytical insights.

 

As the symposium at Cong approaches, it would be great to get some feedback on 3 of the key areas that I think are still challenged in making the utopian vision a reality from a data analytics perspective.

Data & Infrastructure Fluidity

Over the last 10 years, there have been massive changes in the data infrastructure landscape. Having a consistent technology stack or architecture to support has been challenging. With recent announcements of smart metering rollouts still not fully in place, there is obviously still a way to go for the hardware infrastructure to support full smart metering to be in place. The software technologies to support data being available in a single, albeit distributed, memory space to support advanced analytics have been moving and adjusting also. Many a data lake has turned in to a data swamp, as fundamental functionalities required to make those lakes work like data governance and data lineage are only recently being added to many available complete lake stacks based on bitter experience. Is this your reality?

Business Focused Agile Delivery Methodology

Agile, Sprints, Stand-ups, Backlogs, Test-driven Continuous Development, Done-Done Burn up/down checklists are all the latest buzz words in relation to getting quality software delivered fast & to spec. But, I wonder if the product management function of the agile framework is sufficiently cognoscenti on the ultimate business goals and value realization utopia envisaged to the smart energy grid. This is hard, yet fundamental to gaining fast competitive advantage. Is this your reality?

Seamless Organizational Collaboration

This is fundamental to ensuring that any data analytics effort does not go off the rails. Most importantly, it must be sponsored loudly and championed from the very top of the organization. That is because it is hard and takes effort. I have some anecdotal evidence that this change agent can be hard to find in Irish organizations filled with 55+ aged senior execs not wanting to rock the boat or take the risk closer to retirement. I hope that is not true. Is this your reality?

 

About Dave Clarke:

Dave Clarke is the Founder and CEO of Clarke Analytics Ltd, which is a wholly owned Irish consulting company specializing in the area of data analytics development and data science consulting and training. The company was set up in October 2016 by Dave and his wife Helen. Dave brings many skills in key areas for customers in the data analytics space:

Dave has worked in the IT industry for 25 years in software consulting, management consulting, project, program, and engineering team management, solutions architecture, technology evangelist, software development and data science roles.

Over the last 10 years, Dave has been primarily focused on the data analytics space in Ireland, working with global pharma, med device manufacturers and health care research sectors primarily in the areas of data analytics strategy development and data analytics proof of value and research initiatives. His company provides professional services to companies in these areas, not just in the market sectors above, but also in supply chain, logistics, utilities, finance and insurance industries. Check out Clarke Analytics on www.clarkeanalytics.com and please contact for a quote on our services.