Data collection and turning that data into an asset
Good day, all. This issue is a continuation from Issue 15, I encourage to read it before reading this issue, it’ll help your understanding of this one as it continues on the theme of data and its importance in Digital Transformation.
I recently recorded another podcast with Kadia Francis, the Digital Jamaican. You should check out her blog and podcast series, really good stuff. I’ll let you know when it’s out, but we talked about some of the topics in the last few issues of this newsletter. I had a great time recording and it’s something I’d like to do more of in the future. I’m open for propositions. Stay tuned.
On to the issue.
Source : dailyworth.com
Turning data in to a benefit
In the last issue, I noted that data is the new oil in the digital economy, and showed a few simple strategies for finding and analysing data, where data can emanate from and finally a couple of simple tools to use in linking data sets between business applications to break down the silos of data you have in your applications. What I didn’t get in to, is the finer details of those data and what is important when you are collecting and analysing them.
Big Data, you’ve all heard the term, with many definitions as to what exactly Big Data is. I don’t know actually know if there is the “definitive” definition or not, but it certainly doesn’t mean large quantities. This is a myth, and one that persists. Sure, Big Data can be huge in size, and there are data sets being used in everyday activities like weather pattern prediction, that are absolutely gigantic in size, but it’s more pertinent definition hinges on the fact that it is unstructured.
Remember the definition of unstructured data from the last issue:
Unstructured data represents any data that does not have a recognizable structure. It is unorganized and raw and can be non-textual or textual. Unstructured data also may be identified as loosely structured data, wherein the data sources include a structure, but not all data in a data set follow the same structure.
Additionally, Big Data usage and cost has become perfectly aligned with cloud infrastructure, particularly in the database space. Previously companies would have had to invest massively in database infrastructure and data processing infrastructure, mostly locally stored with all the overheads that that requires, cooling, electricity, backup infrastructure, fire safety and large personnel costs. These were often huge investments leaving them available to only the (rich) few. In today’s paradigm, cloud database infrastructure is not only cheap, but offers the possibility to any organisation to use all but the most powerful compute systems in existence — the most powerful being reserved for education, research and military purposes mostly. And remember cloud infrastructure has a plasticity that on-premise infrastructure doesn’t — need 10 processors now but at peak volume, 100 are needed. No problem in the cloud world. Not only that, but analytics add-ons for those cloud databases are available directly from the database supplier and increasingly from other suppliers that have USPs that go above and beyond those of the database vendor. We’ll take a look at some of these in future issues. The thing to remember nowadays is that big data doesn’t necessarily have to come with big costs. And, in some cases you don’t even have to pay a penny to store it. I highlighted this in the last issue but didn’t explicitly mention it in this context; but places like data.gov (US), Facebook, Twitter and many others, do all the back-office stuff for you.
Other data that is unstructured and being generated at a dizzying rate is data the records or tags things with the longitude and latitude, location data. All modern smartphones record location data and many cameras do similarly when taking photos. Clearly the use cases for location data are endless and are sometimes used in surprising fashions, and not necessarily nefariously. If we ourselves recorded our location on an ongoing basis, it would probably provide insights into our own behaviours that could help us with issues as diverse as health — imagine if you discovered you traversed daily a particularly polluted zone and the you have been doing this for years, correlating that with your asthma incidents — to things as mundane as optimising journeys for the best fuel usage.
I briefly mentioned in the last issue that the type of sensors that have been used since the early 2000s are providing “…a near constant avalanche of data…”, in what has now become known as the Internet of Things, or IoT. IoT is no being deployed right throughout the entire value chain in highly digital businesses, and this influx of data is providing insights that were previously inconceivable. At a large Microsoft Conference, called Inspire, I remember a couple of years ago a discussion and demonstration of the power of data collection for ThyssenKrupp:
ThyssenKrupp Elevator wanted to gain a competitive edge by focusing on what matters most to their customers: reliability. Drawing on the potential of the Internet of Things (IoT) and Microsoft technologies by connecting their elevators to the cloud, gathering data from their sensors and systems, and transforming that data into valuable business intelligence, ThyssenKrupp is vastly improving operations—and offering something their competitors do not: predictive and even preemptive maintenance.
Read that statement again, …something their competitors do not (currently) have. Predictive and preemptive maintenance. In other words, allowing ThyssenKrupp to schedule preventive maintenance and even predict lift failures before they happened. If you can get past the obvious advert for Microsoft, this quick video gives you a great idea of what I’m describing about data as an asset. Again, the possibilities are endless for data collection and data analysis. Microsoft has even gone further, and christened the terms Intelligent Cloud and Intelligent Edge and defined them as:
The intelligent cloud is ubiquitous computing, enabled by the public cloud and artificial intelligence (AI) technology, for every type of intelligent application and system you can envision.
The intelligent edge is a continually expanding set of connected systems and devices that gather and analyze data—close to your users, the data, or both. Users get real-time insights and experiences, delivered by highly responsive and contextually aware apps.
Source: microsoft.com
But all this data collection and data storage isn’t and couldn’t be anything useful if it isn’t shaped and presented to provide insights, or what has become known as over the last 20 years, Business Intelligence (BI). I alluded to this earlier in this issue, but advanced BI tools and a few simple skills in Data Science have become a hot topic for most businesses in their Digital Transformation journey. Take a look, for example, how many jobs and how well-paid Data Scientist roles are currently. There is a real scarcity of good data analysts in business roles.
Rather fortuitously for me, this last Wednesday, the NY Times published a long but fascinating article called How Data (and Some Breathtaking Soccer) Brought Liverpool to the Cusp of Glory about the use and subsequent valorisation of data to help Liverpool FC out of the blues of the last 10 years or so. Transforming them into one of the top teams to beat in Europe, with some even saying that this might be the start of a new era for Liverpool FC, like its previous run of form from 1975 to 1990.
For four years, from 2008 to 2012, Graham advised Tottenham. The club was run by a series of managers who had little interest in his suggestions, which would have been true of nearly all the soccer managers at that time. Then Fenway bought Liverpool and began implementing its culture. That included hiring Graham to build a version of its baseball team’s research department. The reaction, almost uniformly, was scorn. “ ‘Laptop guys,’ ‘Don’t know the game’ — you’d hear that until just a few months ago,” says Barry Hunter, who runs Liverpool’s scouting department. “The ‘Moneyball’ thing was thrown at us a lot.”
Graham hardly noticed. He was immersed in his search for inefficiencies — finding players, some hidden in plain sight, who were undervalued. One afternoon last winter, he pulled up some charts on his laptop and projected them on a screen. The charts contained statistics such as total goals, goals scored per minute and chances created, along with expected goals. I was surprised to see Graham working with such statistics, which he had described to me as simplistic. But he was making a point. “Sometimes you don’t have to look much further than that,” he said.
And that’s the point. Some simple statistics may help you better understand a taxing issue and hence aid in your resolution of the problem. Thoroughly recommended.
The risks of data collection
I think it remis of me if I didn’t at least talk briefly about the dangers of data collection. And I’m sure you don’t need me to tell you that holding a lot of data, and more specifically, personally identifiable data, is a risk that you need to assess in your business. In fact, this is the real reason for the General Data Protection Regulation ’s (GDPR) being.
If we look at the intention of the GDPR, it’s that data should be treated like a controlled substance. A controlled substance being something like weapons, drugs and so forth. Clearly the general public shouldn’t be purchasing, storing, using and selling on these types of substances and products. The GDPR makes us look at personally identifiable data in the same way; we shouldn’t need to collect, store, process and/or sell-on these data. Unless…
Where the similarity continues with controlled substances, arms and drugs can actually be legally bought and sold, but under (in most civilised countries at least) strict controls. GDPR, like the controller substance trades, ensure that we justify why we need the data, for what purpose it’s going to be used in explicit terms and a promise that it won’t be used for other purposes. Additionally, GDPR prevents the resale of that data, again, unless there is specific consent from the affected users.
When you develop your data collection strategies, you need to think quite carefully about the data you need and if it is classified as personally identifiable data and plan accordingly, from consent forms, operations structures to ensure that it isn’t used outside the initial scope and that it is secured accordingly and that you can provide proof of its ultimate destruction. Think in terms of the entire lifecycle of the data; creation, collection, processing, transformation, encryption, depersonalisation, restitution, backup and destruction.
Remember, according to GDPR you are responsible for proving this. I’d like to go further in to this subject in the future, especially since we’ve had numerous examples of data care failures from the likes of Facebook and Cambridge Analytica, and I’m currently researching the subject. Soon.
Reading List
Source: StockSnap (Pixabay)/ict-pulse.com
ICTP 056: Building Caribbean-relevant software applications, with the team from Rovika
A talk with Dennison Daley and Manish Valechha from Rovika, a software house based in Montserrat. They have developed apps for Montserrat and BVI governments and have potential for wider-ranging projects throughout the Caribbean.
Source: E-Estonia
20th William G. Demas Memorial Lecture to focus on digitisation
If you weren’t aware, Estonia has been on a national Digital Transformation journey for a number of years now. Estonia even offers full digital citizenship , or e-Residency. Estonia was the first in the world to achieve this. Starting 1997, Estonia implemented their e-Governance strategy and have continually innovated and developed new digital services for their residents, both physical and e-Citizens. They have fully embraced technologies like Blockchain — in 2008 in fact! — and have ambitious plans for future services like intelligent transportation.
In this, the 20th William G. Demas Memorial Lecture, Calum Cameron of Estonia is to speak about “Transforming to a Digital Society: Principles and Challenges”. Unable to be there in person, I’m hoping I can get a feed or at least a video of the speech. If you have any information to help me, let me know please. Thank you.
Source: epthinktank.eu
Europe, a unique partner for Caribbean
I alluded to some of this in the podcast with Kadia, but this essay gives a good overview of how Europe can help the Caribbean and why it’s important for the Caribbean to work together facing increasing competition from around the world. Definitely worth reading even if it’s a little on the propaganda side from a High Representative of the Union for Foreign and Security Policy.
The Future is Digital Newsletter is intended for a single recipient, but I encourage you to forward it to people you feel may be interested in the subject matter.
Thanks for being a supporter, have a great day.
