So, let’s talk architecture. I think we all know that architectures change over time as new things become possible. Those monstrosities of the 50’s and 60’s in London were built that way because new things became possible with concrete, and the cost of doing so reduced. Likewise in the 90’s and 00’s we saw more efficient, ‘connected’ buildings emerge. And today’s new buildings have evolved to be data driven, ‘smart’, sustainable ecosystems.

The same architectural evolution happens with technology. In the digital computing era we have seen first the mainframe (centralised access). Then Client Server (centralised control, distributed access) emerged and still remains the dominant paradigm in terms of volume. In the 00’s and 10’s we saw the evolution into ‘pipelines’, driven mainly by the big tech platforms. For example, Google Search is one giant pipeline/ process with words going in one end and money out the other. The Facebook Like button is the same. Same for LinkedIn; and anything else that is ‘feed’ based.

In the 2020’s we see the older technology architectures really struggling to cope with the vast explosion in connectivity, volumes of data transactions, endless requests for integration, and thus complexity. Client-Server in particular is, in my opinion, a legacy approach that cannot cope with the world we now live in. Some of the problems we all as individuals bump into every day now are symptoms of this:

• We each have a vast number of accounts, and no real means to keep track of them because we can only deal with each silo, one at a time (as the client)
• Sign up/ sign in/ authentication is increasingly a difficult experience
• Terms and conditions and privacy policies written separately for every silo are farcical, and somehow perceived as the best we can do
• There is no means for us to clean up after we have long since moved on; so our data exchaust just keeps on building

And if we look at this from the ‘inside an organisation’ perspective we see:

• Many internal data siloes with a need and desire to connect them (web site, CRM, e-commerce, digital marketing, adtech, martech, supply chain, HR etc etc)
• Data quality going through the floor due to the un-avoidable half life of data (which means that if you are not, or are not connected to the master data then your data deteriorates significantly over time on multiple dimensions).
• Compliance and data protection folks have their heads in their hands desperately hoping that their organisation won’t be the next in the vast collection of data breaches. I could go on…, but won’t.

Does anyone think we fix the above by doing more of the same? The practical reality is that we need to migrate to an architecture that can cope with and enables our current world of vast and ever-expanding data flows and uses. This was the case before the AI explosion, and is un-deniably the case during it.

I contend that what comes next is a network architecture in which:

1. Data control and management is distributed, each participant (node) in the network is an equal and is there voluntarily. This is the equivalent to ‘Server-Server’ in technical terms. Individuals will most likely be represented in this by data intermediaries and fiduciary AI agents.
2. The network is governed by a network agreement/ trust framework that sets outs its governance model and rules of participation.
3. There will be many networks; so we actually build a network of networks; with appropriate ‘crossing points’ through which data transitions from one network to another.

I’ve visualised this transition below.

‍

So what does that mean for Smart Data (as defined in the UK Government Smart Data Programme)? Actually, I think that becomes really simple given the above context. In The United Kingdom we should build The UK Smart Data Network (UKSDN).

• Innovative? Absolutely
• Difficult, not particularly if one accepts the necessary architectural requirements and finds a way to deliver on them

I think there is a good precedent. The CAIS network, hosted by Experian has been around for decades. The innovation behind it was ‘if we (retail financial services) share data safely and securely amongst ourselves then we can build capabilities that are greater than the sum of the parts. Sounds familiar…

Imagine we could do something similar, but with individuals as empowered nodes in the network. A network in which individuals are their own data controllers; ably supported by their AI agents to do the routine tasks and learn. The relationships in that mode become much more worthy of trust; and many improvements derive from that. That would be a lot easier than wrestling seven industry sectors into separate schemes, all of which revolve around humans/ individuals anyway.

So what do we need to do to build the UK Smart Data Network? I think that is:

1. Instantiate the project, ideally with the backing and buy-in from the UK Gov folks in the Department of Science Innovation and Technology who are leading the charge on smart data from the government and engaging industry perspectives.
2. Technology. There will be many choices around databases and user interfaces (wallets, AI agents, personal data intermediaries). But I would contend the core plumbing could be the JLINC protocol for two-way permissioned, inter-operable data exchange between cryptographically secure nodes/ agents on the network - with machine readable data sharing contracts and an audit log. If there is an alternate data exchange protocol out there that can improve on the above then let me know as it will be a hell of a capability and we should deploy it fast. My point is, JLINC is not a future capability; it is a here and now one built for massive scale.
3. Governance. The trust frameworks/ network agreements will need to be proposed and agreed (as they were with Open Banking, and we can learn a lot from that). This is probably not as hard as it may seem; the work being done in IEEE7012 (aka ‘MyTerms’) will publish soon and deliver a range of data sharing contracts/ privacy policies written from the individual perspective. These are coming from the ‘data should flow - appropriately’ perspective; not about lock-down; so a good start point for what amounts to an opt-in data exchange network.
4. Sandbox. There is a lot of change implied in the above; so people, organisations and regulators will need to ‘play’ safely to build knowledge on the proposed model and prototype. We already have the JLINC Sandbox; so happy to offer that into the mix. Others could be spun up as needs be.
5. Membership and Adoption. Finally once the above is in place then we’d need to drive/ facilitate adoption. This is where the UK.Gov work on Smart Data would be very helpful; driving adoption and data mobility across seven main B2C sectors for starters.

What about those individuals and organisations that don’t want to join such a network you may ask? That’s a perfectly valid choice on a case by case basis just as was the case for adoption of online banking, or smartphones. The process of adoption will take many years, beginning with the early adopters and eventually getting to the laggards who just love that client-server control and can cope with running on low grade data.

‍