Linked Data & Business Intelligence – Is this a two sided coin?

I haven’t blogged much lately and there is good reason for that, firstly I’ve recently gone through a restructure and redundancy process, I still work for the council but have moved somewhere else as part of the redeployment process (more on this in a separate post).

However one of the things that remains the same is that I’m still lucky to get involved in very interesting projects and this creates many problems for me. 1) I try to create links between projects when none clearly exist, but when they do they make a lot of sense, 2) my mind is never really quiet for long periods to allow me to find inner peace and 3) there is so much to get involved in that I have to ensure that I stay focused on the now, whilst keeping an eye or two on the future.

I’m fortunate to be involved in some Linked Data work that going here, mainly thinking and planning first steps but it is really fascinating and I can see why Linked Data fans champion the need to do more – I really do see the value, but we need to actually provide it through real world application before some senior folk will really support it.

For those who don’t know what Linked Data is:

Linked Data describes a method of publishing structured data, so that it can be interlinked and become more useful. It builds upon standard Web technologies, such as HTTP and URIs – but rather than using them to serve web pages for human readers, it extends them to share information in a way that can be read automatically by computers. This enables data from different sources to be connected and queried
Via Wikipedia

We had a session today to learn more about Linked Data and the practical first steps we can take in order to start proving value not just internally but across some partnerships.

One of the others areas that I’ve been involved in but not so much on a practical level is looking at the Business Intelligence capability across the council.

For those who don’t know what business intelligence is:

Business intelligence technologies provide historical, current and predictive views of business operations. Common functions of business intelligence technologies are reporting, online analytical processing, analytics, data mining, process mining, business performance management, benchmarking, text mining and predictive analytics……..Often Business Intelligence applications use data gathered from a data warehouse. However, not all data warehouses are used for business intelligence, nor do all business intelligence applications require a data warehouse
via Wikipedia 

 

Now one of the interesting aspects for me is that there are similarities in these two areas which I think I need to explore further as I learn more about each over the coming months etc.

On face value they seem to be two sides of the same coin – Linked Data looking primarily externally to create meaning and context, whilst business intelligence looks across internal systems and applications to create meaning and context.

I would think it is fair to say that we are in the very early stages of each of these projects and it will take time for us to get to grips with how we want to see both develop.

But I can’t help but think that we need to bring these two agendas together somehow without compromising each project – perhaps through shared learning, data modelling, data cleansing and approaches to master data management.

I could be wrong and I maybe over complicating two very complicated areas already….but surely the outputs of the internal business intelligence tools should form the basis of our published linked data for others to consume.

I guess time will tell 🙂

 

Advertisements

15 Replies to “Linked Data & Business Intelligence – Is this a two sided coin?”

  1. Hey Carl,

    I think there’s definitely something to be explored here.

    With the understanding that you can have open data that’s not linked data; and linked data that’s not open data, it does seem that:

    Mapping and modelling key Business Intelligence (BI) datasets as Linked Data, and making available tools for working with Linked Data in the authority is a good way to manage some of the internal data-warehousing needed for good BI. (Linked Data; not necessarily open);

    In doing this, you will quickly find you need to publish various ‘linking’ datasets, code lists, taxonomies etc. to form the connecting points between your different models. As these are rarely sensitive, publishing these as Linked Open Data (and making sure citizens can also get at them as CSV/JSON/XML etc. through a conversion/ copy of the linked data API) is definitely a quick win;

    It’s then fairly straightforward to (a) start publishing additional datasets (the non-private inputs to BI, and the non-private outputs from BI) as linked data, and to (b) pull other linked datasets from third-parties into your BI process

    As you say – it’s worth being careful that an approach like this doesn’t compromise already complex areas. I think it’s BI that stands to be most complicated – as the tool-chains for linked-data are somewhat less developed in many cases than others – and any approach should be agnostic to exactly what format data is processed in as people tend to turn to their favourite tools/best tool for the job. You can of course, agree lists of codes to use in data modelling in the short-term, even if they don’t become linked data URIs and datasets right away… so having an eye on the linked data side of the BI data integration definitely makes sense…

    Will be really interested to hear how this develops. There’s certainly some interesting overlaps with the Knowledge Hub exploration of linked data and a data-hub to be explored if that’s not already on your agenda…

    1. Thanks Tim,

      Excellent comments and I certainly need to explore this further and look at how we can identify the quick wins.

      The knowledge hub is certainly an area I feel can help us encourage some aspects of managing and publishing data in more open and standard ways.

      There is huge scope to raise the profile of these areas as I believe they help inform the strategic commissioning agenda which most councils are proactively considering.

      If you are at localgovcamp I’d love to chat about this further with you.

  2. Carl,

    Couple of comments

    There are people working on ontologies ; finance system providers ( the thoughtful ones ) are working on how LOD will come from the system, or into it

    Personal data stores, services and eco-systems are also coming, fast down the tracks.

    Cheers,

    Alex

  3. Yes, I think you make a very good point: that business intelligence tools and linked data are tackling aspects of the same problem, namely making it possible to combine information from multiple sources to get a useful view of a particular question.

    As you say, output from internal business intelligence tools is likely to be a prime source of information to expose externally as linked data. I think there are also other ways that linked data and business intelligence can interact.

    While the transparency and accountability to the public aspects of linked/open data are interesting and valuable, I think potentially the most important application of linked data may turn out to be ‘professional’ use of data within the council (or other data-owning organisation) and between the various agencies that need to interact and collaborate to deliver services. But this purpose is also well served by a basic approach of open data availability.

    Linked data can of course be used totally inside the firewall as a data integration tool, but it’s greatest value is when it can be used to interconnect internal and external data. Using linked data to help you pull in external data to assist business intelligence is an important benefit.

    I reckon a good objective is to use linked data to set up a basic data infrastructure (interlinked identifiers for council assets, services, places and regions etc) that makes connecting up info from different systems easier. You don’t need to do it all up front – can pick a few areas of interest or specific datasets and build out from there.

    1. Thanks Bill,

      You make some excellent points and i agree that professional use of linked data will be very valuable.

      I think I need to think more about how linked data approaches can inform our data integration and master data management approaches to ensure that we are create a flexible and scalable approach and not something which is high maintenance and inflexible.

      1. Yes, working out a cost-effective way to keep Linked Data (or any other) versions of data up to date and accurate is the hard bit!

        In comparison to other approaches, linked data probably rates pretty well for flexibility – because the data schemas are themselves part of the data, the need to change software when the data structure changes is much reduced.

  4. “Life belongs to the living, and he who lives must be prepared for changes.” – Johann Wolfgang von Goethe
    Good luck in your new role.

  5. Great article Carl, and excellent comments from Tim and Bill. We’re looking to set up an observatory in Hampshire and I’m really interested in the extent to which linked data can support that work. Also how might tie in with our programme to publish open data. (And, of course, how all of that fits with the Knowledge Hub)

    1. The links with the knowledge hub are also something i need to think about in terms of opportunity…i’m assuming this will be easier…but obviously this needs to be checked.

  6. yes, am looking forward to getting hands on with the Knowledge Hub – beta should be starting any moment now

  7. Picking up on Alex’s point, personal data stores (PDS) could revolutionise how individuals control, and allow access to, their data. The paradigm change is putting the individual at the heart of this, and linking all other parties to the individual’s centralised data store. Need to overcome boring things like interoperability, security (not really boring I know!), standards, off shore storage of data, that kind of thing. But rather than, e.g., setting up NHS patient records within the NHS network, why not place patients’ records in the specific patient’s/individual’s data store held in the cloud somewhere. Could also contain education records, bank accounts info, credit card info, etc, suitably partitioned and protected. A single store which everyone else links to, with your permission of course. And why couldn’t this be provided by a social enterprise, rather than the likes of Apple or Amazon. I heard a great presentation recently from Simon Britton from http://www.mydex.org who is doing just that.

    I realise this is a separate point to the main focus of your post Carl, but inevitably these things are linked I’m sure!

    Plus, I haven’t really thought through yet the impact of all of this on big line of business systems in social care such as Care First. Do PDS remove the need for these big applications? Or do they feed each other. Not sure!

Leave a comment or share your thoughts...

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s