Data Quality and Data Governance Frameworks

What are they and do I need both?

"How do a data quality and data governance framework relate to each other?”I get asked this question quite frequently and I think it’s a really interesting one, so I’d really like to help you get to the bottom of it. I think the reason it comes up is because people have been doing data quality and worrying about data quality for many more years than they have data governance.And so, they feel very strongly that there are two different frameworks in action. Another common misconception is that the two are the same. This may come from a lack of understanding of what data governance really is, so let’s break it down…..

Is data governance the same as data quality?

The very short answer is no. Data quality is the degree to which data is accurate, complete, timely, and consistent with your business’s requirements. Data governance, in very basic terms, is a framework to proactively manage your data in order to help your organisation achieve its goals and business objectives by improving the quality of your data. 

Data governance helps protect your business, but also helps streamline your business's efficiency. It ensures that trusted information is used for critical business processes, decision making, and accounting. And so, if you think about it, data governance vastly provides a fabulous foundation for many data management disciplines, its primary purpose is to manage and improve your data quality.

To put it in much simpler terms, if data was water then…

-          Data Quality would ensure the water was clean and prevent contamination

-          Data Governance would make sure the right people had the right tools to maintain the plumbing.

So, why would you want two frameworks relating to data quality?

The simple answer is you wouldn’t. This really isn't a question about how you align two frameworks. You should only have one framework and data quality and data governance should be working in harmony with one another – not against or in opposition.

Data governance and data quality rely very much on each other, I usually describe the relationship between them as symbiotic, as their relationship is based on mutual interdependence. Therefore, of course, you need both! You would not want to do one without the other if you want to successfully manage and improve the quality of your data in a sustainable manner.

Sadly, in my experience, some organisations do not yet fully understand that you do need to do both. Whilst you rarely (if ever) come across a company that is implementing a data governance framework without the intention to improve data quality, it is fairly common for organisations to commence data quality initiatives without implementing a data governance framework to support them. Unfortunately, this leaves many data quality initiatives as merely tactical solutions that only have short-term results.

And, it doesn’t matter whether you call it data quality or data governance (because let's face it, some people really react badly to the term data governance) as long as it gets your business users engaged and understanding what that framework is about.

So, let's just have one data quality framework which encompasses the roles and responsibilities around data, and then there is nothing to go wrong, no duplication, no gaps between two different frameworks. Make this simple and make it sustainable.

You can see the video I originally did on this topic here and if you've got any questions you’d like me to address in future videos or blogs, please just email them in to questions@nicolaaskham.com.

Data Governance Interview with Abel Aboh

I have known Abel through DAMA UK for many years now and have always been impressed by his passion for Data Governance, Data Management and for making complex things simple! So I decided it was time to ask him to share some of his valuable insights with you by asking him to do an interview for my blog.

How long have you been working in Data Governance?

About 7 years now, fully working in the Data Management space.

Some people view Data Governance as an unusual career choice, would you mind sharing how you got into this area of work?

Sure! Growing up, I had the opportunity to experience in Shell Royal Dutch Company, the value people, technology, and data bring to businesses. As a result, when I applied to study at the university, I wanted to study, a joint degree Human Resources (HR) and Technology. Due to the Chartered Institute of Personnel Development (CIPD), graduate membership accreditation requirement, I opted to study HR alone. My HR working experience, gave me a unique perspective about people and organizations. I am passionate about people, and the role they play in an organization. After the completion of the London 2012 Olympic and Paralympic Games, which I was part of the delivery team. I joined the HR Solution team in BAE Systems Naval Ships Business Unit. The company gave me the opportunity to join a Transformation project. One of the mandates for the project was to deliver Data Management (Governance and Quality) as a business capability for the Type 26 programme. This my first opportunity to work in the Data Management role. I was accepted for the role because of my business, people, and technical acumen. However, my passion for data and the opportunities data and technology bring was recognized. The Data Management Transformation project was successfully delivered from Proof of Concept (POC) to Business As Usual (BAU).

What characteristics do you have that make you successful at Data Governance and why?

This is a great question; I need to think about it a bit. I would say, willingness to learn, flexibility and adaptability to new environment and technology, attention to detail, ability to quickly understand people (emotional intelligence) and organization, able to understand the structure of power, influence, and control within team and organization. The ability to influence people positively, the ability to solve problem and provide practical solutions, and make complex things to become simple. Finally, I would say my work ethics, discipline and perseverance – have thick skin. In Data Management, having a ‘thick skin’, be adaptable, and able to influence people positively are essential attributes. I like Aristotle’s Rhetoric ancient Greek treatise called the ‘Art of Persuasion’ – Ethos, Pathos, and Logos. I apply it to myself, why because it is relevant for Data Management practitioners to demonstrate the three. Personally, it is important to communicate your value propositions and the value Data Management can bring to any organization.

Are there any particular books or resources that you would recommend as useful support for those starting out in Data Governance?

It is important for those starting out, to ensure they read the Data Management Book of Knowledge (DAMA Book).

They should read Keith Gordon’s book, Principles of Data Management and Robert S. Seiner Non-Invasive Data Governance book.

It is important, to read books like Games People Play by Eric Berne, Politics and Turf Wars by Patrick M Lencioni and Leading Change by John P. Kotter.

I do find Nicola Askham, Lara Gureje, Chris Bradley, Nigel Turner, Sunil Soares and Peter James Thomas resources useful.

There are various useful resources online such as Simon Sinek, Harvard Business Review (HBR), Mckinsey Global Institute, TED Talks, TDAN.com, LinkedIn Data Groups and BrightTALK webinars.

Finally, I will recommend my book title: 10 Tips to Successfully Deliver Data Management – (writing it now, so watch out for it, coming out later this year).

What is the biggest challenge you have ever faced in a Data Governance implementation?

This is difficult question because People, Communication and Change Management are the biggest. They are the biggest, but they provide incredible opportunities as well. Therefore, I love the Data Management profession – the challenges are great, but the opportunities are greater! As a result, you can be both effective and efficient. You can create and deliver tangible and intrinsic value from operational, functional, emotional and social values in the organizations.

Is there a company or industry you would particularly like to help implement Data Governance for and why?

I like to work in companies, where my daily responsibilities, ties into a higher meaning, to a purpose higher than myself. I like to work across industries, sectors and where possible countries. I would like to implement Data Management in some of the FTSE 100 and Fortune 500 companies. It is not about the list, but because they create massive impact across their supply chains, customers, industries, and sectors. It is not about implementation, but “successful” implementation of Data Management. Helping the company, to know, trust and confidently use their data. Subsequently, this can help the company, make money, save money, manage and minimize risks. Successful implementation of Data Management in these companies can enable high data standards and practices, data investment and leadership, data management literacy etc. – which is good for the profession.

What single piece of advice would you give someone just starting out in Data Governance?

Jump into the ‘data trench’ and get your hands dirty! Change is constant - it is important to understand and appreciate Data Management is an art and science. As a result, human psychology and relationship is very much important. The story of the three men digging a ditch - it is a valid lesson for Data Management practitioners which I do like to share. Yes, the three men had different answers for the question asked ‘why they are digging the ditch’ - the goal remain the same working to build the Cathedral. Yes, there are various aspect of Data Management – the goal should the same! Remember, the profession needs you, so make your contributions, as best as you can. Enjoy yourself, but if you are not passionate about it, find something else to do quickly.

Finally, I wondered if you could share a memorable data governance experience (either humorous or challenging)?

Let me think, oh I remember one, it was a challenging conversation I had with a senior colleague (in Data Management you will have challenging conversations). The colleague said, Data Management is not important (useless) – the business does not need the Data Management team to operate. Calmly, I made the point on why the Data Management and Data teams particularly useful for the business. For the business to know, trust and use their data confidently to make business decisions – Data Management plays a critical role because fundamentally Data can be a critical asset for the business – so governance is important!

You can find out more about Abel and connect with him on LinkedIn.

Interview Questions For A Data Governance Manager Role

Someone recently asked me what questions I recommend asking in interviews for candidates for a Data Governance Manager role. It's been a couple of years since I helped a client with the recruitment process. I remembered a couple of questions that worked well but thought that since the data community are so supportive that this would be a great topic to collaborate on.  I asked on LinkedIn for people to share their favourite/recommended questions with a plan to create a great resource for both those recruiting and those preparing for Data Governance job interviews and this blog is the result.

Please feel free to use and share this resource:

‘Data Governance and you’ questions

• How did you come to be in the Data Governance arena?

• Give me your Data Governance elevator pitch.

•  How do you measure the success of Data Governance initiatives?

• Give me an example of how you went about implementing governance before and what you would do differently this time.

• How you get buy in for the Data Governance programs and how to onboard multiple streams for the data governance initiative

Skills and attributes questions:

• Talk to me about a couple of examples from your professional life where your 'desert crossing stamina' helped you to drive an initiative again and again to success despite being knocked down every other step 

• Give one example of how you overcame the obstacle of a fellow employee who was not interested in what you were trying to say ... even if you didn't win them over, what approaches did you try …

• Are you an emotional or a logical person? Then ask them to explain their answer.

‘Data Governance and the company’ questions

• What are the components / related pillars of Data Governance? Which would be most appropriate for this organisation and why?

• How would you use Data Governance to balance the business need to leverage our data and our compulsion to operate within the confines of regulatory requirements?

• What is your 30/60/90 plan for this position / company?

• What could Data Governance achieve in this company?

Data Governance Knowledge Questions

• What is Data Governance? Explain it to me as if I had never heard of it before.

•  Why is Data Governance important? /

• What is the difference between Data management and Data Governance?

• What does it look like when Data Governance is well executed?

• Who is responsible for Data Governance? Do you perceive Data Governance to be an IT driven process or a business driven process and why? What are the pros and cons of both? 

• How is data localisation going to affect the future of Data Governance?

• Explain a sample Data Governance road map for an organisation.

I hope you have found this useful.  Good luck with your recruiting or job hunting and if you have any questions you would like added to this list please add them to the comments below!

How to Successfully Implement a Data Governance Tool

A while ago I wrote a blog about things you should consider when choosing the right software to help facilitate your Data Governance initiative, but once you have selected and purchased the tool do not assume that everything will now “just happen”. 

One of my clients was worried (and rightly so) that it was at this point of the project that mistakes could be made which would impact the successful implementation of their Data Governance tool.  I thought my advice to her may help others too:

Technical implementation considerations:

Firstly you need to understand exactly what support you will get from your chosen vendor so you can plan what additional support you may need for implementation.

Then make sure that you agree who is going to manage the technical implementation of your tool. Is it going to be an in-house project team or are you going to engage a systems integrator? If the former is the plan, you need to liaise with the vendor to be very clear on what technical skills training they have available. What do they recommend to make sure that your team are suitably skilled before starting the implementation?

 If you're going to use a third party to implement the tool, make sure you do due diligence to ensure that they understand the tool and have significant experience in implementing it. I have worked with organisations where a consultancy has been employed and they stated that they had experience in the tool.  However, it became clear that while the consultancy as a whole may have had the required experience, the consultants working for that particular client did not have any experience and were learning on the job.  This caused unnecessary delays and poor advice on what was and was not possible with the tool.

I also recommend focussing on one area or functionality of the tool for the initial implementation. Just because the tool has lots of features that doesn’t mean you need to implement everything at once.  Choose the most needed functionality and implement that first, then look to implement other features as needed.  Remember, at this stage, this is about giving your business users a tool to help them do Data Governance, not to confuse them with a complex tool and functionality they haven’t asked for. As your users become more comfortable with both Data Governance and using the tool you can implement more Data Governance requirements and tool functionality.

Post-implementation considerations:

It is never a good idea to implement a data governance tool over the whole of your organisation at any one time. So I recommend not seeing the implementation as a one-off project.

It is better to think of it as a phased process with the initial implementation being a pilot or trial. Once you have completed the pilot it is likely that the users and the Data Governance Team may want some changes.  This is common as you are introducing something new and not replacing an existing tool or process.  This makes it very hard to get your requirements exactly right on the first attempt.  So you may wish to make some tweaks to the setup of the tool before continuing a phased implementation across the whole organisation.

It could take a very long time to implement the tool fully.  You need to make sure that this is well planned and that you are constantly working out what the next phases are going to cover.

You also need to consider how you are going to keep the data in the tool up to date. I recommend that you have a regular review of the content, for example, an annual review where Data Owners look at the content for the data owned by them.  They can then either confirm that the definitions are still correct or, if necessary, provide updates to keep the tool up to date and useful for the business users.

How to roll out a data governance tool to Data Owners and Data Stewards:

As I mentioned in my previous blog about choosing the right Data Governance tool, it is essential that your Data Owners and Data Stewards (or at least a representative number of them) are involved in the initial implementation project. Often they have not asked for this tool and they do not react well to having the tool forced upon them.  It is vital that they are involved in the design stage, to make sure that it's set up in a way that is going to appeal to them and make them happy to use this new tool.

Even if your Data Owners and Data Stewards have been involved in the early stages, remember that doesn't mean they won't need additional briefing and training when the tool gets implemented.  I recommend having a section of your overall Data Governance Communications and Training plan dedicated to the implementation of your data governance tool.  This will include things like initial high-level briefings to explain what the tool is and why it will be useful to your organisation.  You will then need some specific focused sessions:

 ·     Sessions with Data Owners to tell them what they're expected to do with the tool and showing them exactly how to do it.

·      Sessions for Data Stewards which will be a little longer and more detailed as they will be doing the bulk of data entry and review of data in the tool.

Both sets of training need to be accompanied by some kind of user guide or aide memoir, to make it very easy for them to quickly check what they need to be doing once the training is over and they are using the tool for real.

Taking all the above into account may seem like a lot of undue effort when you just want to get on with implementing the tool, but doing so will make a huge difference over whether it is a success or not.

If you have other tips for a successful Data Governance tool implementation that I haven’t included above please let me know!

Data Governance Interview with Neil Storkey

Earlier this year, in the days before social distancing, I was lucky enough to catch up Neil over breakfast and he kindly agreed to be interviewed for my blog. Neil is an independent data evangelist who has worked in large multinational companies from the early years of the data adoption. He passionately believes that everyone is a data citizen or as he says a ‘Citizen Steward’.

He views Data Governance like safety, it stems from individual behaviour, and how we shape that to form the day to day activities embraced as a culture.

How long have you been working in Data Governance?

In 1991 I began my data career before Data Governance was even talked about.

Somewhere around 2008 Data Governance started to become more of a mainstream activity with the software vendors adding the word Governance to their sales pitches.

I have always represented the business side of data, advocating that business stakeholders must be leading and supporting data initiatives. If I take the simplest aim of Data Governance to apply a consistent lens or approach to the use of data, then the business must take the lead.

Some people view Data Governance as an unusual career choice, would you mind sharing how you got into this area of work?

I didn’t deliberately set out on the data path as a career, I kind of fell into it. I was working in Finance and realized I was spending all my time as a spreadsheet jockey, which made me question my value. I was fortunate to be asked onto a major finance transformation programme as the reporting lead where data through the migration was critical to my success. Many years of data migrations through a progressive roll-out, and a very good mentor, convinced me there was a fledgeling career in data.

It wasn’t until 2005 that my Data Management role was formalised as the global MDM manager on an SAP finance transformation programme. Across the world, Master Data Management was an emerging discipline and I was lucky to be able to network with like-minded early adopters. 

It was an exciting time to be part of something new.

Back then 90% of the focus was on the technology and IT was wrestling with adoption across their businesses. It was a hard sell to land data as a discipline, and that remains true today.

But at the heart of any data initiative is the need to articulate what it is you are trying to manage and how you measure whether it is working or not.  Data Governance wasn’t seen as an enabler to the business processes, more a compliance and control regime which business areas could choose to adopt. To overcome these hurdles Data Governance needed a business lens applied with a focus on behavioural change.

So, in 2006 I started to develop the processes that would help the business to adopt and embrace Data Management. We now refer to this as a culture change, but it is a ‘hearts and minds’ challenge.

What characteristics do you have that make you successful at Data Governance and why?

Passion, integrity, honesty, resilience, patience, adaptability, storytelling, simplicity, and being able to talk to various stakeholders in the language that they feel most comfortable. I call it ‘talk business’.

Once you come to the realization that this is about changing people’s perceptions about data, how they contribute to its management and how they would benefit from making those changes, your approach becomes far more tactile. I cannot understate the importance of developing soft skills. And like any relationship, you must adapt your style to different personalities. For example, at C Level, the message needs to grab their interest in the first 30 seconds which means presenting a concept, with language that supports that message.

I always put myself in the position of the recipient and try to anticipate what I would want to know and ask, how they think, their pet subject, things that would influence a positive discussion.

I use an ice breaker unrelated to the data narrative because Data Governance will challenge their beliefs and I am trying to develop a relationship that creates trust and ultimately influence.

At the end of the day, each of us develops our own styles through trial and error. Go with those that you feel most comfortable.

Never underestimate the power of WIIFM, what’s in it for me. Understanding those personal drivers of your stakeholders and how they would benefit from Data Governance will be fundamental to your success. 

Are there any particular books or resources that you would recommend as useful support for those starting out in Data Governance?

I’m not a big reader, but in 2006 Jill Dyché and Evan Levy published, at that time, an inspirational book called Customer Data Integration. This has been the only data book I have ever taken to heart because of its narrative. Jill is a wonderful storyteller where she brings to life the data challenges. If you ever get the opportunity to talk with Jill or listen to her, go out of your way to do so.

What is the biggest challenge you have ever faced in a Data Governance implementation?

Getting started. We hear the saying ‘they just do not get it’ and use that as an excuse for not landing our data message. Looking under the covers I normally find that they do get it but data is not high on their list of priorities, or they have been subjected to the technology bias too often, or they cannot see the value in dedicating energy to a vague concept.

My biggest challenge has been turning that supertanker. Convincing stakeholders who are either disinterested or openly negative to the changes being proposed to establish a company-wide data discipline. Remember changing a culture requires commitment.

Is there a company or industry you would particularly like to help implement Data Governance for and why?

Company or industry for me is no different. Data is data and its management are broadly the same process.

However, I am a little different in that I look to a Chief Financial Officer (CFO) as the ultimate consumer of data, they would benefit directly through a well oiled and efficient data discipline. Harks back to my days as a spreadsheet jockey. Give me better data that I can trust.

Many people would disagree with my target audience, and to be honest, 5 years ago I would have agreed with them.

My rationale is that Data Governance starts predominantly through the management of master data. These are the foundations of every business process, customer, supplier, material, people etc. Every process executed in business has either an explicit or implicit financial impact that lands in finance. Much of the master data is touched by a finance process, for example, customer credit, material costing, supplier bank details, payroll.

Therefore, by inference finance really does have ‘skin in the game’ when it comes to consistent trusted data. Why would you not at least start the data journey in finance? 

What single piece of advice would you give someone just starting out in Data Governance?

Keep it simple. The saying ‘think big, start small’ really rings true to Data Governance. You want to take your stakeholders on a journey of discovery and enlightenment, not a slog up Mount Everest. 

There is no right answer, just many paths with potentially different outcomes. Choose wisely.

Finally, I wondered if you could share a memorable data governance experience?

My first day in a company as the Global Data Governance Manager I attended the inaugural Data Governance Executive Forum, only to be told by my boss that he was not able to attend and that I should chair the meeting.

My first day.

I didn’t know the people, their subject areas, what had gone in the past, even the format of the meeting. This was to be my introduction into the world’s largest multinational of this industry.

I was petrified.

I learnt a great deal about the people, but more importantly about myself. In the room were a group of supporters, a group of antagonists with the remainder ambivalent.

By the end of the 3-hour meeting we had achieved a consensus on the way forward for Data Governance, the challenges I would have to overcome and most importantly the frequency in which I would sit down with them one on one over a coffee.

The outcome was the embryo of Data Governance that would ultimately get established and span the entire company.

On reflection, if I had made a mess of that first-day induction, Data Governance would have been consigned to the ‘failed project’ bin.

The Communication Perspective

Good communications have always been vital for successful Data Governance, but with most of us working from home, communicating our message well has never been more important. Justin York often supports me with my clients and I am so pleased that he agreed to share some advice on good communication: 

We live in uncertain times where most of us have to adjust to navigating the online world of business. For this reason, clear communication is even more important in everything we do, particularly where businesses are trying to keep functioning with the majority of their staff working from home. While there is excellent technology available for professionals, which enables them to conduct operations and maintain communication, this may come with reduced effectiveness.

In these circumstances, it becomes even more important for the messages sent to your colleagues to be clear and unambiguous. This can be a real challenge, particularly if the technology you have doesn’t allow for the use of camera. You should not assume that your audience has any prior knowledge of the subject area you are trying to convey, as this may lead them to misinterpret it according to their own personal experience. Therefore, providing sufficient context in any messages sent will help to assure more effective communication.

Even outside of these challenging times, communication around subjects such as data governance can prove difficult, as some find quite dry and uninteresting. The issues around individual perspective and context are still relevant. I have worked in several large to multi-national companies, delivering change around data governance, where many individuals seemed hesitant to engage in this process. This is predominantly because they had their own viewpoint on what it will mean; often influenced by rumour (the opinion of others on the topic) or by personal experience in other organisations.

In addition to the need for senior leadership buy-in in data governance, which as we all know is key to success, we also need to engage other people in the organisation to do the things we need them to do. The key is engaging them from their own point of view (or perspective) and providing essential context to them around what we expect of them. Specifically, this can be achieved by:

§  A clear introduction the topic aligned to their business area.

·       Giving examples of the consequences of not following the initiative suggested and, as we like to say, present them with their own ‘data horror stories’.

·       Providing sufficient context to allow for an accurate interpretation of what is expected; detail such as ‘we don’t want you to get your hands dirty’ or perhaps ‘this is a leadership / decision maker role’ can engage senior leaders

·       Expecting both negative and positive feedback. Although we may prefer the latter, if we get a lot of the former then we haven’t quite gotten down to the actual pressing issue, which prevents them from stepping forward and making progress.

The message is everything. Without a clear, concise, unambiguous message then there will always be a degree of difference in how its received. The reality is that we always apply our own perspective to any message we receive, and that can colour our judgement of what has been said.

When on the receiving end of messages, I always feel it best to stop for a couple of seconds and just consider what was said, who said it and try and put myself in their shoes to get a better idea of the direction that is required.

Also, when receiving messages, we may adopt a particular ‘lens’, which will colour our view of what we are reading, for example:

·       We might treat the words as facts and accept them as such; if we do this, we tend to have no emotional attachment to them and thus we process them as they are said and we wait for the next instructions.

·       We perceive them as a personal attack; we might receive words that are directed at an audience to varying degrees as only meant for us. That can feel like a stinging assault on our emotions and we might react to those words with disbelief, though we might, after consideration, accept them as true.

·       We might treat the words in a cavalier manner e.g. well that’s not directed at me and thus the response might be ‘so what’.

When people receive communications, it is essential that we the communicator can gauge their reaction. Obviously this is much easier when the communication happens face-to-face. I personally have found that, in terms of data governance, there are three main responses:

§  “I fully understand and I’m onboard, let’s get on with it!”

·       “I’m far too busy to be engaged in this task, but I am happy to delegate it to a junior member in my team?”

·       “That’s nothing to do with me, it’s a problem for IT, I don’t own any data!”

I’m quite sure that many of you will also have experiences the same or similar responses!

The reality is that it falls to us to make clear what we are talking about, what we are trying to achieve and, most importantly, what’s in it for them; after all, who wouldn’t want quicker response times, with more accurate data!?

So in the troubled times we are living in, we need to be even more cognisant about the communications that we send, whether that’s work related or personal. Over distance and without direct interaction, perspective will come to the fore and who knows what a message might turn into.

You can find out more about Justin here.

Data Governance Interview with Theerachai Aurprasertwong

It was an honour to be asked to travel to Thailand to deliver training last December. I had the most amazing three days with Theerachai and his wider team. I was thrilled when he agreed to be interviewed for my blog, to share some of his experiences with you.

Theerachai is experienced in data governance and data architecture with more than 15 years of experience in the financial services industry. Outside of work, he is a father of a 6-year-old son and enjoys playing badminton, horse riding, archery and ski-ing.

How long have you been working in Data Governance?

Totally, 10 years. Five years at the business consulting firm. The other five years at the current work (Krungsri Bank).

Some people view Data Governance as an unusual career choice, would you mind sharing how you got into this area of work?

When I worked as a data architecture under the enterprise architecture team, data quality was one of the most frequent and most important issues that we handled in the project. So, I stepped in, little by little.  I’ve fixed both short-term and long-term DQ issues. Along the way, I realized that data quality can’t be fixed on a project basis. It needs dedicated structure and framework to keep it in the good shape. A few years later, I decided to work full time to build data governance from scratch at Krungsri Bank. That was when the real data governance journey began.  

What characteristics do you have that make you successful at Data Governance and why?

You need good common sense and always to be open to ideas and knowledge from outside. When you are stuck at something and you research on the internet, some concepts / discussions / articles seem impractical and unrealistic. However, if you apply them with your common sense, it can be very useful in certain situations.   

Are there any particular books or resources that you would recommend as useful support for those starting out in Data Governance?

Books and articles on the internet are great.  To be honest, I could not understand what Data Governance was after I read my 1st Data Governance Book.  Learning by doing is my best approach. If you have a budget, you can engage an expert from time to time to do a health check on your data governance framework.

What is the biggest challenge you have ever faced in a Data Governance implementation?

The biggest challenge is to turn the concept into reality. All those concepts online can look like the dreams of people from Mars!  Great communication may not enough, you need a practical way to apply data governance over time.

Is there a company or industry you would particularly like to help implement Data Governance for and why?

Many years ago, Data Governance was popular only in financial services and the telecommunication industry.  Now, every organization in which AI/ML or data analytics are the core competency need a strong data governance practice. Moreover, it cannot be constructed in a few months or years. It takes time and evolution.  

What single piece of advice would you give someone just starting out in Data Governance?

Data governance is not a sexy job compared to data science or data analytics. It is not IT project, it is the business matter. However, it will pay out great benefits to the organization and yourself over time.  

Finally, I wondered if you could share a memorable data governance experience?

Some data fields are very contentious. We have had a meeting with 20+ data stewards and we spent 4 hours trying to get an agreement on “Who is our active customer?”. We couldn’t get an agreement that day. Everyone felt exhausted and unproductive. I couldn’t remember exactly how we accomplished the first version for the data dictionary. However, it was the tipping point that shapes us today.  These days, those people are now our influencers/promoters who help us to build and sustain data governance.

Do you need Data Governance over a Data Lake?

There continues to be a lot of excitement about data lakes and the possibilities that they offer, particularly about with analytics, data visualizations, AI and machine learning. As such, I’m increasingly being asked whether you really need Data Governance over a data lake.  After all, a data lake is a centralised repository that allows you to store all your structured and unstructured data on a scalable basis.

Unlike a data warehouse, in a data lake you can store your data as-is without having to structure it first.  This has resulted in many organisations “dumping” lots of data into data lakes in an uncontrolled and thoughtless manner.  The result is what many people are calling “Data Swamps” which have not provided the amazing insights they hoped for.

So the simple answer to the question is yes – you do need Data Governance over data lakes to prevent them from becoming data swamps that users don’t access because they don’t know what data is there, they can’t find it, or they just don’t trust it.  If you have Data Governance in place over your data lake, then you and your users can be confident that it contains clean data which can found and used appropriately.

But I don’t expect you to just take my word for it; let’s have a look at some of the reasons why you want to implement Data Governance on data being ingested into your data lake:

Data Owners Are Agreed

Data Owners should be approving whether the data they own is appropriate to be loaded to the Data Lake e.g. is it sensitive data, should it be anonymised before loading?

In addition, users of the data lake need to know who to contact if they have any questions about the data and what it can or can’t be used for.

Data Definitions

Whilst data definitions are desirable in all situations, they are even more necessary for data lakes.  In the absence of definitions, users of data in more structured databases can use the context of that data to glean some idea of what the data may be.  As a data lake is by its nature unstructured, there is no such context.

A lack of data definitions means that users may not be able to find or understand the data, or alternatively use the wrong data for their analysis.  A data lake could provide a ready source of data, but a lack of understanding about it means that it can not be used quickly and easily. This means that opportunities are missed and use of the data lake ends up confined to a small number of expert users.

Data Quality Standards

Data Quality Standards enable you to monitor and report on the quality of the data held in the data lake.  While you do not always need perfect data when analysing high volumes, users do need to be aware of the quality of the data. Without standards (and the ability to monitor against them) it will be impossible for users to know whether the data is good enough for their analysis.

Data Cleansing

Any data cleansing done in an automated manner inside the data lake needs to be agreed with Data Owners and Data Consumers. This is to ensure that all such actions undertaken comply with the definition and standards and that it does not cause the data to be unusable for certain analysis purposes— e.g. defaulting missing date of births to an agreed date could skew an analysis that involved looking at the ages of customers.

Data Quality Issue Resolution

While there may be some cases where automated data cleansing inside the data lake may be appropriate, all identified data quality issues in the data lake should be managed through the existing process to ensure that the most appropriate solution is agreed by the Data Owner and the Data Consumers.

Data Lineage

Having data flows documented is always valuable, but in order to meet certain regulatory requirements, (including EU GDPR) organisations need to prove that they know where data is and how it flows throughout their company.

One of the key data governance deliverables are data lineage diagrams. Critical or sensitive data being ingested into the data lake should be documented on data flow diagrams.  This will add to the understanding of the Data Consumers by highlighting the source of that data.  Such documentation also helps prevent duplicate data being loaded into the data lake in the future.

I hope I have convinced you that if you want a data lake to support your business decisions, then Data Governance is absolutely critical.  Albeit that it may not need to be as granular as the definitions and documentation that you would put in place for a data warehouse, it is needed to ensure that you create and maintain a data lake and not a data swamp!

Ingesting data into data lakes without first understanding that data, is just one of many data governance mistakes that are often made. You can find out the most common mistakes and, more importantly, how to avoid them by downloading my free report here.