Given so often the person who is the applied statistician on a college campus is asked to conduct professional development training, I am in the midst of a series of postings on what should be covered during such training for administrators responsible for data driven decisions. (I introduced this topic in this post https://statisticalsage.wordpress.com/2014/07/24/data-driven-decisions-doing-it-right/)
Central to effective data driven decisions is high quality data, which requires an articulated Data Management Protocol. In short, the goal of the Data Management Protocol should be to effectively communicate to everyone in the organization how data are being collected, organized, coded, and stored in a manner that maximizes integrity and increases the usefulness of the data set. (Note: I addressed issues of data integrity in a prior post https://statisticalsage.wordpress.com/2014/08/07/assuring-data-integrity/ )
Let’s face it, it is not possible for an organization’s leader to be well versed in all areas. As a result, the Data Management Protocol is often left up to someone with such a specialty. However, that does not relinquish the administrator’s responsibility from assuring the data management protocol is actually effective, leading to increased data usefulness and integrity. Thus, when training such administrators, the focus on data management protocol has to be on how to tell if it is working or not. Critical to this is “human capital” … do you have the right people in the right positions to manage your data?
Here are some tips to evaluating your “human capital”
- The Right People Matter: Place people in charge of data management who have the right training and experience. This is not the place to save on resources by hiring a person with no experience or formalized experience. Bad data means bad decisions.
- Make use of Data Issues: Even with the best plan and best people, data issues will still arise. Seize on such situations as a way to evaluate the effectiveness of your team. An effective team will be looking at what went wrong to cause the problem. They will identify the root of the problem, so the fix will take care of the immediate problem and all future problems. Many times such issues require an improvement in the data management protocol. Let’s say you are working on workload evaluation for a faculty member who ended up getting underpaid. The quickest fix would be to go to HR and adjust the pay. The proper fix would be to determine what was in the data file that generated this error in the first place. In such an example, I discovered that a new class had been entered into our data system incorrectly. It was coded as a 1 credit lecture class (amounting to a 1 credit workload hour) instead of the 1 credit science laboratory (that amounts to 3 credits of workload hours). If all that happened was the person’s pay was hand adjusted, then the next time a faculty member taught the same lab, the same error word have been made. The identification of this error also precipitated an evaluation of the coding of all science lab classes, identifying and fixing other problems at the root. This situation should have also yielded changes to the protocol of how courses were entered (and by whom). Be cautious of people managing your data who seek quick fixes over permanent fixes.
The right team will automatically create a data management protocol that includes the following:
- Specification for Data Organization including a protocol for how data files are named, the structure of the files (that eases data analysis of varying types), and file naming and storing for ease of retrieval.
- Security Mechanisms to protect the data including limiting access to the base data files.
- Back up Protocol for data storage failures.
- Document Data Details. Identify the method for assuring the details associated with the data (Who, What, Where, When, How, Why) is stored with the data. Data can come from lots of different sources (on campus/off campus; enrollment services/academic affairs). Knowing the source of data is critical in making decisions, as is knowing if the data are self report data, which careers with it a great deal of systematic error, or direct measures.
- Consistent Coding of data protocol (e.g., CBM for College of Business Management; CAS for College of Arts and Sciences)
- Metadata Collection identification (when appropriate for web site and other electronic presence). Understanding who is visiting your web site and what they are looking at is filled with great information and is often an under utilized source of information at many college campus and other organizations.
Now, if you are working with an organization that doesn’t have a Data Management Protocol, you are not alone. In one study, it was found that 22% of major organizations who needed to be making data driven decisions had no Data Management Protocol, and of those who did, only 40% were actually enforcing them. In other words, the plan was just collecting dust on the shelf! (Sorry, I read this the other day, and can’t locate the source.)
Creating a Data Management Protocol is not as daunting as it seems.
- As with all major changes to an organization, a creation of a Data Management Protocol is most likely to be successful with overt support from Organizational Leaders.
- Conduct an assessment of where the organization currently is. The use of surveys and focus groups makes the most sense.
- What data do people need, and for what reasons?
- Who needs access to the data and why?
- Who currently “owns” that data? Who is currently responsible?
- When is the data needed (e.g., for external reports or internal decisions)
- What is currently in place? What is working; what is not? What is truly effective?
- Once you know where the organization currently is with regard to data needs and management, seek out examples of Data Management Protocols from other similar institutions (remember, I’m teaching at a state sponsored university where such sharing of information among our sister schools is not only possible but expected. In the for profit world, this is less likely to be possible).
- Data Management Protocol are truly best created by experts and not in large committees. Given that Data Management Plans help facilitate the interface between data collection, storage, and use by humans, please make sure to include as one of your experts your Quantitative Psychologist. He or she will not only understand the uses of data and how formatting and ease analyses, but such a person will also understand how to aid the humans who will be using such data in the future.
- Once created, pilot the plan before fully implementing it. Expect that revisions will be needed.
- Don’t forget to have an assessment plan put into place for future data driven improvements.
The quality of decisions is limited by the quality of the data. A well implemented Data Management Protocol will increase the quality of the data, decrease the man-hours for obtaining data, and will facilitate the use of data to make great decisions. It is worth the investment!
I am including a few links to short articles that go into more detail regarding Data Management.