A Capability Maturity Model for Research Data Management
CMM for RDM » 4. Data Dissemination » 4.4 Process Assessment

4.4 Process Assessment

Last modified by Arden Kirkland on 2014/06/03 19:13

4.4 Process Assessment

Process Assessment includes Measurement and Analysis and Verifying Implementation. Measurement and Analysis describes the need to measure the process and analyze the measurements, and typically includes examples of the measurements that could be taken to determine the status and effectiveness of the Activities Performed. Verifying Implementation describes the steps to ensure that the activities are performed in compliance with the process that has been established, and typically encompasses reviews and audits by management and quality assurance.

Process assessment for data dissemination follows the general guidelines as stated in chapter 1. It should be pointed out that the assessment of the dissemination process area can be easily confused with the outcome assessment such as impact and usage of data. Assessment of the data dissemination process aims at establishing appropriate quantitative measurements so that through consistent data gathering on these measurements, the RDM personnel can assess the process systematically on a regular basis for continuous improvement. 

4.4.1 Measurement and Analysis

Assessment of the data dissemination process should stay focused on measurements that can tell how effectively and efficiently the process was performed. Example measurements include the time taken from data submission to release with full metadata description, number of venues used for dissemination, and the increase/decrease in data access that may be attributed to data dissemination efforts. 

Collecting data on the dissemination process is not always straightforward. For example, once a dataset is ready for dissemination, metadata has to be created and reviewed, rights terms and access permissions defined, and venues for dissemination organized. Some of these steps may take longer to complete (e.g., the rights terms may involve legal consultation) while others may be in the form of notes rather than quantitative data. Having tools and procedures for collecting the data will not only make the data collection efficient and consistent but also enable the process assessment to occur routinely, rather than on an ad hoc basis. 

4.4.2 Verifying Implementation

A higher level of capability maturity (level 4 or 5) requires that the implementation of policies and procedures be verified to ensure that the process is adequately executed with a reasonable degree of quality. For example, questions may be asked during the verification review:

  • Are data being shared?
  • Is the data archive accessible? 
  • Are confidential data secure?

Verifying implementation of policies, ability to perform, and activities performed provides the opportunity for RDM personnel to identify problems early and hopefully correct the problems early enough, before they become worse. 

Rubric

Rubric for 4.4 - Process Assessment
Level 0
 This process or practice is not being observed
No steps have been taken to establish procedures for measurement, analysis, or verification to ensure accessibility and security of data
Level 1: Initial
 Data are managed intuitively at project level without clear goals and practices
Measurement, analysis, or verification to ensure accessibility and security of data have been considered minimally by individual team members, but not codified
Level 2: Managed
 DM process is characterized for projects and often reactive
Measurement, analysis, or verification  to ensure accessibility and security of data have been recorded for this project, but have not taken wider community needs or standards into account
Level 3: Defined
 DM is characterized for the organization/community and proactive
The project follows approaches to measurement, analysis, or verification to ensure accessibility and security of data, as defined for the entire community or institution
Level 4: Quantitatively Managed
 DM is measured and controlled
Quantitative quality goals have been established including measurement, analysis, and verification to ensure accessibility and security of data, and practices are systematically measured for quality
Level 5: Optimizing
 Focus on process improvement
Processes regarding measurement, analysis, or verification to ensure accessibility and security of data are evaluated on a regular basis, and necessary improvements are implemented

<--Previous Page / Next Page -->

Created by Jian Qin on 2013/10/08 21:25

XWiki Enterprise 5.1-milestone-1 - Documentation