A Capability Maturity Model for Research Data Management
Welcome to CMM for RDM » The CMM for RDM Blog » A Rubric for the CMM4RDM

A Rubric for the CMM4RDM

Last modified by Arden Kirkland on 2014/07/02 08:12

Jul 02 2014

Last fall I joined the team for the CMM4RDM as a research assistant, as an MLIS student at Syracuse University. I jumped in and quickly learned all I could, having previously had little experience with research data management (beyond trying to manage my own research data for various projects). As Jian, Kevin, and Charlotte worked to flesh out each section of the model, I was charged with proofreading and formatting the entire document, so I found myself constantly moving between the details of the description of each practice and the bigger picture of the model as a whole. The model takes its structure from the original CMM for software development (Paulk et al., 1993), broken down into thematic process areas as chapters, and then further broken down by the practice areas associated with the Commitment to Perform, Ability to Perform, Activities Performed, Analyzing Measurements, and Verifying Implementation. For the CMM4RDM, the last 2 are combined into 1 practice area. You can read more about the structure of the document at http://rdm.ischool.syr.edu/xwiki/bin/view/CMM+for+RDM/0.4+Structure+of+this+Document.

While working on the model, I found it helpful to create a table that places the practices on a matrix, by process area and practice area, to see the relationships across the document. You can view this as a PDF at http://rdm.ischool.syr.edu/xwiki/bin/download/CMM+for+RDM/Matrix/Matrix%2DCMMforRDM.pdf.

Screen Shot 2014-06-30 at 9.00.50 AM.png

Figure 1 – Structural Matrix of the CMM4RDM (view PDF)

We continue to have some internal discussions as to whether some sections are redundant, or whether they really belong under a different practice area. Your feedback, especially from applied use of the model, will be very helpful to clarify these areas moving forward.

Once the model was fully developed, I then found it useful to take the ideas from each practice and turn them into a rubric that looks at what each maturity level would look like for each practice area. At the end of the page for each practice area within a chapter, a table displays the rubric for that section. The full rubric is also available as a PDF at http://rdm.ischool.syr.edu/xwiki/bin/download/CMM+for+RDM/Rubric/CMMforRDM_Rubric.pdf.

Screen Shot 2014-06-30 at 8.23.04 AM.png

Figure 2 - Sample page from the Rubric for the CMM4RDM (view full PDF)

A next step will be to test the usefulness of this rubric for evaluating specific projects. However, it is important to note that this rubric is currently in a qualitative and not a quantitative format - each practice is not necessarily weighted evenly, so a quantified "grade" will not properly represent the maturity level of the project as a whole. Moving forward, if it is desired to turn this into a more quantitative format, work will need to be done to weight the different sections appropriately. Several other tools already employ assessment models, such as DMVitals by Sallans & Lake (2014), the Australian National Data Service Capability Maturity Guide (ANDS, 2011) and the Community Capability Model for Data-Intensive Research (Lyon et al., 2012), but they all take different approaches than the CMM4RDM, as discussed by Charlotte Flynn in a note within the model at http://rdm.ischool.syr.edu/xwiki/bin/view/CMM+for+RDM/Note+1.

I look forward to using this rubric to apply the principles of the model to some of my own projects, both as a researcher and as a data manager helping researchers. I’ve created a blank worksheet to help me do so, adding more rows to the full rubric to take notes about each specific practice: http://rdm.ischool.syr.edu/xwiki/bin/download/CMM+for+RDM/Blank+Worksheet/CMMforRDM_Worksheet.pdf.

 

Screen Shot 2014-06-30 at 10.14.09 AM.png

Figure 3 – Sample Page from the Blank Worksheet for the CMM4RDM (view full PDF)

While I fear many of the projects I am working on will be considered very “immature,” this aspirational model will help me to see the steps I need to consider, and in what order, to someday reach a higher level of maturity.

References

Australian National Data Service (2011). Research data management framework: Capability maturity guide. Retrieved from http://ands.org.au/guides/dmframework/dmf-capability-maturity-guide.pdf

Lyon, L., Ball, A., Duke, M., & Day, M. (2012). Community capability model framework. White Paper. UKOLN, University of Bath & Microsoft Research. Retrieved from http://communitymodel.sharepoint.com/Documents/CCMDIRWhitePaper-24042012.pdf

Paulk, M. C., Curtis, B., Chrissis, M. B., & Weber, C. V. (1993). Capability Maturity Model for Software, Version 1.1 (No. CMU/SEI-93-TR-024). Software Engineering Institute. Retrieved from http://resources.sei.cmu.edu/library/asset-view.cfm?assetID=11955

Sallans, A. & Lake, S. (2014).  Data Management Assessment and Planning Tools.  In J. Ray (Ed.), Research Data Management:  Practical Strategies for Information Professionals (Charleston Insights in Library, Information, and Archival Sciences).  West Lafayette, Indiana: Purdue University Press. Retrieved from http://books.google.com/books?id=qZStAQAAQBAJ&pg=PA87&dq=dmvitals&source=gbs_toc_r&cad=3#v=onepage&q=dmvitals&f=false

 

 

 


XWiki Enterprise 5.1-milestone-1 - Documentation