CRKN responses to Libraries and Archives Public Consultations – Ottawa, October 4, 5 1. What is your mandate and who are your members? The Canadian Research Knowledge Network is a partnership of 75 Canadian universities dedicated to expanding digital content for the academic research enterprise in Canada. Through the coordinated leadership of libraries, researchers and administrators, CRKN undertakes large-scale content acquisition and licensing initiatives in order to build knowledge infrastructure and research capacity in Canada’s universities. (Please find our current three-year Strategic Plan attached in Appendix 1.) 2. From your collective perspective, what challenges or issues are most prominent for your organization today? • • • • Economic pressure on member institutions results in members demanding more value from, and questioning the value of consortia-licensed resources. New models of content, purchase, and delivery are challenging the way we serve our members. Providing a long term preservation strategy for digital content purchased by members is an enduring challenge. Finding ways to coordinate with existing and emerging Canadian initiatives to leverage existing work, find efficiencies and support coordination in the areas of licensing, digitization, research data management, Open Access publication repositories, digital preservation, and research facilitation tools. 3. What do you anticipate as future challenges (by 2020), and how do you see you membership contributing to address them? • Procuring and licensing content for learning models that are only now being established; virtual learning environments, MOOCs and others. Our membership will address these by applying collaborative approaches to licensing and managing digital content. • Challenges in our financial ability to license content; because of a declining Canadian dollar, because of steady increases in the price of content, and because of reduced 1 budgets for member institutions. Our membership will address these by demonstrating the value of library resources to funding bodies and other stakeholders and by leveraging our buying power with content creators. • We anticipate challenges in our ability to preserve and provide access to the borndigital content that is being licensed and that is being created as an output of research in member institutions. We will address this challenge through partnerships and by leveraging existing initiatives and resources in Canada to create a Canadian knowledge infrastructure. 4. What would your organization be doing if funding were increased? And what are you not doing because of cutbacks or reduced funding? • CRKN would put a greater focus on acquiring new content and new kinds of content. Cutbacks to the budgets of Canadian universities and a lack of central, federal funding, limits the access that the Canadian research and teaching community has to research knowledge. CRKN’s smaller university members are increasingly challenged to participate in licenses of value to their faculty and students. • CRKN would more aggressively pursue collaborations for the development of a Canadian knowledge infrastructure to provide better discovery for Open Access content being created by Canadian universities together with commercial content and reliable preservation for all digital content being licensed by Canadian universities. (Please find our Integrated Digital Scholarship Eco-system Concept diagram attached in Appendix 2). 5. How does your organization relate to Library and Archives Canada (LAC)? Or, in your view, how should LAC relate to major archival and librarian organizations? CRKN currently has an arm’s-length relationship with Library and Archives Canada. We are partners in a project to digitize Canadian Heritage material, but this partnership is entirely transacted through Canadiana.org. In our view, LAC should take a leadership position with Canadian archival and library organizations. Projects of national interest in the library community, appeals for federal funding for digital content to support research, plans to create a national research knowledge infrastructure, platforms for the hosting of research data, the advancement of open access initiatives should all enjoy the support of LAC. LAC could help to ensure that resources are not being used to duplicate efforts, but are being used to build robust and enduring programs and services. 2 SERVICES 1. How would you describe the services Canadians, including Aboriginal Canadians and new Canadians, are currently receiving from libraries and archives in Canada? CRKN: • It seems like many large urban PL’s have great new Canadian programs but this is not our area of expertise. Services for Aboriginal Canadians and new Canadians outside of major urban centres seem underdeveloped. 2. Libraries are currently hybrid operations, constantly pulled toward traditional services by many core users and pulled, equally, by a concern for relevancy from other users and potential users. What issues are libraries facing as they try to make the transition to new service models? CRKN: • Shift from print to digital environment forcing a need to change access and preservation models • Prioritization of budgets ; making choices between where to invest resources. • Lack of formal communication channels between libraries and archives nationally may result in redundancy – missed opportunities to share work, expertise, and resources when transitioning to new service models. 3. How do libraries and archives measure outcomes of their service and community impacts? CRKN: • Within our member community there is a continuous focus on metrics and measurement on digital resources, utilizing vendor or platform supplied usage analysis. One set of measurements that we supply annually is our VITaL Dashboards (attached in Appendix 3) • CRKN through the self-study undertaken for the Platform Outcomes Measurement Study for the Canada Foundation for Innovation, attempted to measure the impact of digital licensing on the research community in Canada. Although the impact is difficult to quantify, some analysis was provided and several anecdotal reports were provided by researchers about the impact that CRKN had on their increased access to content for research. (Please see the POMS self-study pages 23 – 29 attached in Appendix 4 and see the POMS Expert Panel report in Appendix 5 for further examples of anecdotal and quantitative outcome measurements) • It is our experience that input measurements are easier to develop and implement than outcome measurements. 1 4. Are libraries the appropriate institutions to catalog, store, and provide access to research data? If not, which institutions should provide these services? CRKN: • Yes, libraries are the appropriate institutions in partnership with research organizations, and other producers of data (i.e. federal, provincial, municipal gov’ts) to ensure shared standards, interoperability and broad access to research data. COMMUNITY OUTREACH AND AWARENESS 1. Would Canadians know of, or understand, the contribution you make to library/archival service in Canada? CRKN: • As a library consortium, average Canadians would not know of, or understand, the contribution our organization makes to library/archival service in Canada. Hopefully Canadians do know of the contribution that university libraries make to library/archival service in Canada and thus as part of that eco-system our contributions have value. • Our concern as a member-driven consortium where Canadian universities are our members, is to ensure that researchers as end users understand the contribution that we make to their Universities and continue to support our need for funding. We know that librarians are aware, but we could enhance the understanding within the end user research and teaching community and University administration. 2. Describe the services provided directly to users within your context, or whether they are consortial in nature; please describe the mechanisms in place to define, refine and measure the impact of the services. CRKN: • Our services are consortial in nature and we do not supply content and services directly to end users. • Our member libraries have mechanisms in place to measure the impact of the content and services they acquire through CRKN. • We have a communications outreach survey sent to members every three years to measure the impact of our services on our members. • Additionally we provide regular electronic newsletters that can be re-purposed by our members to reach out to end users. • Services are defined and impacts measured through self-study programs like POMS (see above) as well as by benchmarking against our peers internationally. International comparisons occur through rigorous study, for example by conducting organizational reviews, as well as informally by interacting with peer organizations through conference participation and on listservs. 2 3. In the digital era, what support for patrons do/should libraries provide? CRKN: • In a world where information is increasing exponentially, the role of libraries (and librarians) in directing patrons to the information that they require cannot be overemphasized. o Providing information for patrons: Libraries purchase or license digital content in a wide range of formats and from a broad set of disciplines and provide access to this content to users. o Arrangement of information for patrons: Through detailed metadata and robust discovery mechanisms, libraries enable patrons to sort through digital content and retrieve the information they need for their research, study, or leisure. o Provision of open access services: libraries enable open access publishing through funds directed to APCs, by hosting published research in institutional repositories, or by funding and supporting open access publishers. This support provides information and research output to the broadest possible community of patrons. • Reference and Research support through reference desk and virtual reference services, increasing the awareness of the resources available to the researcher or student through their library • Information literacy instruction both in the classroom and in library/information resource centre 4. What in your opinion are the specific roles of libraries and/or archives and/or museums and other heritage institutions in community building and memory building? CRKN: • Preservation of cultural materials – both physical and born digital • Providing access to information to a broad community of users • Ensuring the gathering of culturally significant materials for research and preservation • Ensuring public space for community building where our cultural heritage can be examined and built upon and shared. • Literacy – promote all forms of literacy throughout the lifecycle of Canadians. NEW DIRECTIONS Digitization 1. What are the main challenges of born-digital material for your institution? CRKN: 3 • • • • • • • Preservation – digital formats vary for content and are not designed to be readily transferred to alternate platforms and to future technology innovation o Additionally there is a need for web-archiving, in particular for governmental information Discovery (i.e. how to ensure end users are accessing all of the information at their disposal) Sustainable Access: In an environment where resources are licensed in an electronic format (not purchased and placed on a shelf), perpetual or archival access to these materials upon the termination of a license (or the transfer of that resource from one publisher to another) is a chief concern (linked to this is preservation, of course). Additionally, given the lack of a physical item to catalogue and as a result of the fact that many library resources are now bundled in large packages of sometimes upwards of 1,000 serials or 10,000 eBooks at a time, the provision of accurate holdings information and metadata to libraries is an area where publishers have not fulfilled their responsibilities as suppliers of information resources. In many cases, libraries and consortia alike are left to take on this task which results in substantial staffing hours dedicated to verifying, modifying, and maintaining lists of electronic resources. Most notably, the lack of accuracy and clarity in the data delivered by publishers has implications for access at the user level. Equitable access to expensive digital resources for Canadian researchers regardless of geography or institutional affiliation Determining fair price in a world where pricing is still based on print revenue models or in the case of journals, on historic print spend. The development of digital content has added the opportunity of providing content in an open access format. Universities need to recognize the value of the open access principle and support it accordingly by implementing policies that incentivize the publication of open access articles by their researchers. Tenure track committees need to recognize that open access is a viable model for publication, and support its use in their deliberations. 2. What role should libraries and archives take in the digitization, the dissemination and the long-term preservation of Canadian heritage (print publications and archives)? CRKN: • Libraries and archives should take a leadership role in the digitization, hosting, and long-term preservation of Canadian heritage materials by: o Working as a community to create standards for digitization and metadata o Digitize content for current accessibility o Coordinating efforts to eliminate redundancy o Ensuring equitable access for all Canadians to our heritage material o Sharing infrastructure including providing back-up hosting/archiving of collections. 3. What will be the function and future of a brick-and-mortar library or archive in a paperless future? CRKN: • Still a huge role for brick and mortar libraries and archives: 4 o print preservation, community building, face to face exchange of knowledge and ideas, public access to electronic resources, information literacy instruction, reference services. Education 1. What changes, in your judgment, are necessary in the professional education and training of librarians/archivists in the 21st century? CRKN: • Include some business and leadership training – money and personnel management, negotiation skills, an understanding of the need in any situation to have a win on both sides • A greater emphasis on digital technologies and the electronic environment is a shift that is greatly needed among Library Studies and Archives programs. Students should be familiar with electronic licensing, link resolvers, ERM systems, and next-gen ILS systems. • Additionally, familiarity with concepts like open access, digital preservation, and the scholarly communication process should be a key component of library students’ curriculum. 2. What conversations do you think need to take place with library, archival, and information studies programs about professional competency requirements, and have they begun? CRKN: • A focus on interdisciplinarity and preparation for non-traditional library roles in the workplace are conversations that should be had with library school directors. Certainly an emphasis on the interdisciplinary nature of research has begun at some schools, but there may be a lack of knowledge among MLIS graduates of employment opportunities outside of traditional libraries (at consortia, for example). • This question stimulated a discussion in our team about whether there is a need for a professional library designation/certification program. Could/should there be an opportunity to specialize with credentials associated with that? Would this provide an opportunity for staying current with professional development? At the least, this should be a conversation even if it is not necessary. Resources 1. Public libraries are primarily funded by local municipalities, with little funding from any other level of government. Many towns and rural communities are too small to support needed technology. How do we encourage the creation of library systems (or consortia) that can meet the increasingly sophisticated technology-driven needs of libraries—whether urban or rural? CRKN: 5 • • From our discussion during our consultation, is there a way to leverage CRKN’s license negotiation skills and publisher relationships on behalf of the Canadian public library community? A funded separate stream of negotiation? Consortia are often created out of a need to manage/disperse a central pool of funding. Financial incentive would encourage libraries to begin to coordinate, and enjoy the extended benefits of working collaboratively (eg. reduced redundancy, greater sense of community, idea-sharing). 2. Assuming academic host institutions have financial resource constraints, and assuming academic libraries are equally constrained, how might these libraries attract funding adequate to meet the expectations of their users? CRKN: • We would concur with this assumption on both sides: libraries and their consortia need to align the outcomes of projects with the mandates of federal and nongovernmental funding bodies in order to attract funding. • Work collaboratively to identify redundancy and free resources. 3. What percentage increase to your current budget would permit you to realize the aspirations of your users? If you received an increased budget and consistent adequate resources, describe your library/archives in 2017. CRKN: • Some background: CRKN was born as an innovative initiative to leverage the buying power of all of Canada’s universities, to save on all member costs and reduce duplication of effort to increase access to digital research content across the country. Seed funding enabled us to take the first steps, but now CRKN’s operating budget is entirely funded by membership fees. • In keeping with the changes to the academic scholarly publishing environment, CRKN is in a position to once again be innovative; however we are in a deficit budget position and are in the midst of a multi-year deficit reduction plan. We recognize the need to seek special project funding from members in order to fund new initiatives. • We continue to explore how we can align our undertakings with the priorities and agendas of funding agencies to have specific projects be considered as viable candidates for funding. • With consistent increased budget, as a national organization, we have the capacity to be an effective driver of a unified digital ecosystem in Canada, to collaborate with other organizations and potentially coordinate multiple initiatives to create a much more robust ecosystem within Canada so that our research and teaching community can be competitive in the global research environment. • We could potentially expand our mandate and leverage our negotiating experience to the benefit of other library communities such as colleges, public libraries or agencies who fall outside of other consortia. This would require additional staff and licensing infrastructure to support. 6 CRKN STRATEGIC PLAN 2013 – 2016 Strategic Plan 2013 – 2016 MISSION The Canadian Research Knowledge Network (CRKN) is a partnership of Canadian universities dedicated to expanding access to digital content for the academic research enterprise in Canada. Through the coordinated leadership of librarians, researchers, administrators, and other stakeholders in the research community, CRKN undertakes large-scale content acquisition and licensing initiatives in order to build knowledge infrastructure and research and teaching capacity in Canada’s universities. VISION CRKN is a catalyst for innovation & development and achieves the greatest possible impact in support of Canadian research, scholarship, creative activity and knowledge transfer by working creatively and collaboratively with other stakeholders. Strategic Plan 2013 – 2016 VALUES Accountability: CRKN is driven by and accountable to its membership. Broad access: CRKN is committed to securing the broadest possible access to the world’s knowledge for the benefit of its member universities and the communities they serve. Collaboration: Commitment to collaboration and the building of partnerships among stakeholders underpin all CRKN operations and activities. Transparency: CRKN operates transparently, engaging in open dialogue, communicating its actions effectively, and responding to input in an ongoing fashion. Consensus building: CRKN recognizes the diversity within its member institutions and the research community, and builds consensus and understanding in achieving common goals. Excellence: CRKN is committed to excellence and the pursuit of continuous improvement in its services and operations. Strategic Plan 2013 – 2016 Strategic Objective 1.0 Collaborate to Advance Digital Scholarship Strategic Objective We will assess the value of current and potential partnerships to best focus our resources and effort nationally and internationally. Our goal is to reduce redundancy, share and distribute expertise, and identify opportunities for joint funding initiatives. Strategies: 1.1 Map the Canadian university library digital scholarship eco-system (including existing content and data infrastructure, open access solutions, perpetual access options/solutions, Canadian documentary heritage, data and text mining solutions). 1.2 Identify areas of demonstrable success within the university library eco-system in creating solutions and finding funding for digital scholarship. 1.3 Collaborate to build a consensus within our universe for confederation and coordination of resources/solutions at the national level. 1.4 Contribute to the emerging consensus on the broader national digital infrastructure eco-system. Canadian Research Knowledge Network Page 4 Strategic Plan 2013 – 2016 Strategic Objective 2.0 Expand Content and Service Offerings Strategic Objective CRKN will continue to support, strengthen and evaluate existing licenses, while developing and implementing a systematic process to identify and respond to the diverse and evolving content needs of CRKN members. We will assist our membership to meet the evolving needs of its stakeholders by offering new infrastructure and additional services, and by expanding its role in the scholarly communication life cycle by developing a systematic way to support open access publishing. Strategies: 2.1 Create and develop an open-access strategy. Tactics: • Adopt an advocacy role with the publisher community for open access • Collaborate with CARL and other organizations to advance open access objectives 2.2 Build on existing content program to develop new criteria for accessing content as a reflection of member needs. Tactics: • Establish a task group reporting to the NRT to develop criteria for assessing future content acquisition opportunities in consultation with membership • Offer a range of tools and services to meet member needs for assessing and demonstrating the value of licensed resources 2.3 Identify lesser value licensing activities that can be discontinued to free resources for new services. 2.4 Complete the work of the Perpetual Access Task Group by establishing a perpetual access option to provide sustainable stewardship of licensed scholarly resources. Canadian Research Knowledge Network Page 5 Strategic Plan 2013 – 2016 Strategic Objective 3.0 Engage Members and Stakeholders Strategic Objective We will continue to develop and deliver strong member services and commit to engaging our members in authentic dialogue. We commit to fostering a high degree of trust and accountability with both our members and our stakeholders. Strategies: 3.1 Develop and implement a multi-pronged engagement strategy for members (shareholders). Tactics: • Continue to develop and strengthen engagement channels to enable members to affect content decisions, negotiations, etc. such as investment in ongoing development of the members-only License Information Module (LIM) • Continue development of the key performance metrics and indicators, demonstrating organizational value, to support informed decision making and to strengthen member trust • Foster professional engagement among members to develop and share expertise • Recognize volunteer leadership though the annual awarding of the Ron MacDonald Distinguished Service Award 3.2 Engage the broader stakeholder community to leverage their expertise. Tactic: • Identify and develop means for engagement with stakeholders (researchers, potential funding partners, publishers and other consortia) Canadian Research Knowledge Network Page 6 Strategic Plan 2013 – 2016 Strategic Objective 4.0 Maintain and Strengthen Sustainability Strategic Objective Through visionary leadership and resourcing of strategic priorities, we will leverage the total capacity of CRKN’s membership and staff to provide a full range of seamless, consistently high quality services in both official languages. CRKN will build the trust of its members and ensure their full understanding of the CRKN value proposition. CRKN will mitigate financial risk by providing a high ROI for members and by identifying initiatives or partnerships that will attract new funding. Strategies: 4.1 Strengthen our organizational capacity. Tactic: • Encourage staff retention and build staff capacity through training and career development opportunities • Explore the possibility of using secondments and research leaves to reinforce our staff value proposition and augment organizational capacity 4.2 Strengthen governance and management processes. . Tactic: • Adopt a financial exigency plan and other risk mitigation strategies • Develop strategies to identify potential new member segments for targeted growth • Diversify revenue streams and develop initiatives that can be used to attract new funding • Conduct an academic style external review • Complete by-law and committee changes in connection with Governance Review • Continue to codify organizational knowledge and develop systems and processes to leverage organizational capacity 4.3 Develop more effective means of engaging members in both official languages. Canadian Research Knowledge Network Page 7 Strategic Plan 2013 – 2016 INTEGRATED DIGITAL SCHOLARSHIP ECO-SYSTEM (IDSE) CONCEPT Key objectives    Advance research capacity and innovation in Canada Create an integrated digital scholarship eco-system by coordinating and complementing a number of existing and emerging Canadian initiatives; Build on demonstrated success in leveraging investment to the Canadian academic community with an agile organizational structure and outside funding. Commercial e-articles & ebooks Digital preservation (HathiTrust North) OA peerreviewed earticles & ebooks Collaboration Coordination Confederation Prepublication research & data Canadian documentary heritage Spatial & numeric data Annual THE CANADIAN THE IMPACT OF Report 2012 – 2013 RESEARCH KNOWLEDGE NETWORK: HOW DO WE MEASURE UP? 2012 - 2013 COLLABORATION COORDINATION CONFEDERATION NARRATIVE AND DASHBOARD Prepared by the CRKN Value, Influence, Trends and Leadership (VITaL) Task Group September 2013 OVERVIEW CRKN has developed a framework of indicators in order to benchmark performance, share the value of achievements with others and to use the indicators for information-based decision-making. Since late 2009, CRKN’s Value, Influence, Trends and Leadership (VITaL) Task Group has been creating this dashboard at the request of the Board. To ensure the framework was strategic, it was aligned with the Strategic Plan, although it can be adapted to accommodate changes in strategic focus. Three themes or “umbrellas” were created to capture the essence of CRKN outcomes, using the narrative to shape the messaging directed to different target communities. Three conceptual umbrellas have been turned into dashboards which drive the impact discussion. They are: 1. Membership Leverage 2. Scholarly Content Offerings 3. Community Engagement 1. MEMBERSHIP LEVERAGE This dashboard summarizes the value of membership – it illustrates how the results of collaboration are transformed into community gains that individual members could not achieve alone. This is demonstrated in several ways: • CRKN revenues increased steadily until 2010 benefiting from new acquisitions and healthy interest rates. In 2013 the exchange rates were slightly less favorable than 2012 and there were no DCI Project renewals (low recurring access fees) adding to CRKN’s overall revenues ($92.1M) and expenses. • CRKN operations remained lean comprising 1.6% of total revenues. • Of the 16 license renewals in 2013, 12 were for three years which provides greater predictability and savings for members. These renewals were negotiated for cumulative $62.6 million (compared with the vendor price of $152.4 million outside the consortium. • Three licenses were renewed for only one year due to several factors including the proposal of unfavorable license clauses, revised licensing terms that did not align with CRKN principles, and the anticipation of a new pricing model not yet launched. The negotiated price for these three licenses was $18.9 million (compared with vendor pricing of $22.1 million). • Since its inception and including the activities of its forerunner the CNSLP, CRKN has purchased more than $869 million (CAD in 2013 dollars) of content on behalf of its member institutions. A conservative estimate suggests that if CRKN-licensed content were to be acquired on an institution-by-institution basis, costs to CRKN members would be nearly $2.48 billion over the same time frame. 2. SCHOLARLY CONTENT OFFERINGS This dashboard illustrates the diversity of the licensed and purchased content available to CRKN Members. In 2012-2013 CRKN membership remained at 75 institutions and the dollar amount associated with content increased to $92.1 million: • Participation in licenses has increased exponentially in the past ten years from 64 members to 75, and from 7 licenses to 54 across 2,629 agreements. Much of this increase took place since 2007 before the DCI Project when 33 licenses were in place across 1,487 agreements. The most significant growth has been with Social Science & Humanities content. This steady increase indicates that members perceive value from participation, that they trust CRKN with their library budgets and that the content is beneficial to students and researchers. • CRKN content has been calculated according to dollar amounts between Science, Technology and Medical (STM) (72%) and Social Sciences and Humanities (SSH) content (28%). While the content has remained largely constant for the past 3 years, the low DCI Project access fees have made SSH content much more affordable. • SSH content is increasingly diverse in types of materials accessible, including video, images, music, e-books, newspapers, e-journals and primary source material. • CRKN has retained the same SSH content over the past few years, and most Canadian and French language content fall into this category. The low DCI Project access fees have made the content more affordable and it now appears to have reduced from 6% to 2%. Since Canada generates approximately 6% of dollar value of the published scholarly material, this indicates that what CRKN purchases reflects Canadian content available in alignment with its objective to focus on Canadian content in both languages. Note: Canadian content is defined as digital content published in Canada, or published elsewhere but of special interest or significance to Canada. Not all French content is Canadian. 3. COMMUNITY ENGAGEMENT This dashboard highlights the many ways that CRKN reaches out to members to engage in discussion, planning and product selection. The different channels of engagement are outlined: • AGM participation has increased from 59 in 2004 to 74 in 2012. Member feedback indicates the event is highly valued for networking, educational and collaborative purposes with satisfaction ratings in excess of 75%. • In response to feedback from the Communications Outreach project, the monthly NewsBrief was launched in February 2010 and after more than 40 editions, still receives over 30% open rate (English & French) by stakeholders, well above the industry average of 15-20% (Source: Constant Contact – leading software for newsletters). • Over the past 13 years, CRKN has been invited as a trusted voice to speak at over 53 national and international events in order to share expertise and knowledge pertaining to licensing of digital content. This speaks to CRKN’s credibility and positive reputation in the community at large. • Since its inception, CRKN has enjoyed the sustained and rich contributions of time and expertise of its member contributors. Over the past year, 53 members filled 80 positions by serving on the Board, Standing Committees and Task Groups contributing a total of 2,673 hours. Contributors come from three groups (senior university administrators, researchers and library staff) who through their work, contribute to decision-making and program oversight. That CRKN continues to attract this caliber of member interest is an indication that the return on investment makes this contribution worthwhile. • CRKN maintains organizational and members-only License Information Module(LIM) websites in English and French. Visits and page views climbed for both English and French sites. Visit duration on the LIM is still above 6 minutes, well above the one-minute industry average. • To ensure that CRKN is aligned with Member priorities, it engages in planning outreach, license renewal information teleconferences, communications outreach, Listserv+ and OpenLine sessions. • CRKN issued the second annual customized report to members to capture different qualitative and quantitative benefits received by each member. Building on last year’s feedback, two new areas were added (Benefits of Centralized Negotiation & Administration; Usage Statistics) to the areas tracked last year (Use of Licensing Program, Membership fees, CFI Funding, Addressing National Issues, Member Engagement). This content is made available for information, to support member decision-making and possible reuse. IN SUMMARY This framework of indicators has been compiled to share with members and stakeholders as a demonstration of the type and level of activity undertaken at CRKN. It provides benchmarks for discussion, for refinement, and for celebration of achievement. The Board of Directors, Management and stakeholders can use this information to evaluate the impact CRKN is having as it works to achieve its objectives. ey Blackwell dor price otiated price DASHBOARD 1 - MEMBERSHIP LEVERAGE CFI SEED MONEY LEVERAGES SUSTAINED INVESTMENT $120M $100M $80M $60M $40M $20M 0 2001 2002 2003 Voluntary Member funding Provincial Matching funding 2004 2005 2006 2007 2008 2009 2010 University Required funding CFI funding 2011 2012 2013 (Figures drawn from audited financial statements.) The first year investment of $6.2 million made by CFI in 2001 for the Canadian National Site Licensing Project ($20 million over 3 years) has leveraged more than $92 million in 2013 through investment from provincial funding partners and the member institutions. While license participation has steadily increased, favorable exchange rates and the low recurring costs of the DCI Project result in lower costs for the same content. The DCI Project sustainability period is shown in the last two years for which $5.2 million (2013) is University Required funding. This demonstrates how value is created and shared through CRKN’s collaborative infrastructure. ONE YEAR RENEWAL - 3 LICENSES $20M THREE YEAR RENEWAL - 12 LICENSES $60M $60M $50M $50M $40M $40M $30M $30M $20M $20M $10M $10M $15M $10M $50M 0 Informa Wiley Blackwell ACS Vendor price Negotiated price Informa ACS (Figures drawn from audited financial statements.) The negotiated price for three one-year license renewals was $18.9M compared with the vendor price of $22.1M. 0 2013 Vendor price Negotiated price 2014 0 2015 2013 Vendor price (Figures drawn from audited Negotiated price financial statements.) CRKN delivered value for membership with 12 regular license renewals negotiated for a cumulative $62.6M, in contrast with the vendor price of $152.4M outside the consortium. 2014 2 DASHBOARD 2 – SCHOLARLY CONTENT OFFERINGS INCREASED PARTICIPATION BY CONTENT TYPE # Agreements 3,000 2,500 2157 2271 2371 2519 2616 2629 2012 2013 2,000 1487 1,500 1067 1,000 500 0 448 448 2001 2002 2003 618 509 2004 Social Sciences & Humanities 729 2005 2006 2007 2008 2009 2010 2011 Science, Technology & Medical In 2013 CRKN had 54 licenses with participation through 2,629 agreements with members. This participation has increased significantly from 33 licenses across 1,487 agreements in 2007 before the DCI Project. The most significant growth has been with Social Science & Humanities (SSH) ENGLISH & FRENCH CONTENT BY $ VALUE 2013 CANADIAN AND INTERNATIONAL CONTENT BY $ VALUE 2013 2.21% 2.21% 1.96% 1.96% 97.79% 97.79% 98.4% 98.4% EnglishEnglish (97.79%) (97.79%) French French (2.21%) (2.21%) CRKN licenses and one-time purchased content has remained fairly constant over the past couple of years, but due to the low access fees of the DCI Project licenses, the percentage of French-language (and bilingual) content has declined from 7% to 2% in 2013. International (98.4%) International (98.4%) Canadian (1.96%) Canadian (1.96%) CRKN licenses and one-time purchased content has remained fairly constant over the past couple of years, but due to the low access fees of the DCI Project licenses, the percentage of French-language (and bilingual) content has declined from 6% to 2% in 2013. CRKN purchases are aligned with Canadian content available. DASHBOARD 2 – SCHOLARLY CONTENT OFFERINGS (CONTINUED) Social Science & Humanities Science, Technology & Medical 27.20% 72.80% Total 2013 SSH License Expenditure 2013 PORTFOLIO OF CONTENT Total 2012 2013 SSH License STM License Expenditure by $ Value E-journals Current (61.77%) E-journals SSH Current (9.97%) Primary Source/Historical (9.03%) E-journals backfiles (7.08%) E-journals French (4.15%) Historical Newspapers (3.77%) Primary Source/Current (1.76%) Image (1.31%) E-Books Historical English (0.92%) Music (0.18%) Video streaming (0.07%) E-journals STM Current (90.67%) Indexing, Abstracting & Citation Databases (9.19%) E-journals STM Backfiles (0.14%) CRKN licensed products can be divided into two main categories – Social Sciences & Humanities (SSH) and Science, Technology & Medical (STM). As collections are enhanced increasingly diverse content will be made available in the form of databases, journals, e-books, videos, music, images, primary sources, newspapers, etc. Over the past two years the size of the categories, measured by $ value, appear imbalanced owing to the nature of the low recurring costs of SSH content making up the DCI Project. 72 DASHBOARD 3 – COMMUNITY ENGAGEMENT NEWSBRIEF OPEN RATES (APRIL 2012-MARCH 2013) AGM PARTICIPATION 200 190 60 60 50 50 40 40 30 30 20 20 10 190 10 150 119 119 116 102 102 100 79 116 84 72 79 84 74 74 59 50 0 2004 2005 2006 2007 2008 2009 2010 2011 2012 005 2006 2007 2008 2009 2010 2011 2012 0 In person Online son e 0 April May June July Aug Sept April Dec June Mar Aug Sept Nov Dec Nov May Feb July Industry average English French Industry average English French AGM participation has increased from 59 in 2004 to 74 in 2012. Member feedback indicates the event is highly valued for networking, educational and collaborative purposes with satisfaction rated at 75% and above. The NewsBrief was published 10 times and distributed to library, administration and stakeholder communities. It reached open rates in excess of 30%, well above industry averages of 17%. In January 2013 CRKN adopted an “opt-in” policy for NewsBrief distribution the result was a decrease in circulation combined with an increase in % opened. INVITED PRESENTATION DELIVERY NATIONALLY & INTERNATIONALLY 12 11 10 8 7 6 6 5 0 4 4 4 2 6 3 2 2 2 2007 2008 1 0 2000 2001 2002 2003 2004 2005 2006 2009 2010 2011 2012 Since 2000 CRKN representatives have been sought out as a trusted voice to deliver 53 keynotes and general session presentations to national and international audiences. 6%) me (25%) ns (20%) 12%) n (11%) DASHBOARD 3 – COMMUNITY ENGAGEMENT (CONTINUED) HOURS OF MEMBER TIME DEVOTED TO CRKN WORK NRT time on renewals (6%) NRT meeting & prep time (26%) CRKN meeting and prep time (25%) CRKN time on negotatiations (20%) Membership Services time (12%) CRKN post-purchase admin (11%) Cost Sharing Advisory Group (864) Negotiations Resource Team (862) Board of Directors (370) Value, Influence, Trends and Leadership Task Group (150) Search Committee for Executive Director (129) Serials Management Task Group (128) Finance & Audit Committee (70) Open Access Working Group (54) Executive Committee (43) Perpetual Access Task Group (3) Cost Sharing Advisory Group (864) Time contributions from 53 members carrying out 80 roles including the Board of Directors, Standing Committees and purpose-driven task groupsNegotiations Resource Team (862) and around 58 meetings. comprised a total of 2,673 hours in Board of Directors (370) Value, Influence, Trends and Leadership Task Group (150) Search Committee for Executive Director (129) Serials Management Task Group (128) Finance & Audit Committee (70) Open Access Working Group (54) Executive Committee (43) USAGE OF Group (3) Perpetual Access TaskCRKN WEBSITES (APRIL 1, 2012 – MARCH 31, 2013) AVERAGE TIME VISITS PAGE VIEWS License Information Module (Members English) 13,805 90,870 6:37 License Information Module (Members French) 2,069 11,255 6.08 CRKN Organization (English) 21,605 77,668 3:27 CRKN Organization (French) 2,496 9,276 4:05 (MINUTES) CRKN tracks website usage statistics for both its public organization site and the member-only License Information Module (LIM). The LIM supports 599 accounts and provides a technical information source that is consulted for an average of over 6 minutes per visit compared with one-minute industry visit averages. Its use has increased significantly (21%) in the past year, driven in part by the Listserv+ with 1,800 visits and by the introduction of the French site in March 2012. The English organization site enjoyed a 30% growth in visits, and a small decrease in visit time. The French organization site visits increased slightly, while the time on site decreased marginally by 7%. Canadian Research Knowledge Network Platform Outcome Measurement Study (POMS) Report Submitted to the Canada Foundation for Innovation Expert Panel October 29, 2012 1 Acknowledgements CRKN would like to acknowledge the people who made contributions to this report right from the early stage of refining the framework, planning of content, and writing the report. Value, Influence, Trends and Leadership (VITaL) Task Group (since 2010)– comprised of:       Joyce Garnett, Chair, University Librarian, Western University Sylvie Belzile, Director General, Library and Archives Services, Université de Sherbrooke Ronald Bond, Provost Emeritus, University of Calgary Helen Clarke, Associate Vice Provost, Collections, University of Calgary Katherine Schultz, VP Research, University of Prince Edward Island John Teskey, Director of Libraries, University of New Brunswick Aaron Lupton, Electronic Resources Librarian, York University Kathleen Shearer, Consultant © Canadian Research Knowledge Network 301-11 Holland Avenue, Tower A Ottawa, ON K1Y 4S1 Tel: 613.907.7040 Fax: 866.903.9094 Email: info@crkn-rcdr.ca Web: www.crkn-rcdr.ca 2 Table of Contents 1.0 An Overview of the Research Platform .............................................................................................. 6 1.1 High-level description of the Research Platform ................................................................................ 6 1.2 Governance, management and advisory structure ............................................................................ 9 Governance structure ......................................................................................................................... 10 Management structure ....................................................................................................................... 13 Advisory structure ............................................................................................................................... 14 Host institution linkages ..................................................................................................................... 15 Roles of stakeholders .......................................................................................................................... 15 1.3 Platform planning process ................................................................................................................ 18 Formally approved plans..................................................................................................................... 18 External influences ................................................................................. Error! Bookmark not defined. Internal performance monitoring ....................................................................................................... 19 1.4 Cultural, organizational and structural change enabled at the various stakeholder organizations . 23 Impact on stakeholders....................................................................................................................... 23 Cultural impact .................................................................................................................................... 27 Branding .............................................................................................................................................. 28 2.0 Research Capacity ............................................................................................................................. 30 2.1 Capital investment value .................................................................................................................. 30 Value ................................................................................................................................................... 30 Enhancements..................................................................................................................................... 31 2.2 Operating & maintenance investments ............................................................................................ 32 2.3 Platform Capabilities and Sustainability ........................................................................................... 33 Current infrastructure ......................................................................................................................... 34 Service functions ................................................................................................................................. 34 The platform personnel ...................................................................................................................... 35 Sustainability ....................................................................................................................................... 36 2.4 Foregone Costs.................................................................................................................................. 36 2.5 Highly Qualified Personnel ................................................................................................................ 38 Quality and richness of the training activities and environment........................................................ 38 3 A snapshot........................................................................................................................................... 39 2.6 Influence of CFI ................................................................................................................................. 40 3.0 Leadership Development & Research Enabled ................................................................................. 42 3.1 Users / access .................................................................................................................................... 42 Policies and mechanisms .................................................................................................................... 42 User community .................................................................................................................................. 43 Utilization ............................................................................................................................................ 43 3.2 Leadership and competitiveness ...................................................................................................... 46 Platform personnel and program competitiveness ............................................................................ 46 Leadership ........................................................................................................................................... 49 Mechanisms to enhance competitiveness.......................................................................................... 50 3.3 Linkages............................................................................................................................................. 50 Formal MOUs signed with other countries and international organizations ..................................... 51 Realized research linkages and characteristics ................................................................................... 51 3.4 Research outputs .............................................................................................................................. 52 Overall research influence and reach ................................................................................................. 52 Research opportunities ....................................................................................................................... 55 3.5 Influence of CFI ................................................................................................................................. 58 4.0 Extrinsic Benefits -Impact on Local, Regional and National Innovation ........................................... 59 4.1 Mechanisms and strategies .............................................................................................................. 59 Impacts ................................................................................................................................................ 59 4.2 Intellectual property development and protection .......................................................................... 59 IP outputs ............................................................................................................................................ 59 4.3 Benefits from knowledge translation and transfer........................................................................... 60 4.4 Influence of the CFI ........................................................................................................................... 61 Sustainability issues ............................................................................................................................ 62 Evolving publisher business models ................................................................................................... 63 Impact measures ................................................................................................................................. 63 5.0 Challenges 62 4 References ............................................................................................................................................. 65 Appendix 1: Flowchart of Licensing Renewal Process ........................................................... 67 Appendix 2: List of CRKN Content Providers ............................................................................. 68 Appendix 3: CRKN Members............................................................................................................ 69 Appendix 4: Flowchart of Governance Review ......................................................................... 70 5 1.0 An Overview of the Research Platform 1.1 High-level description of the Research Platform Mission The Canadian Research Knowledge Network (CRKN) is a partnership of Canadian universities dedicated to expanding digital content for the academic research enterprise in Canada. Through the coordinated leadership of librarians, researchers, and administrators, CRKN undertakes large-scale content acquisition and licensing initiatives in order to build knowledge infrastructure and research capacity in Canada’s universities. University libraries are the drivers of CRKN’s initiatives, and play a primary role in leveraging expertise and resources for the benefit of Canada’s scholarly research community. Membership CRKN’s membership has increased from 64 institutions in 2000 to 75 institutions in 2012; no members have withdrawn despite the difficult economic circumstances facing all universities. CRKN members include the majority of the universities that belong to the Association of University and Colleges of Canada (AUCC). While AUCC has 92 members, many of these are affiliated colleges or campuses of larger universities, and as such are included in CRKN by virtue of their affiliation with CRKN member institutions. All AUCC members are eligible for CRKN membership, but a small number (primarily faith-based, first nations, and highly-specialized universities) do not participate in CRKN at this time. Services CRKN’s core business is to provide universities with expanded access to the materials and outputs of scholarly research in digital formats. Currently, CRKN-licensed content represents over 50% of total spending on digital content by the libraries of member universities. Through these libraries, digital content secured by CRKN is made available to approximately 99% of university researchers and students in Canada. 6 History CRKN began as a pilot project called the Canadian National Site Licensing Project (CNSLP) in January 2000, after securing a major award from the Canada Foundation for Innovation (CFI) in its first competition. The award recognized that high-quality digital content constituted an essential element of research infrastructure in the knowledge economy, and worthy of pan-Canadian investment. CNSLP’s goal was to bolster the research and innovation capacity of the country's universities by licensing electronic versions of scholarly publications on a national scale. The CFI award of $20 million leveraged an additional $30 million in matching funds ($20 million from provincial government funders and the Atlantic Canada Opportunities Agency (ACOA) plus $10 million from 64 participating universities) for a total project worth $50 million over three years. The University of Ottawa served as the project’s host and administrative centre, and a Steering Committee oversaw all aspects of project development. In January 2001, following a formal Request for Proposal and evaluation process, CNSLP implemented multi-year license agreements with seven major scientific publishers, providing access to over 1,000 ejournals and key citation databases for researchers nation-wide. This critical mass of content was composed primarily of full-text electronic journals and citation databases in science, engineering, health and environmental disciplines, as these were areas of priority for CFI and where the needs and costs for universities were most acute. CNSLP was also successful in establishing a “made-in-Canada” model license agreement, which put in place superior terms of access and usage for the academic community. In the following years with the end of CFI and matching funds, CNSLP continued to add high-impact collections of journals and backfiles to the content portfolio, funded solely by its member libraries. By 2004, CNSLP had more than doubled its initial budget as a result of this member-funded content growth. On April 1, 2004, CNSLP was incorporated as a not-for-profit organization and renamed the Canadian Research Knowledge Network (CRKN). This organizational change formalized the governance structure with a Board of Directors, and put in place the administrative underpinnings to move the initiative from project to ongoing program. Incorporation also opened the doors to new participants, and later that year 10 more universities joined CRKN, bringing total membership to 74 institutions. In 2005, CRKN took the next step in organizational development, assuming responsibility for its own financial management through the implementation of policies, procedures, internal controls, banking arrangements, and the establishment of a Finance & Audit Committee. Also in 2005, with the growing need for digital content in social sciences and humanities disciplines, CRKN began planning a threephase content expansion project that would secure a portfolio of content in these disciplines and would also dovetail with a further proposal to CFI for program expansion. The first two phases of the content expansion proceeded on an incremental and member-funded basis, in tandem with development of the proposal to CFI. In February 2007, CFI announced a $19.1 million award to this initiative under its National Platforms Fund. With matching funds totaling $28.6 million from 67 universities and provincial governments, the Digital Content Infrastructure for the Human and 7 Social Sciences (DCI) project was launched, accelerating CRKN’s program expansion and extending its impact. By June 2008, fourteen major research collections in social sciences and humanities disciplines had been secured under the DCI project, making available thousands of e-books, e-journals, primary source materials, videos, classical music scores, etc. In 2009, CRKN committed the remaining DCI content funds to digitization of unique international and Canadian historical materials. As during its pilot phase, CRKN continues to license additional content on a member-funded basis. Over its history, CRKN has licensed or purchased over $767 million (CAD in 2012 dollars) of digital content on behalf of its member institutions, and now administers an annual budget of just under $90 million. CRKN administrative offices are located in leased space in Ottawa, Ontario, having outgrown the space generously provided by the University of Ottawa up until December 2007. CRKN currently employs seven full-time employees plus one on contract, with other specialized functions handled through shortterm contracts or outsourced to project-based consultants. CRKN’s core function and capability is in the area of content licensing. All of CRKN’s program and administrative functions, including member services, finance, communications and administration, are designed to support CRKN’s core business. CRKN today is characterized by a robust governance and administrative structure, best practices for procurement embedded in its content program, engaged participation of members, and growing collaboration and partnerships with stakeholder organizations. Evidence reflects that CRKN’s core business – extensive digital content made available to researchers and students, and new endeavours achieved through large-scale collaboration – continues to be highly valued by researchers and students across the country. Major functionalities and capabilities As a national platform, CRKN is an enabler, providing digital content that fosters innovative research across multiple institutions and disciplines. Unlike many of CFI’s platform projects, CRKN does not generate research or manage specialized equipment, labs, or facilities. Instead, CRKN negotiates agreements with publishers or content vendors to provide the best financial, access and usage terms for digital content made available directly from publishers’ or vendors’ sites. CRKN concentrates its efforts on licensing content that is of broad interest and high need for researchers at member universities. To date, CRKN has negotiated and continues to administer 52 licenses with total content expenditures of $89 million in 2012. This represents approximately 56% of all academic library expenditures on electronic content in Canada.1 Negotiating license agreements for digital research content is not a trivial task. Each new license agreement or renewal undertaken by CRKN is by nature complex, and requires formal methodologies 1 Figures taken from CARL statistics of electronic serials expenditures for 2010-2011, which is estimated at $171 million. CRKN license expenditures in 2011 totaled $96.5 million. 8 and detailed processes that may take up to a year or longer. The flowchart in Appendix 1 illustrates the key steps that are involved in these licensing processes, including consultation with members, identifying appropriate cost models, and often lengthy negotiations with publishers in order to secure the most favourable licensing conditions and pricing for all CRKN members. The content licensed by CRKN is also very diverse. While the majority of the content continues to be made up of electronic journals and citation databases, CRKN-licensed materials increasingly encompass e-books, newspapers, videos, images, music, and primary source material. In dollar amounts, 65% of the content licensed by CRKN is in Science, Technology and Medical (STM) fields and 35% is in the Social Sciences and Humanities (SSH) fields. Over 98% of spending is on licenses with international publishers, with 98% English content and 2% French content. CRKN manages licenses with Elsevier, Sage, Springer, Taylor and Francis, Wiley-Blackwell, and numerous other content providers. A full list of CRKN publishing partners is available in Appendix 2. 1.2 Governance, management and advisory structure Figure 1: CRKN’s organizational chart Sound governance and management are hallmarks of CRKN. During the CNSLP pilot phase, organizational oversight was provided by a Steering Committee and committees (as set out in the CFI grant agreement and inter-university agreement) under the legal and administrative auspices of the University of Ottawa, with an Executive Director and project staff in charge of operations. Upon incorporation, CRKN formalized its governance structure with a Board of Directors and appropriate 9 committees to support the Board’s work. Corporate by-laws set out CRKN’s objectives and overall means for decision making: membership eligibility, composition and responsibilities of the Board of Directors, Officers, and committees, meeting requirements, etc. In turn, CRKN has established numerous policies and procedures to ensure that its governance and management conform to current best practices. Governance structure In 2010-2011, CRKN undertook a governance review that focused on the size and composition of the Board of Directors. The review took into account best practices in governance as well as new federal legislation covering not-for-profit organizations. The review was highly consultative, both with members and key stakeholder organizations, and culminated in recommendations to restructure the Board, increase the representation of member libraries, and reduce Board size by almost one-third (from 16 to 10 members). These recommendations were endorsed by CRKN members at the 2011 Annual General Meeting, and the resulting by-law changes were implemented in February 2012. Board of Directors: The Board of Directors is responsible for overseeing the affairs of the corporation. Throughout its history, the CRKN Board of Directors has consistently attracted highly-respected and well-regarded members of the Canadian academic community. The Board of Directors is composed of the eight individuals elected by the membership:     Four University Librarians, one from each geographical area (Atlantic, Ontario, Quebec, Western); One University Librarian to serve as Chair of the Negotiations Resource Team; One University Librarian from member institutions with over $100 million in external research funding (as contained in the latest Research InfoSource publication); Two university researchers and/or senior administrators. Two additional individuals are subsequently appointed by the Board:   One University Librarian appointed to add any balance or required experience or skill set that did not result from the election process One university administrator to serve as Chair of the Board of Directors. Current members of the CRKN Board of Directors are:        Ronald Bond, Provost Emeritus, University of Calgary (Chair) Donna Bourne-Tyson, University Librarian, Dalhousie University (Vice-Chair) Christopher Callbeck, Assistant Vice President, Financial & Administrative Services, University of New Brunswick, Saint John Campus (Treasurer) Robert Clarke, University Librarian, Trent University Richard Dumont, General Manager, Libraries Branch, Université de Montréal Charles Eckman, University Librarian and Dean of Library Services, Simon Fraser University Benoit Séguin, Library Director, Université du Québec à Trois-Rivières 10 CRKN Board of Directors - October 2012    Carol Shepstone, University Librarian, Mount Royal University Raymond Siemens, Canada Research Chair in Humanities Computing, University of Victoria Leslie Weir, Chief Librarian, University of Ottawa The changes that proceeded from the Governance Review have prompted the need for a comprehensive review of CRKN’s committee structure and composition. This review will be undertaken over the coming year under the leadership of a new Executive Director. At present, the following Committees and Task Groups are active: Executive Committee: The CRKN Executive Committee is established by the Board of Directors as a Committee of the Board pursuant to section 12 of the Corporation’s By-laws, and is responsible for:    Ensuring effective implementation of the Corporation’s policies and programs as established by the Board; Overseeing the operational and administrative elements of the Corporation; and Maintaining regular communication with the Officers with a view to promoting operational effectiveness and efficiency of the Corporation. The Executive Committee is composed of not more than five Directors, being:  Vice-Chair of the Board;  Treasurer of the Board;  Representative of the administrative unit of the Contact Institution;  Up to two additional Directors. Other Standing Committees and Task Groups: The Board relies on the following Standing Committees and Task Groups to support its work. All Committees and Task Groups have formal Terms of Reference that set out objectives, accountability, composition, meeting protocols, and communications. 11 Standing Committees: Finance & Audit Committee: The Finance & Audit Committee assists the Board in fulfilling its oversight responsibilities by reviewing and making recommendations to the Board regarding: financial information; strategic financial plans and the annual operating budget; systems of internal controls; and investment management activities. Governance Committee: The Governance Committee is responsible for: developing and recommending corporate governance principles and practices applicable to CRKN; identifying and nominating Finance & Audit Committee - October 2012 candidates for election to the Board of Directors; identifying and nominating candidates for Board appointment to fill vacancies on committees and task groups; undertaking longer-term succession planning for the Board, committees and task groups. NB: at present, these responsibilities are being assumed by the Executive Committee, pending the overall review of committees. Negotiations Resource Team: The Negotiations Resource Team (NRT) is responsible for: coordinating member / regional consultation and building consensus regarding CRKN electronic resources priorities, requirements and specifications; developing and recommending procurement strategies, processes and vehicles; recommending vendor licenses and contract terms to the Board of Directors for Negotiations Resource Team approval; contributing additional strategic and operations expertise to CRKN planning and communications; recommending to the Board of Directors the establishment of specialized task groups to address policy and procedural 12 issues proceeding from negotiations or license implementation, as required. This includes recommending lead license negotiator(s), their terms of reference and specific negotiations parameters to the Board of Directors for approval, and assisting with strategy development and providing functional direction to the lead negotiator(s). Task Groups: CRKN supports a number of task groups that are project-focused and not intended to operate indefinitely. Typically, a task group is created as a result of issues that are deemed to have significance to CRKN as a whole or to the majority of the membership. In 2012, CRKN is coordinating the work of five task groups:  Cost-Sharing Advisory Group: Launched in 2011, the objective of the Cost-Sharing Advisory Group is to build sensitivity to the challenges of this issue within CRKN’s membership, and make recommendations for action on license cost-sharing that enhance CRKN’s national mission, while maximizing member value and ensuring price predictability.  Joint CARL-CRKN Open Access Working Group: Launched in 2011, the objective of the Open Access Working Group (OAWG) is to collaboratively advance the exploration and promotion of sustainable open access models as an element of a cost-effective scholarly content ecosystem in Canada.  Perpetual Access Task Group: Launched in 2010, the objective of the Perpetual Access Task Group (PATG) is to examine the issue of perpetual and post termination rights to ensure longterm access to CRKN licensed content for member institutions. This group has delivered its final report to the Board, and work is now turning to exploring feasibility of implementation.  Serials Management Task Group: Launched in 2010, the objective of the Serials Management Task Group is to plan and complete work to customize vendor-supplied title lists to accurately reflect CRKN entitlements.  Value, Influence, Trends and Leadership (VITaL) Task Group: Launched in 2009, the VITaL Task Group’s objective is to develop indicators of CRKN performance and impact, to aid the Board, Committees and staff in decision-making and to strengthen reporting and communications with members, funders and other stakeholders. Management structure CRKN management consists of the Executive Director, who reports to the Board of Directors through the Vice-Chair (Chair of the Executive Committee), and staff who report to the Executive Director. A senior management team (Executive Director, Director of Communications, Director of Operations, and Manager of Content Programs) leads and coordinates work among key functional areas and supervises additional staff and/or contractors. Key functional areas include:  Member Services (three staff members) that handles the content program, vendor negotiations and renewals, member queries and problem resolution, and support for NRT and other task groups. 13   Operations (one staff member and one contractor) with responsibility for finance, facilities, IT and human resources, and support for the Finance & Audit Committee and other committees as needed. Communications (one staff member) with responsibility for strategic and tactical communications, member engagement, performance measurement, the public and member websites and other communications channels, and support for the VITaL Task Group. Advisory structure A CNSLP Advisory Committee was established in 2000 as both a best practice and a condition of the CFI funding award, to ensure that the pilot project benefitted from international experience and developments. CNSLP committees such as NRT also included representatives from other stakeholder organizations in Canada, including Library and Archives Canada (LAC) and the Canada Institute for Scientific and Technical Information (CISTI), so as to foster cross-fertilization of ideas and alignment with other national initiatives. Following incorporation, the Board of Directors formalized the advisory role by establishing a blueribbon Advisory Board in 2005 to enlarge the Board of Directors’ strategic vision and respond to matters referred to them by the Board of Directors. The Advisory Board was chaired by a member of the Board of Directors, and included 8 members drawn from the library, research, publishing, and public/private sectors in Canada, as well as two members from the international community (primarily university libraries in the US and UK). The Advisory Board played a valuable role in ongoing environmental scanning with regard to trends and developments, and was instrumental in identifying opportunities for new collaborations, partnerships, and program development. As the Advisory Board had no responsibilities for governance, its members were free to think broadly and creatively about challenges and opportunities facing CRKN, and in so doing brought diverse skillsets, experiences, and perspectives to bear on the governance deliberations of the Board of Directors. Examples of contributions by the Advisory Board include:     input on proposal development to CFI and successful implementation of the DCI Project; provision of counsel on key trends such as open access; exploration of linkages to strengthen relationships between university libraries and university IT departments; support for and input into the strategic planning processes for 2007-2009 and 2010-2012. At the end of March 2011, the Board of Directors took the difficult decision to disband the Advisory Board, prompted by the serious need to reduce CRKN's operating expenses in the face of ongoing financial constraints of members and the organization as a whole. On the operations side, beginning with CNSLP and continuing today, staff participate actively in the work of the International Coalition of Library Consortia (ICOLC). This engagement has helped CRKN build a strong network of colleagues internationally, and allowed the organization to both contribute to as well as learn from other organizations that are involved in large-scale collaboration and content licensing. 14 Host institution linkages University of Ottawa served as the host or “lead” institution during the CNSLP pilot phase of the organization, and played a formative role in the development and ongoing operations of CRKN. As CRKN matured as an organization, taking on responsibilities for its own financial management, office leasing, and human resources management, the University of Ottawa’s role has been likewise reduced. Currently, the University of Ottawa continues to fulfill a key role as contact institution for the DCI Project, with CRKN responsible for all other project management and administration. These roles and responsibilities are set out in the following agreements: Administrative Agreement: An Administration Agreement between the University of Ottawa and CRKN sets out roles and responsibilities and safeguards the interests of both parties. The University serves as the recipient institution with respect to CFI Funds, with responsibilities to distribute these funds to CRKN and report back to CFI in accordance with the CFI Award contract. CRKN’s responsibilities are to secure matching funds (see Inter-Institutional Agreement, below), administer all funds in compliance with CFI policies and requirements, and report to the University to fulfil its obligations to CFI. Inter-Institutional Agreement: This agreement between the University of Ottawa, CRKN, and the participating institutions sets out responsibilities of the respective parties in meeting their financial commitments and administering the DCI project in compliance with CFI policies and requirements, during both the grant-funded portion of the project and the three-year sustainability period following termination of CFI and matching funds. Roles of stakeholders CRKN has developed formal and informal relationships with various stakeholders and communities. CRKN’s major stakeholders are its member libraries, who participate actively in CRKN initiatives and operations on a daily basis. Other important stakeholders are the content users (researchers and students), Canada’s regional library consortia, the Canadian Association of Research Libraries (CARL), and other organizations in the academic community (such as AUCC, the Canadian Federation for the Humanities and Social Sciences, Canadian Association for Graduate Studies, Canadian University Council of Chief Information Officers, etc.) Prior to the Governance Review, CRKN’s formal relationship with many of these stakeholder organizations was through participation of their delegate(s) on the CRKN Board of Directors, with other informal relationships built between the Executive Directors and respective staff of the various organizations. With its restructured Board of Directors, however, there is a need for CRKN to develop more formalized relationships with these organizations, to ensure more strategic alignment, systematic communications, and leveraging of other investments being made in research and infrastructure in Canada. CRKN’s major stakeholders are described below, along with an explanation of their relationship with the organization: Members: CRKN membership is open to any Canadian, not-for-profit, degree-granting post-secondary institution. All research-intensive and undergraduate universities in Canada are members of CRKN, and 15 its 75 members represent the majority of AUCC membership, with only a small number of faith-based, First Nations, or specialized institutions not participating at this time. CRKN membership encompasses small, medium and large universities, anglophone and francophone institutions, with membership distributed fairly evenly across the four regions of the country (Figure 2). A complete list of CRKN members may be found in Appendix 3. Figure 2: CRKN Membership by Size and Region CRKN Membership by Size and Region Source: "Top 50 Research Universities List 2011", Re$earch Infosource Inc. Numberr of Members 12 10 SMALL (<$10M External Research) 8 6 MEDIUM ($10M-$100M External Research) 4 LARGE (>$100M External Research) 2 0 WEST (22) ONTARIO (21) QUEBEC (18) ATLANTIC (15) Library Directors at member institutions are the voting members of CRKN. Library staff have many points of contact with CRKN in order to participate in licenses, monitor access to content, pay invoices, and plan for the future. Numerous Library Directors and other library staff participate in CRKN governance and operations by serving on the Board, Committees and Task Groups. Senior administrators, including the President, Vice-Presidents (Research and/or Academic), Directors of Finance and Administrations, etc. are also important stakeholders within the membership, and are kept informed of CRKN activities and developments by the Library Director, or through CRKN’s relationships with other national organizations such as AUCC. User groups: CRKN’s licensed content is available to researchers, students, staff, and affiliates at the member universities. The user community comprises over 1.2 million people consisting of approximately 42,000 full-time professors, 898,400 fulltime students as well as part-time faculty and students, and university staff and affiliates. These users at CRKN member institutions make up approximately 99% of the total number of students and faculty represented by AUCC. While member libraries are the main point of contact at member institutions, CRKN maintains connections with the research community through focus groups and surveys that assess the benefits and impact of CRKN resources on research and teaching. Researchers are able to serve on the Board of Directors at CRKN and also participate in Committees and Task Groups where appropriate. Although CRKN has little direct contact with students, member libraries assess and bring forward the information needs of their students, and these help to shape CRKN’s priorities and licensing activities. 16 Canadian Association of Research Libraries (CARL): CARL is a membership organization comprised of the 29 largest academic research libraries across Canada, plus three national memory organizations (LAC, CISTI, and the Library of Parliament). All of CARL’s academic library members are also members of CRKN. CARL was instrumental in the development of the Canadian National Site Licensing Project proposal to CFI, and since then has been an important stakeholder for CRKN. Collectively, CARL members are the major beneficiaries of CRKN licensed content. Over 70% of CRKN’s users are at institutions served by CARL libraries, and these libraries contribute approximately 80% of CRKN revenues for content and operations. Although CARL no longer delegates individuals to CRKN’s Board of Directors, many of the Library Directors who serve on the Board (as well as Committees and Task Groups) are from CARL libraries and bring the perspective of large, research-intensive universities. Regional library consortia: Canada’s four regional library consortia (Conférence des recteurs et des principaux des universités du Québec, Council of Atlantic University Libraries, Council of Prairie and Pacific University Libraries, and Ontario Council of University Libraries) are key stakeholders for CRKN. All CRKN members are also members of one of the regional library consortia. These consortia provide a number of services, including licensing content of regional interest. CRKN has a close and mutuallysupportive working relationship with the regional consortia, with staff leadership meeting regularly to share knowledge and expertise, for the benefit of all involved. The CRKN model license, for example, was adopted by the regional consortia for use in their own contexts, and regional participation on Committees such as NRT also provide points of coordination and opportunities to learn from each other. CRKN consults with the regional consortia to ensure that there is no duplication of effort or overlaps in content licensed. Recently, CRKN has been working with the regional consortia to develop a framework for improving licensing workflows, identifying the types of licenses most suited for work at the national, regional, and institutional levels, and rationalizing the maintenance of agreements at the most supportable level. Other stakeholders: CRKN engages on an informal but ongoing basis with other stakeholder organizations listed below. These organizations represent important players within the research/infrastructure community, and provide important input to CRKN in terms of trends in higher education and evolving information needs of the research community. They are also natural partners in future collaborations. CRKN continues to explore how to build on these relationships and develop more formal linkages and communications to engage strategically with these organizations.  The national funding agencies (Canada Foundation for Innovation, Canadian Institutes of Health Research, Natural Sciences and Engineering Research Council of Canada, Social Sciences and Humanities Research Council)  Organizations that support digital infrastructure (Canadian Access Federation, CANARIE, Canadian University Council of Chief Information Officers, Compute Canada, Synergies)  Organizations representing the university research community (AUCC, Canadian Federation for the Humanities and Social Sciences, Canada Research Chairs, Centres of Excellence, etc.) 17 1.3 Platform planning process CRKN undertakes strategic planning on a 3-year cycle, involving a 4-phase iterative process: Communications Outreach, Strategic Planning, Operationalization, and Performance Measurement. The aim of this process is to ensure that CRKN’s activities are based on the requirements of members, and also take into account major trends. The Communications Outreach, described in more detail below, involves gathering comprehensive feedback from members and other stakeholders in order to ensure that CRKN’s strategic plan is based on stakeholders’ needs and priorities. Currently, CRKN is in the final year of its 2010-2012 Strategic Plan, and has been successful in achieving many of the objectives and making good progress on others. The planning cycle for 2013-2015 has begun with a new Communications Outreach completed during spring 2012. This base of information, coupled with ongoing consultations with members, will inform the Board’s strategic planning process taking place in January 2013 and for implementation in the 2013-2014 fiscal year. Formally approved plans Strategic Plan: The organization’s first strategic plan was developed in September 2001, following successful implementation of the grant-funded license agreements under CNSLP and covering 2001 to just prior to incorporation in 2004. Subsequent plans for 2004-2006 and 2007-2009 were developed through iterative consultative processes, guiding CRKN as a not-for-profit organization through periods of major program expansion, organizational development, also financial constraint. Most recently, CRKN strengthened its collaborative planning process in development of the 2010-2012 Strategic Plan, a process that was recognized through an award of excellence from the International Association of Business Communicators. The 2010-2012 Strategic Plan outlines three strategic directions with related objectives: Direction #1: Enhance our products and services Objective 1.1: Develop and manage cost-effective and stable content licensing agreements. Objective 1.2: Develop tools to aid in content selection and management. Objective 1.3: Work collaboratively with content creators and publishers on best practices, new access/economic models, and alternative publishing models for provision of expanded and diversified digital content. Direction #2: Serve our community Objective 2.1: Develop engagement strategies with priority communities to share knowledge, achievements and content priorities. Objective 2.2: Measure performance and provide meaningful evidence of CRKN value to members, their user communities, and diverse stakeholders. Objective 2.3: Identify priorities for and develop partnerships with like-minded organizations in areas of mutual interest and benefit. Objective 2.4: Encourage collaboration among members to leverage the expertise in our community. Direction #3: Focus our organization Objective 3.1: Strengthen governance, management, operations and planning processes. Objective 3.2: Restructure budget for multi-year planning and sustainability. 18 Communications Outreach: An important part of CRKN’s strategic planning process is the Communications Outreach. This comprehensive communications initiative gauges member satisfaction in order to develop a strategic plan that is based on a solid understanding of members’ needs. The recent Communications Outreach that will inform the 2013-2015 Strategic Plan adhered to the best practice and award-winning methodology undertaken in 2009-2010. The Communications Outreach gathered information in both qualitative and quantitative formats, through three stages: Stage 1 – Brainstorming sessions: Three (2 English and 1 French) brainstorming sessions with CRKN member libraries took place in April 2012, using an online platform. Participants were invited to participate for half an hour each day for three days. They received three questions per day, and were encouraged to comment on the responses of others. All participants were anonymous. The sessions were moderated and the report generated by a third party as per market research best practice. Stage 2 – Library Consortia Workshop: Stage 2 was designed to connect with CRKN’s stakeholder community, with the priority placed on the other library consortia in Canada. On May 30, 2012, CRKN hosted a workshop with library consortia and related stakeholders in order to share information and best practices, consider new areas for collaboration, and avoid duplication of effort and licensing activity. Stage 3 – Member e-survey: The third stage of the outreach consisted of an online survey for member library staff and took place from June 18-29, 2012. The e-survey incorporated the themes and priorities identified from the qualitative input of Stages 1 and 2, and resulted in representative, quantitative input from members. Responses to the e-survey also allowed CRKN to measure progress against targeted benchmarks from the 2009-2010 member e-survey. Building on the Communications Outreach, the themes that emerged from these consultations were developed into workshop topics at the CRKN’s Annual General Meeting in October 2012, allowing feedback to members on survey results, face-to-face engagement among members, and development of additional context and perspectives to feed into the Board’s planning. Other formal plans have been developed by CRKN in the areas of risk mitigation and financial planning. Risk Mitigation Plan: In 2009, the Finance & Audit Committee raised the importance of a risk management/mitigation plan for the organization, focused primarily on financial risks. Subsequently, staff worked to develop a more comprehensive risk management framework that considered risks to CRKN’s strategic & organizational integrity, reputation & image, content licensing program, financial health, operations, and human resources. Development and utilization of the framework, as well as Board oversight regarding the adequacy of risk measures in place, continue as a work in progress. Ultimately, the goal of this framework is to instill a mindset of systematic risk appraisal, mitigation and management in all of CRKN’s activities, as a best practice of governance and management. In addition to this overarching framework, CRKN management prepares an annual Statement of Compliance for the Board of Directors, confirming that CRKN has in place the necessary protections (e.g. 19 tenant insurance, Directors and Officers Liability Insurance) and has fulfilled all statutory and organizational requirements (e.g. payroll deductions at source, tax remittances, filing of reports, etc.) This systematic review provides confidence that the assets of the organization and investments of members/funders are not at risk, and that both CRKN and members of the Board are not exposed to liabilities. Multi-Year Financial Model: In addition to the annual operations budget that is approved by the Board, CRKN has implemented a multi-year financial model that projects the ongoing effects of decisions. This model is an important tool as CRKN undertakes a planned, predictable process to reduce its operational reliance on interest revenues, introduce new sources of revenue, reduce/contain costs, and adjust membership fees – all toward achieving a balanced budget following several years of planned deficits. In situating current year revenues and expenditures within a longer timeframe, this financial model helps to avoid short-term thinking and allows CRKN to better anticipate timing and requirements of future initiatives. The stakeholder role As stated earlier in this report, CRKN members and stakeholders are actively involved in the organization’s planning and performance assessment processes. In addition to participating in the Communications Outreach and strategic planning, members and key stakeholders are also involved directly in CRKN’s core business of content licensing. Licensing Renewal Process: Each year, CRKN negotiates new licenses or license renewals with publishers and content vendors. For renewals, CRKN undertakes product surveys to assemble systematic feedback from members, and also holds conference calls to provide information and answer questions regarding new or renewing licenses. These interactions allow members to identify vendor performance issues or deficiencies, and identify new areas for development and value creation. The systematic process for gathering and analyzing member feedback helps shape CRKN’s negotiations objectives, identify priorities for problem resolution or service enhancement, and define requirements for the resulting license agreements. The effectiveness of this approach is illustrated in an example concerning member dissatisfaction with the Globe and Mail newspaper archives that were licensed through the vendor ProQuest but hosted by the Globe and Mail. Based on member feedback and dissatisfaction with database functionality, CRKN made platform performance and usage a priority during renewal negotiations with ProQuest. As a result of the attention CRKN brought to bear on this issue, ProQuest took up the matter with the Globe and Mail, with CRKN helping to build the business case to resolve the platform and functionality problems. As a result, the Globe and Mail agreed to migrate the product to the ProQuest platform, boosting the product’s accessibility, extending functionality, and allowing users to integrate searching of this product with other ProQuest databases. CRKN’s clout as a national organization and its strong working relationships with vendors make the organization effective in achieving members’ objectives and priorities. Consultation with Regional Consortia: CRKN meets on an ad-hoc basis with regional consortia in order to identify what licenses are in the national interest and relevance. Together, CRKN and regional 20 consortia identify products that CRKN and others should consider as licensing priorities over both the short and longer-term. These consultations and meetings will take place on a more formal basis in the future. Annual General Meetings: Each year, CRKN hosts an Annual General Meeting (AGM) at locations that rotate among the regions. During these events, members and representatives from other stakeholder organizations (regional consortia, CARL, etc.) have an opportunity to discuss and debate issues through workshops, program, and townhall sessions. In addition, the AGM business meeting is the venue in which members exercise decision-making on by-law changes, appointment of auditor, changes in membership criteria, and governance through an elected Board of Directors. The AGM also provides an opportunity for members to provide input and help shape changes CRKN services and operations. The AGM is attended by member library directors and other library staff involved primarily content acquisition, but may also include outside speakers or participants depending on the program agenda. The AGM is an important way of building CRKN’s national community, accounting to members, and recognizing the leadership and contributions of peers. Participation has increased from 59 at CRKN’s first AGM in 2004, to 102 in 2011, with a record high of 190 participants for the 2009 online-only AGM. Evaluation by AGM participants indicates that the event is highly valued for collaboration, educational and networking purposes. External influences The CRKN Board of Directors, Committees, Task Groups and staff are continually monitoring external trends in the academic publishing and research environment. This is a rapidly-changing landscape in which CRKN must remain responsive to evolving needs, challenges and opportunities. Many external factors have an influence on CRKN’s operations. These include political developments, technological innovations, shifts in research funding priorities, changes in program and research specializations at universities, and changes in acquisition budgets at member libraries. These influences, in turn, will have an impact on the strategic priorities and licensing activities of CRKN. Open access is one of the most important developments that has an impact on scholarly publishing and communications. In 2008, CRKN developed a position statement on alternative publishing models in scholarly communications, complementing the advocacy work of CARL. More recently, CRKN and CARL developed a joint working group to make recommendations on how the two organizations could best leverage their respective roles and efforts in order to advance open access in Canada. Changes in digital technologies also have a major impact on the types of resources licensed by CRKN. As technologies evolve, the amount, range and functionality continues to expand as do the expectations and requirements of researchers and students. For example, in 2000, the types of digital resources licensed by CRKN were almost exclusively e-journals and citation databases. Today, diverse digital content such as streaming audio and video with interactive Web 2.0 capabilities are the norm. These developments continue to push the boundaries of technology, enabling new research approaches and connections, but also increase the complexity of the environment in which CRKN and its members operate. 21 Finally, as a national not-for-profit organization CRKN must be attuned to relevant federal legislation and policies, as well as understand the provincial legislation and requirements that affect member universities. For example, CRKN monitored development of the new Canada Not-for-Profit Corporations Act and ensured organizational compliance when the legislation came into force in October 2012. Equally, CRKN stays informed on GST and HST developments within the different provincial jurisdictions, to ensure that operations are compliant with tax legislation and are in line with financial systems and management at member institutions. Internal performance monitoring CRKN benchmarks its internal performance against objectives identified in the strategic plan, and employs a number of mechanisms in order to monitor its management and governance. The Executive Committee evaluates the Executive Director’s performance annually, and staff undergo annual evaluations with their managers. Other specific performance monitoring activities are as follows: External reviews: CRKN has been through 3 external reviews by established third party consultants. The reviews have provided insight into the operational effectiveness of CRKN, and contained recommendations on changes to practices and procedures that enhance organizational processes and strengthen the organizational culture of CRKN. In 2013, CRKN will go through another external review to examine the organization’s licensing principles and provide recommendations in terms of strengthening the transparency of the organization. Governance Review: In 2010, CRKN undertook a Governance Review as part of an ongoing commitment to good governance. This review was led by the CRKN Governance Committee, and offered an opportunity for CRKN and its members to consider the size and composition of the Board of Directors in the context of the organization’s mission, vision, values, and strategic directions. The review recommended a restructured Board of Directors, a more than 30% reduction in Board size, greater representation from member libraries, and revised by-laws complying with the new Canada Not-forProfit Corporations Act. The recommendations contained in the Governance Review were endorsed by members at the 2011 Annual General Meeting and are currently being adopted by CRKN. The flowchart in Appendix 4 provides details of the Governance Review. Organizational Indicators: The CRKN Value, Influence, Trends and Leadership (VITaL) Task Group has compiled a framework of indicators according to which CRKN can benchmark its performance. The indicators fall into 3 areas: 1. Membership Leverage 2. Scholarly Content Offerings 3. Community Engagement The information, which began to be collected systematically in 2010, is used to demonstrate the value of CRKN activities to its members, and is used by the Board of Directors, Management and stakeholders to evaluate the impact CRKN is having as it works to achieve its objectives. 22 1.4 Cultural, organizational and structural change enabled at the various stakeholder organizations CRKN has had a profound and enduring impact on the availability and use of electronic research content in Canada, leading to significant structural and organizational changes within Canadian academic libraries. Impact on stakeholders Surveys and focus groups over the years have shown that CRKN and its predecessor, CNSLP, have had a number of important and positive impacts on member libraries. Most recently, a 2012 member survey revealed that about 75% of the 114 respondents considered CRKN to have had a transformative impact on their libraries (Figure 3). (Partners Marketing Inc., 2012, p. 39) Figure 3: Agreements with statements of impacts of CRKN work Several respondents provided specific comments about the type of transformative impact CRKN has had on their library (Partners Marketing Inc., 2012, p. 5):         Savings in staff time that would otherwise be invested in negotiating with vendors The opportunity for many academic libraries to get together in a common venue for a common purpose, allowing us to learn from each other A greater range and depth of scholarly content than members can acquire on their own, making teachers and students more competitive because of an enriched information environment Better pricing for electronic content through economies of scale and knowledge Credibility of pricing and acquisitions decisions Technical expertise back-up for library content Standardized licensing to make content accessible in ways that match our users' needs, and allow us (at least in Ontario) to ensure perpetual and enduring access through local loading Predictable costs for budgeting 23   Support in challenging Access Copyright Resources and expertise to negotiate with publishers that we do not have in-house, who would be costly to acquire Chief among the impacts of CRKN is the spirit of cooperation that has been created through the program. CRKN is one of the largest and most impressive examples of universities collaborating to reduce institutional and regional disparities, and to share vitally important research infrastructure at a national level. CRKN was one of the first endeavors to engage institutions across provincial jurisdictions and has set a precedent for nation-wide collaboration. In addition, CRKN has been a catalyst for the development of a community of practice around content licensing in Canada. CRKN’s predecessor, CNSLP, was launched at a time when large scale electronic content licensing was very new and few librarians in Canada had expertise in this area. However, through their institutional membership in CRKN, librarians from across Canada have been able to learn about licensing practices, discuss issues and challenges, and share best practices. Through CRKN, a whole cadre of expertise has been developed in the area of collections and acquisitions in Canada. Other impacts derived by library members from their participation in CRKN are: financial benefits, greater availability of scholarly content, and accelerating the transition to electronic journals. Financial benefits: A major consequence of the CRKN collaboration has been the significant and ongoing financial benefits for participating institutions. Prior to CRKN, price increases by publishers were unpredictable and played havoc with universities’ ability to budget for knowledge resources. With annual cost increases for content routinely exceeding the general rate of inflation, universities were caught in a downward spiral of paying progressively more for less content. CRKN licenses made it possible for universities to meet the expanding needs of their researchers, providing them with digital content at a price the system could predict and afford. Since its inception, CRKN has purchased more than $767 million (in 2012 Canadian dollars) of content on behalf of its member institutions. Estimates suggest that if CRKN-licensed content were acquired on an institution-by-institution basis, the costs to CRKN members would be nearly $2.2 billion over the same time frame. The “vendor price” figure of 2.2 billion is calculated based on publisher quotes and standard institutional prices gathered from institutions, wherever possible. Although CRKN cannot identify the “vendor price” for every licensed resource, data that has been collected show that CRKN is paying about 30-40% of what institutions would pay if licensing the content on their own. Through negotiations, CRKN has also been very successful in achieving reductions from the vendors initial Request for Proposals (RFP). An analysis recently undertaken by CRKN found that for the renewal of seven regular licenses for 2012-2014, CRKN was able to negotiate products for a cumulative $18.03 million, less than the RFP price of $18.3 million, and in contrast with the vendor price of $126.89 million outside the consortium. Further financial benefits have been achieved through the efficiency of centralized licensing. Each CRKN member need not have staff specifically devoted to licensing content for the institution and can redirect staff resources elsewhere. Furthermore, the specialized negotiations skills of CRKN staff bring greater 24 efficiency and effectiveness of the licensing negotiations. A time-tracking pilot undertaken by CRKN in 2011 demonstrated that CRKN licenses deliver significant value through contract efficiency. For the pilot, CRKN staff tracked time spent on the SAGE license renewal in order to understand what scale of time savings a CRKN renewal process can deliver to members. The findings point to time savings of at least 13.6 hours per CRKN member, or 910 hours in total. In order to gather further information about these types of cost savings, CRKN will be undertaking other time-tracking pilots for select 2012 renewals and the contract efficiency measures will be further developed. Increased depth and breadth of scholarly content: Another important impact of CRKN has been an increase in the volume of content available to members. This impact has been especially profound for medium and small universities that cannot afford to subscribe to a large number of licenses on their own. Over the years, CRKN has more than quadrupled the number of licenses and agreements it negotiates and manages for its members (Figure 4). CRKN now manages 52 licenses with 2,616 individual agreements across the membership. This represents a doubling of content since 2007 before the DCI project. Figure 4: Number of CRKN Managed Licenses CRKN has also greatly expanded the variety of content types available. The initial CNSLP project focused mainly on electronic journals and indexing databases in the science, engineering, health and environmental disciplines. Since then, and with help from a further investment from CFI, CRKN has expanded its portfolio to include electronic journals in the social sciences in the humanities, as well as primary source material, e-books, music and image databases. Figure 5 provides a breakdown of CRKN content types according to costs for 2012. 25 Figure 5: CRKN Portfolio of Content by $ Value for 2012 Accelerating the transition to electronic journals: A major objective of the original CNSLP project in 2001 was to accelerate the adoption of electronic information delivery by university libraries, which was already under way at a number of institutions, but by no means common. Prior to CNSLP a number of university libraries had begun moving away from print formats to electronic delivery (e-journals), often through regional consortia. However, progress across the country was uneven, transaction costs were high2, and prices were not the best available because the volume of purchases was not large. By procuring a series of national site licenses and supporting their implementation, CNSLP intended to accelerate plans to adopt e-journals and to reduce the financial and technical risk for universities in their transition to electronic information delivery. The original CNSLP project was very successful in accelerating the adoption of electronic resources in Canada. Bringing new partnership money to the table through CFI and other funding partners lowered libraries’ e-journal entry costs and the risks associated with it, and made it easier for libraries to justify an increased university investment in the required electronic information delivery infrastructure. A 2004 survey found that CNSLP’s national site licensing activities increased all libraries’ holdings to the full 718 STM titles of these key publishers, representing a 446% increase in access on average across all member libraries. In other words, on average researchers at institutions that previously held 134 titles immediately gained access to an additional 586 scientific publications. (The Impact Group, 2004) 2 Each university had to negotiate its own arrangements and this required significant staff time. 26 Cultural impact Digital content has become a critical underpinning for research in the 21st century. Across all fields of research, there is a growing reliance on digital content in support of research activities. By improving access to digital content for researchers across Canada, CRKN has helped to support researchers in their acceptance and transition from print to electronic resources. A 2004 survey of researchers looking at the impact of CNSLP found that the use of e-journals had nearly doubled in all university categories over the first 2 years of the project. (The Impact Group, 2004) Ithaka S+R, a US-based research and consulting service that conducts large-scale surveys with faculty members internationally, reported in 2010 that there had been a profound and rapid shift in research practices towards reliance on information discovery via network-level electronic resources. This trend was consistent for researchers from all domains. In their survey, over 80% of faculty indicated eresources were very important for their work (Schonfeld & Housewright, 2010, p. 23). Figure 6: Percent of faculty responding “very important” to the question “For each item that you use, please indicate how important that item is to your research or your teaching.” (Schonfeld & Housewright, 2010, p. 23) The impacts of electronic resources on research practices include greater efficiencies and serendipitous discovery. A comprehensive survey conducted in the UK in 2011 reported that libraries’ subscriptions are by far the primary source of article readings. The survey also found that, “electronic collections allow academics to access information from outside the physical library. This saves time in locating and obtaining articles and increases the time they are able to spend on work” (Tenopir & Volentine, 2011, p. 7). In addition, through the browsing process, academics discover on average “seven articles in addition to the one they located” suggesting that access to electronic content via the library facilitate the “discovery of additional relevant articles” (Tenopir & Volentine, 2011, p. 6). Testimonials gathered by CRKN offer additional anecdotal evidence of the cultural impacts of electronic resources on researchers: 27 Dr. Patrick Fournier Dr. Patrick Fournier, Full Professor, Department of Physics, Université de Sherbrooke: The transition to online resources brought about many changes for Dr. Fournier’s research, the most significant of which is the considerable increase in the quantity of information available. As a researcher, it is essential for Professor Fournier to be up to date on everything published in his field. Prior to the advent of online resources, Professor Fournier would read about five or six scientific journals every day. Today, he tries to get through about 30 publications on his computer. Much of Professor Fournier’s work takes place in his laboratory. The Internet enables real-time adjustments during experiments. Formerly, in the event of questions or problems, he would have to stop the experiment, do some research in the library and sometimes wait for days or weeks to obtain the required research documents in order to continue. Dr. J. David Clemis, Associate Professor of History, Mount Royal University: As a researcher at one of CRKN’s newest institutional members (since 2010), Dr. Clemis is already noticing the benefit of recent CRKN investments in humanities and social sciences content through the DCI project. Mount Royal now benefits from unprecedented access to historical and literary studies resources. Dr. Clemis notes that access to these collections will vastly enhance opportunities for European, Canadian, and American historical study and scholarship. They will provide new opportunities and allow his students to do real, substantive research. He anticipates that in his courses alone, more than 60 students per Dr. David Clemis semester will make use of these resources to undertake real, substantive research. Students in the honours stream of the BA in history will now have the resources necessary to prepare honours theses in European history comparable to those at any university in the country. Branding CRKN is a recognized world-leader in the area of licensing content and has been recognized internationally within the library and licensing communities for its activities and achievements. Furthermore, Canadian researchers have become recognized internationally as having access to a rich array of scholarly resources; often, far more resources than are available elsewhere. In 2011 CRKN commissioned a study to measure the impacts of its activities with the research community. The study involved focus groups of researchers at four universities across Canada. Participating researchers overwhelmingly agreed that having access to online scientific information was “helping them to build research networks at home and abroad.” (The Impact Group, 2011, p. 13) The report states, “Canadian university researchers increasingly find themselves at the centre of the networks, acting as ‘information gatekeepers’, because in many instances they have access to a larger body of information than their colleagues at universities in many other parts of the world. Canadian researchers are now seen as being 28 so well connected to current sources of information that they have become magnets for international collaboration.” (The Impact Group, 2011, p. 13) The study also found that investigators at smaller institutions were especially aware of how having expanded access to online information has raised the profile of their research. Statements from experts in the library and licensing communities further reveal that CRKN is recognized and highly regarded internationally. Julia Blixrud, Assistant Executive Director, Scholarly Communication, for the Association of Research Libraries writes (From private communications), Julia Blixrud “CRKN has created an efficient mechanism to maximize the shared investment of its members to acquire scholarly content. It has demonstrated the value that a coordinated, collaborative model can have by increasing the availability of information to a research community in a cost-effective manner. The investment made in CRKN has been much more than seed money – it provides a demonstration of the value that Canadians place in support of Canadian research in a global environment. It has served as a positive example for other countries.” In a 2012 survey of members, participants were asked to contribute words they felt most accurately described CRKN. Their responses were turned into a “word cloud” contained in Figure 7. French words were translated into English. The size of the words indicate the number of times each word was mentioned; the word colours are random. The most common adjectives describing CRKN were National, Helpful, Collaborative and Effective. Figure 7: Word cloud representing members’ descriptions of CRKN 29 2.0 Research Capacity 2.1 Capital investment value Value In the case of CRKN, “capital investment value” is the value of the licenses negotiated through the organization. CRKN’s investments in licenses have grown exponentially since the inception of the CNSLP. In 2001, the original investment amounted to $50 million over 3 years (or approximately $15 million in 2001). This has risen steadily over the last 11 years to a maximum of over $100 million in 2010 and $89 million in 2012. The vast majority of the funding comes from member universities. Figure 8 shows the levels of investment from members, CFI, and provincial partners and illustrates the rapid growth in investments in CRKN from 2001 to 2012. Figure 8: Investments in CRKN from 2001 to 2012 30 Enhancements Over the years, CRKN has adapted and revised licensing terms to improve access and usage provisions for users. These amendments have included requirements for usage data from publishers, the right to locally load electronic copies, and inclusion of content in course packs. Many vendors also now provide support for Shibboleth, an open-source implementation for identity-based authentication and authorization that CRKN support helped to advance through the recently-implemented Canadian Access Federation, which provides federated access management services for identity providers (including universities and libraries) and service providers (such as publishers). CRKN continually seeks to improve its productivity through more efficient membership engagement and administrative workflows. In terms of its core function negotiating licenses, CRKN has been working to refine its licensing processes as follows: Unbundling of licenses: When CRKN began as the Canadian National Site Licensing Project (CNSLP), license costs were shared according to a base allocation across four regions of the country. At the 2005 Annual General Meeting, members ratified a resolution approving a new license cost‐sharing model and principles. The model proposed that 100% of annual ‘core’ licensing costs be apportioned among institutions according to their CRKN Research Metric3 percentage. Cost‐sharing for non‐core licenses was to proceed on an ad‐hoc basis using models developed by participants. The metric was applied to a bundle of core licenses that was renewed in 2004 and 2007. Core licenses were unbundled in the 2010‐ 2012 renewal period to give members greater flexibility in license participation and to improve the sustainability in the event of changes to participation. The aim was to remove historical cost-sharing formulas when licenses came up for renewal, but only if there were likely to be no significant impacts to individual members and overall license participation. Currently 75% of CRKN content has no cost-sharing component and the remainder is comprised of a mix of five different models. Expedited licensing negotiations: Over the years, CRKN has also worked to streamline its license renewal processes in order to become more efficient and make more strategic use of its resources. Licensing renewal is an extremely time consuming and resource intensive process. So, in 2010, CRKN introduced an "expedited" renewals process, applied when the renewal is routine. The process can be used when the following conditions apply: There is an existing license agreement for the product; the ongoing access fee is proportionately small; where CRKN has member input about the specific publisher; where no substantive technical or service issues have been raised by members; and where negotiations may cost more than the subsequent savings achieved. Automating office tools: CRKN has implemented office automation software systems to automate routine applications such as accessing a membership database and document management system. 3 CRKN Research Metric was developed using indicators such as FTEs, research intensity, etc. to allocate costsharing percentage. 31 Tools were chosen to simplify registration for teleconferences and the AGM, as well as to automate the voting system. New services: CRKN has enhanced its operations with the development of a number of new services, additional staff members, and several new working groups. The following initiatives have been launched since 2001:  License Information Module (LIM). In 2009, CRKN launched the self-serve License Information Module (LIM). The LIM is an open source information management system custom-designed to support CRKN’s members by facilitating access to license information for all the CRKN agreements. The LIM provides profiles for all CRKN license agreements, downloadable title lists and contracts for each agreement, and institutional profiles that highlight members’ current and previous license participation. Members can run reports to view and compare license participation across the membership, view and compare product features and license terms across agreements, and view a list of all current agreements sorted by renewal date. The system offers an RSS-enabled Product News section, vendor/publisher directories, a financial update section that offers an invoicing schedule for planning purposes, quarterly financial updates and a feedback form. The LIM has saved CRKN member libraries both time and resources in the management of these collections, and has eliminated duplication of effort among CRKN members. It is an innovative and highly valued service for which CRKN was awarded the 2011 Canadian Association of College and University Libraries Innovation Achievement Award for its implementation. According to Scott Shannon, Library Assistant at the University of New Brunswick, “The implementation of the LIM by CRKN was one of the best steps forward the organization has made in the 10 years I’ve been managing access to this type of content.” (CRKN, 2011, p. 9). The site is now bilingual and supported by a self-serve tutorial (LIM 101).  Building the community of practice. CRKN has significantly improved its methods for member communication. Since 2001, CRKN has implemented numerous new mechanisms for engaging and informing members. The 2012 survey notes significant changes in overall quality of services (16%) and communications (35%) since 2009 (Partners 2012, p36). While these activities are directed to CRKN members, the communications are all available on the CRKN website and contribute to the visibility of CRKN beyond its immediate stakeholder community.  Task group work. Several new Task Groups have been formed to address issues of national importance to CRKN including the impact of open access, preservation and local loading of content, revisiting cost-sharing practises and how to better integrate vendor content with individual cataloguing systems. 2.2 Operating & maintenance investments CRKN is a lean organization that is focused on efficiency, steady growth and conservative management. CRKN licenses are invoiced at cost and CRKN spent $87.7 million for this purpose in 2012. (Figure 9) An extremely low proportion of CRKN’s budget is dedicated to operations. In terms of expenditures, 97.91% of CRKN expenditures are for content licenses. Operations – including administration, office, salaries and benefits, professional fees, and governance– account for 2.09% of expenditures (Figure 10). 32 CRKN’s primary revenue source for operations is derived from membership fees and interest revenues. Membership fees range from $3,677 to $31,519 per member per year, for a total of $916,350 in 2012. CRKN has received no funding from the CFI’s Infrastructure Operating Fund, and has received only very minor one-time funds from CFI for O&M. It should be noted that CRKN has been running a planned deficit for the past 2 years. To address this, CRKN is entering the second year of its multi-year financial model designed to reduce the organization’s reliance on unpredictable interest revenue, increase revenue from membership fees and other sources in a planned and predictable way over several years, and reduce and contain costs wherever possible. Figure 9: CRKN 2011-2012 Revenues Figure 10: CRKN 2011-2012 Expenditures 33 2.3 Platform Capabilities and Sustainability Current infrastructure CRKN does not itself manage physical infrastructure, but rather negotiates the conditions for access to digital content hosted by scholarly publishers and vendors. The CRKN Model License Agreement sets out the most favorable access conditions for users, which include the rights of users to:    24/7 remote access for authorized users year round Electronically save parts of the licensed materials for personal use Distribute for teaching purposes to all individual student authorized users in a class at a member institution  Incorporate parts of the licensed materials in printed course packs and electronic reserve collections for the use of authorized users During licensing negotiations, CRKN aims to secure all of the conditions set out in the model license in order to provide the most flexible use environment for researchers and students. In terms of its core function, CRKN is considered “state of the art” worldwide. The CRKN Model License reflects international best practices as outlined in the International Coalition of Licensing Consortia (ICOLC) statements and guidelines. The CRKN licenses secure the greatest and most flexible access possible for their users, and the conditions negotiated by CRKN are comparable or better than licenses negotiated regionally and internationally. Service functions CRKN manages an annual budget of close to $90 million dollars, maintains 2,616 contracts with dozens of publishers, and institutional agreements across 75 member universities. Organization staff must manage currency fluctuations, foreign currencies and deal seamlessly with 8 different provincial tax jurisdictions. CRKN is audited regularly and has always received “clean” audit reports with no management letter, providing evidence of a well-run financial system. CRKN continually strives to improve its process and use the information provided via systematic member input to improve licensing negotiations. As outlined in section 2.1, and further described in section 3.2, CRKN has added a number of other new services for members since 2001. In terms of communications, CRKN has implemented numerous mechanisms to provide members with information and updates about CRKN activities including the monthly NewsBrief, OpenLine issue-based teleconferences, improved member services, etc. While publishers provide the access services to the content, CRKN has mechanisms through its Request for Proposals process that ensures publishers maintain levels of service quality, and works with publishers to improve service quality when there are issues identified by members. As discussed in more detail in Section 1.4, a 2012 survey conducted by CRKN found a high level of satisfaction by members with the content and services provided by CRKN. 34 The platform personnel Content licensing operates using a detailed process requiring a mixture of information management skills and business expertise. Negotiators must have both a deep knowledge of the needs of users, as well as a solid understanding of business practices and contracts. The UK Serials Group writes, “Publisher licenses present libraries – and their patrons – with useful rights but also with important responsibilities, and a thorough understanding of licensing language, and its effect, has become critical... The key for information professionals is to gain a good understanding of these new dynamics and develop the skills which will enable them to gain the best from every license and each negotiation.” CRKN’s staffing requirements are rigorous, and employees work in a high-paced, rapidly changing environment with diverse and competing member expectations. CRKN must also compete for qualified staff with the private, academic and government sectors, which offer good salaries and often significant benefits for employees. As a result, CRKN has had to contend with significant staff turnover, especially in the last 2 years. CRKN has 7 FTEs and a 1 Finance Officer on a contract position. Core competencies of CRKN staff include negotiations, member services/enquiries, operations, communications, and member engagement. There are currently 4 CRKN staff members responsible for negotiating licenses including the Executive Director. CRKN also relies heavily on the licensing expertise of librarians at member institutions. In particular, the Negotiations Resource Team, which consists of 10 member librarians, plays a critical role in assisting staff negotiators and monitoring the execution of agreed-upon negotiations strategies. The Member Services function is comprised of 3 people, all with Masters of Library Services degrees. They handle member queries and liaise with vendors. They are also in charge of the content program to oversee the ongoing management and renewal of content licenses. The Executive Director, who has an MLS degree, is also active in this area. Operations include financial management, facilities maintenance, and IT infrastructure. The function is carried out by 3 people; one with a CMA and MBA designation, one with bookkeeping expertise and the third with administration experience. Communications are strategically integrated into the overall operations and include member engagement, website management, market research, strategic planning and performance evaluation. This function is carried out by one person with a business background and a Master of Management Studies degree. No training is provided to platform personnel other than an “on boarding” process. A modest annual professional development budget of $7,500 is shared among the staff. CRKN staff members are responsible for executing negotiations, but the organization draws on the knowledge and expertise of the Negotiations Resource Team, and beyond that to the broader library community to identify licensing principles and priorities. As such, CRKN harnesses the collective intelligence for the betterment of the Canadian academic community. CRKN relies on an integrated network of front line acquisition and collections librarians and depends on numerous in-kind contributions of CRKN members. In 2011 for example, 57 people filled 65 positions by serving on the 35 Board, Standing Committees and Task Groups. Of these positions, 10 were senior university administrators, 1 was a researcher and 45 were library staff. These volunteers contributed 2,867 hours of their time to CRKN decision-making and program oversight. Sustainability CRKN employs a number of mechanisms to ensure the sustainability of the organization. However, it should be noted that sustainability has been an ongoing challenge for CRKN. Stagnant acquisition budgets at member libraries, combined with regular increases in vendor prices have meant that it will be difficult for CRKN to respond to members’ requests for CRKN to grow and maintain sufficient staffing support. This issue of sustainability is discussed in more detail in the challenges section of the report. A major part of CRKN’s approach to ensuring sustainability involves its 4-step planning process (described in more detail in section 1.3). This process ensures that CRKN’s objectives and activities are based on members and stakeholder input and trends identified in the broader academic environment. CRKN has numerous formal mechanisms that gather this information, after which it is integrated into CRKN’s Strategic Plan. The Strategic Plan is then used to guide CRKN operations. CRKN’s activities are based on good governance and the Board of Directors and committee structure has been developed to ensure that CRKN is accountable to members and the broader stakeholder community. CRKN’s operations are infused with a culture of risk awareness and mitigation. In addition, a 10-year financial model has been developed to ensure that CRKN returns to a balanced budget in the coming years. 2.4 Foregone Costs CRKN has realized a number of “foregone cost” efficiencies for universities and the research community. These include more improved content for less money, lower staff costs, and greater research efficiency, allowing members to re-direct their time and monies to other activities. More content for every dollar. With an annual acquisition budget of close CAD $90 million, CRKN wields significantly more power to negotiate lower prices than any individual member library would have if negotiating with publishers alone. While CRKN cannot identify the “vendor price”4 for every licensed resource, data that has been collected show that CRKN is paying about 30-40% of what institutions would pay if licensing the content on their own. Through negotiations, CRKN has also been very successful in achieving significant reductions from the vendors initial Request for Proposals (RFP). Not only do members benefit from large-scale negotiated pricing, they also gain protection from excessive annual increases. CRKN negotiates price protection with caps on annual increases set below market norms. For example, for seven recent license renewals and eight DCI Project renewals, CRKN 4 The “vendor price” figure of 2.2 billion is calculated based on publisher quotes and standard institutional prices gathered from institutions, wherever possible. 36 negotiated prices that represented about 19% of the publishers vendor price for the next 3 years. (Figure 11) Millions of dollars Figure 11: Comparison of Vendor Prices with CRKN Negotiated Prices for regular and DCI renewals 50 45 40 35 30 25 20 15 10 5 0 Vendor Prices CRKN Negotiated Prices 2012 2013 2014 Looking specifically at the renewals of Swets and Elsevier licenses, the value of the foregone cost savings for CRKN members is further corroborated in Figure 12 with a cumulative negotiated price of $88.6 million compared with the vendor price of$181.3 million outside the consortia. Figure 12: Comparison of Vendor Prices with CRKN Negotiated Price Lower staff costs: Another foregone cost is the efficiency achieved through savings in staff time that would otherwise be invested in negotiations with vendors. Input from a 2004 study and 2012 members survey establish that savings of staff time from not having to undertake negotiations at individual libraries is one of the important impacts that CRKN has on its members. The survey results are further supported by a recent time tracking project undertaken by CRKN. The project, described in detail in Section 1.4, that found that CRKN realizes significant time savings for member libraries. 37 A quote from the 2012 survey (Partners Marketing Inc., 2012, p. 5) highlights this: “CRKN has brought HUGE value to our Library. From its very inception (and in its "roots" as CNSLP), CRKN has allowed us to greatly expand the range and depth of scholarly content that we provide to our academic community. The power of consortial licensing and shared expertise has provided us with much better pricing, many efficiencies in negotiating with vendors, and standardized licensing which ensures that vendors give us their best deals financially, make content accessible in ways that match our users' needs, and allow us (at least in Ontario) to ensure perpetual and enduring access through local loading. I am an older librarian, and I can remember when none of this was true, or was much more difficult to achieve when each library negotiated on their own. As a mid-sized institution, we were not able to have much leverage when dealing with publishers, and every publisher had a different business model and different restrictions around access, quality of MARC records, etc. We invested a lot of staff time in doing individual negotiating and problem-solving with multiple vendors. I don't miss those days at all, and CRKN has helped enormously in moving us forward.” Research efficiency: Studies and focus groups over the years have found that access to CRKN resources has contributed to improving the efficiency of research, both in terms of costs savings and time savings. At the 2011 focus groups, many researchers reported being able to monitor more information, within and across more fields, more efficiently than ever before. They have access to an expanded range of online information within their disciplines and are using strategies such as pre-programmed information searches to obtain up-to-the-minute information about research advances in their field. Focus group participants also consistently reported that they spend less time travelling to and working in their libraries, which leaves them more time to work with students, prepare for teaching, conduct research and carry out other academic duties. A 2011 UK survey of researchers had similar conclusions, finding that the “library’s search tools and e-journal collections save the readers’ time in terms of obtaining quality material more rapidly. Electronic sources allow the reader to obtain the article from their office or lab, and they rarely read in the physical library” (Tenopir & Volentine, 2011, p. 3). Saving money - especially for travel - was another dominant theme in the CRKN focus groups. Largescale content acquisition not only saves libraries money through bulk purchase discounts, it also has a direct impact on researchers’ budgets. Many researchers reported that a combination of online access to information together with new digital collaboration tools reduced their need to travel and thus had a direct impact on their budgets and on their ability to do research. 2.5 Highly Qualified Personnel Quality and richness of the training activities and environment 21st century research skills involve navigating and deciphering volumes of information resources in new formats across new mediums. According to Industry Canada (2012), “digital skills can be understood as the ability to locate, organize, understand, evaluate, create and share information using digital technology”. Digital skills are “important, not only for the ICT sector, but for the entire workforce, as well as all other Canadians, be they homemakers, students or seniors.” 38 By providing access to state-of-the-art information resources and services through CRKN, Canadian universities are creating an enriched environment for training highly qualified personnel. CRKN is contributing to the development of digital skills amongst university students in Canada, ensuring that students have the most up-to-date information in their field, addressing institutional disparities and geographical location. Young researchers and graduate students have access to a full range of international sources, anchoring their work within the global context. Surveys (Davidson & Kyrillidou, 2012) conducted at several Ontario universities (representing about 25% of CRKN members) have found that the electronic resources made available by CRKN are highly used by students. A 2004/2005 survey found that 46% of the users of the electronic resources were undergraduate students and 42% of the usage was for the purposes of coursework. According to a 2010 survey, 50% of the users were undergraduate students and 55% of the usage was for the purposes of coursework. CRKN resources also enhance the teaching and learning environments at Canadian universities. A 2003 study sponsored by CRKN found that 65% of researchers agreed that having access to electronic journals made them a more effective teacher (The Impact Group, 2004). A focus group study undertaken for CRKN in 2011 further demonstrates the positive impact of CRKN resources on teaching and learning practices (The Impact Group, 2011). The study found that expanded access to online information has made the teaching process both more efficient and more effective. Researchers reported they have expanded choice in selecting course materials. One researcher reported it was easier for him to find and assemble teaching materials and that electronic resources also benefited students, who spent less time accessing and more time synthesizing information. Another participant noted that, “Online access has transformed graduate teaching; it is infinitely expanding possible topics – for example a masters student worked entirely on digital English records”. A snapshot CRKN resources are available to approximately 1.2 million people enrolled in 75 universities nation-wide including researchers, staff, and affiliates. CRKN members represent 75 of 92 AUCC members and approximately 99% of the students enrolled in AUCC member institutions are enrolled in CRKN member universities. Over 192,000 of these students are graduate students, who more than any other segment of the research community, require in-depth access to the research literature in their disciplines as part of their training and thesis writing process. Since the launch of the original pilot project, the numbers of students who have had access to CRKN licensed resources have increased significantly. CRKN has gained 11 new members since 2001, increasing the number of student users. According to AUCC (2011), enrolment of undergraduate and graduate students at Canadian universities has been climbing steadily from about 700,000 in 2000 to over 1.2 million in 2011. This represents an increase of about 500,000 potential student users for CRKN resources since 2001. The number of graduate students with access to CRKN licensed resources has also risen steadily since 2001. Figure 13 contains data provided by Statistics Canada (2010) on the growth of student enrolment. 39 Although the graph does not contain data from all years of CRKN operations, it provides a sense of the steady growth of student enrolment for a 5-year period from 2004/05 to 2008/09. Figure 13: Statistics Canada (2010) data on student enrolment at AUCC Universities 1000000 950000 900000 Graduate students 850000 800000 Undergraduate students 750000 700000 2004-05 2005-06 2006-07 2007-08 2008-09 2.6 Influence of CFI The Canada Foundation for Innovation’s (CFI) investments in CRKN have been absolutely critical for the establishment and growth of a national digital knowledge infrastructure in Canada. In 2001, CFI invested 40% of $50 million ($6.2 million per year for 3 years) in the Canadian National Site Licensing Project. Provincial governments and member institutions provided the remaining 60%. Leveraging that initial investment, CRKN has had exponential growth through further investments by its members, and has been creating additional value for members through licensing greater amounts of content and developing value added services. In 2008, CFI invested again in CRKN to further enhance access to humanities and social sciences content through the DCI project, which was matched by members and provincial partners. Without CFI’s original investments, it would have been extremely unlikely that provincial and institutional partners would have invested in CRKN, and questionable whether 75 member universities would have collaborated in order to form the organization. In the absence of CFI’s investment, Canada’s universities would be acquiring content individually and through their regional consortia, duplicating efforts across regions and universities and in many cases paying much greater licensing costs. CRKN’s original cost sharing agreements have meant that small and medium sized universities have been able to provide access to a world-class collection of digital content on par with the large researchintensive universities, essentially democratizing access to scholarly content in Canada. This has contributed to their ability to recruit and retain higher quality researchers and provide them with a richer environment for conducting their research. Tom Sanville, Director of Licensing and Strategic Partnerships at LYRASIS (a US library consortium) and active member of the international licensing community for many years, says of CRKN (private communications, 2012), 40 Tom Sanville “Without CRKN, Canadian universities’ access to the world’s research would lag behind that of other developed countries including the United States where “national” level licensing must be accomplished state-by-state. CRKN is providing the necessary means to achieve a level of cost effectiveness, control, and expended information access that individual libraries cannot hope to achieve individually. Globally, national level licensing is used as the most effective tool to keep pace with the growth in scholarly research and increased prices. CRKN is among the leaders in doing this effectively and enabling Canadian universities to compete globally.” CFI’s most recent investment in humanities and social sciences content licensing (via the DCI Project), which provided access to 14 major research collections, has further increased research capacity in Canada, by providing unprecedented access to digital resources in these disciplines. In these fields especially, primary sources and information artifacts are the fundamental building blocks of research. CFI’s investments in CRKN have meant that 99% of academic faculty and students in Canada have extraordinary access to a depth and breadth of world-class content from the convenience of their desktop. CFI’s investments have contributed to significant cost savings, improved research efficiency, recruitment of first class researchers, increased international collaborations, and greater research excellence across the entire country. Colleen Cook, Trenholme Dean of Libraries at McGill University, who relocated from the United States in 2010 talks about the value of having a national licensing consortia in an article published by the McGill Reporter (McDevitt, 2011), Dr. Colleen Cook “Just by virtue of being in Canada – national licenses for electronic information in Canada allow research libraries to capitalize upon economies of scale so that every dollar that goes to electronic resources goes a lot farther than it would in the U.S. On the collection side, the breadth and depth of our electronic holdings are great and the Rare and Special Collections here are superb.” 41 3.0 Leadership Development & Research Enabled 3.1 Users / access Policies and mechanisms CRKN licenses allow all authorized users (over 1.2 million researchers, students, staff, and affiliates) to have 24/7 remote access to CRKN licensed resources - 365 days a year. In addition, many of the CRKN licenses allow for walk-in public access to the licensed resources. The model licensing language used by CRKN is as follows: “2.1 The Publisher hereby grants to the Consortium and the Members the non‐ exclusive and nontransferable right to permit Authorized Users wherever located to access the Licensed Materials via a Secure Network and using the access methods specified in Schedule 3 for the purposes of research, teaching, private study, and administrative use associated with the normal practices and activities of the Consortium and the Members, subject to the terms and conditions of this Agreement.” CRKN licenses also secure important usage rights for CRKN members including:  Make such local electronic copies of part of the licensed materials by means of caching as may be necessary solely to ensure efficient use of such materials by authorized users and walk‐in users.  Allow authorized users and walk‐in users to have access to the licensed materials including an integrated author, article title and keyword index of licensed material, from the server via the secure network.  Provide single printed or electronic copies of single articles upon request to individual authorized users.  Allow authorized users to have access to the licensed materials through the member’s electronic learning environment.  Display, download or print the licensed materials for the purpose of internal marketing or testing or for training authorized users. In the absence of the usage rights outlined in the CRKN licenses, many institutions would be required to pay higher tariffs and fees to the various copyright collectives. 42 User community CRKN resources are available to all students, staff, faculty and affiliates at the 75 member universities. The communities served by CRKN are comprised of over 1.2 million users, including approximately 192,000 graduate students5 and over 42,000 full-time faculty members6. There has been a steady growth in the number of researchers with access to CRKN resources since 2001. According to AUCC (2011), the number of professors at AUCC universities (99% of whom are represented by CRKN) has been increasing steadily by about 3-5% per year. In addition, the CRKN user community was expanded with the addition of 11 new members since the inception of the original CNSLP project in 2001. Enrolment of undergraduate and graduate students at Canadian universities has also been climbing steadily from about 700,000 in 2000 to over 1.2 million in 2011 (AUCC, 2011). This represents an increase of about 500,000 potential student users for CRKN resources since 2001. Utilization Utilization of the platform can be demonstrated in a number of ways. CRKN collected usage statistics from some publishers from 2001 to 2004, and then began receiving more comprehensive statistics from all publishers in 2009. Between 2005 and 2008, CRKN did not have the staffing resources available to manage this type of data. Although the statistics are not comparable across publishers (for example, queries vs downloads) and there are some anomalies across years, the data show that aggregate usage of five major CRKN licenses from member institutions has increased significantly since 2001 (Figure 14). Figure 14: Aggregate usage statistics of 5 major CRKN licenses, 2001-2011 American Mathematical Society (Number of Queries) 1500000 1300000 1100000 900000 700000 500000 Thomson Reuters - Web of Science (Number of Queries) 9000000 7000000 5000000 3000000 1000000 5 Taken from AUCC 2011 enrolment data of CRKN member universities 6 Taken from Statistics Canada 2009/2010 data of teaching faculty at Canadian Universities 43 Royal Society of Chemistry (Number of Downloads) Institute of Physics (Number of Downloads) 450000 350000 250000 150000 50000 420000 320000 220000 120000 20000 2001 2002 2003 Missing 2010 Data 2011 American Chemical Society (Number of Downloads) 2000000 1500000 1000000 500000 Data from two studies conducted in Ontario also demonstrate significant usage of CRKN resources by researchers (reflecting about 25% of CRKN members). The studies used the MINES (Measuring the Impact of Networked Electronic Services) methodology, an online, transaction-based survey, to collect data on the purpose and use of electronic resources and on the demographics of users. The results of the 2004/2005 study found that CRKN content accounted for 98% of the content accessed for research purposes by survey participants. In the 2010/2011 studies, a time when libraries had access to many more sources of electronic content, CRKN resources still accounted for about 60% of all usage for research purposes. (Davidson & Kyrillidou, 2012) It should be noted that usage data is difficult and time consuming to collect and is not always an appropriate indicator of the value or impact of a particular information resource. For example, there are some resources licensed through CRKN that are used almost exclusively by specialized researchers across Canada and therefore download or query rates are relatively low. However, the resource is absolutely critical for researchers in that field. For this reason, CRKN does not base its purchasing decisions on usage statistics. In terms of international comparisons, this type of data is not publicly available and would likely be difficult to evaluate if it was available. However, Elsevier, CRKN’s largest content provider that accounts for approximately 25% of CRKN licensing dollars, has provided some comparative figures that demonstrate the high levels of usage and impact of their licensed content in Canada since the inception of the original CNSLP project. The data contained in Figure 15 shows a tremendous increase in usage of 44 the Elsevier platform, Science Direct just after the inception of the CNSLP project. It also shows much higher download rates of the platform in Canada as compared to the US and China. Figure 15: Number of downloads and number of articles published of Elsevier content Figure 16, which has also been provided by Elsevier, shows that there has been significant growth in article downloads by Canadian researchers since 2002. The graph demonstrates much higher usage of Science Direct content by Canadian researchers as compared to researchers in Australia, China, the United Kingdom and the United States. Figure 16: Usage of Science Direct by Country 45 3.2 Leadership and competitiveness Platform personnel and program competitiveness CRKN has been recognized both in Canada and abroad for its innovative procurement processes and model license. It was one of the first national licensing organizations in the world and has cultivated specialized skills and knowledge since its inception in 2001. Licensing experts from other countries recognize the value of this national collaborative approach. In the following quote, Chuck Eckman, Chair of the Negotiations Resource Team at CRKN and University Librarian and Dean of Library Services, Simon Fraser University compares CRKN with the California Digital Library (CDL), where he used to work until 2010. Dr. Charles Eckman “CRKN is much more comprehensive with a national scope. CRKN consists of 75 institutions; CDL supports a single institutional entity with 10 campuses. CRKN makes a much larger impact on the marketplace and research community by nature of its national scope. It is highly collaborative considering the geographically broad and diverse membership. CRKN's achievements and reputation are very highly regarded in the United States and this was one of the factors that encouraged my relocation.” (CRKN NewsBrief, Sept, 2012) Making comparisons across licensing consortia is extremely difficult. The types of organizations that undertake licensing differ greatly in terms of their organizational structures, missions, as well as membership criteria. For example, many organizations that negotiate licenses have other core operations, such as preservation and advocacy, and are not exclusively focussed on licensing like CRKN. Numerous other national consortia have a broader membership than CRKN, for example they represent public, special and/or school libraries as well as academic libraries. Other consortia are not national in scope, but represent a small number of very similar institutions. Licensing consortia can be federal agencies, not-for-profit organizations, or managed by a single university. These factors all contribute to a diverse international licensing community, with large differences in licensing objectives. Figure 17 contains a selective list of other licensing consortia illustrating the diversity in their activities and structures taken from the member profiles of the International Coalition of Licensing Consortia (ICOLC). The consortia listed demonstrate the range of difference in membership sizes, types and varieties of additional services offered by these types of organizations. 46 Figure 17: ICOLC Member Profiles Licensing consortia Jurisdiction Number of members Types of members Number of staff Other functions California Digital Library (CDL) California 10 campuses; 1 member Academic N/A Open access publishing and repository services; Archival finding aids and digital object aggregation; Data curation; Electronic content loading/presentation; Interlibrary loan/document delivery; Preservation; Storage facilities; Union lists/shared online catalogs; Cataloging services; Collections sharing Consorzio Interistituzi onale per Progetti Elettronici (CIPE) Council of Australian University Librarians (CAUL) Italy 27 Academic 2 Collections sharing; Electronic content loading/presentation; Interlibrary loan/document delivery; Union lists/shared online catalogs Australia 39 Academic Special 2.7 Advocacy/communication; benchmarking/statistics; quality and assessment; digital repositories; national borrowing program; statistics collection for universities in Australia and New Zealand; copyright advice BIBSAM Consortium of University and Research Libraries Canadian Research Knowledge Network (CRKN) Sweden 63 Academic 6 None listed 7 Providing a forum for member engagement on key issues relating to scholarly communications and digital content infrastructure. Working with other organizations at the regional, national and international level to help advance knowledge infrastructure for the benefit of Canadian academic researchers Special Canada 75 Academic 47 Consortium universitaire de publications numériques (COUPERIN) France 215 Academic Public Special Museums Hospitals 3 None listed Denmark's Electronic Research Library Denmark 230 Academic Public Special School 7 None listed Portal de Periódicos da CAPES Brazil 407 Academic Public School 20 Electronic content licensing Electronic content loading/presentation Preservation Training LYRASIS MidAtlantic, Northeast, Southeast, and West regions of the U.S. 1600 60 Collections sharing; Interlibrary loan/document delivery; Preservation; Training; Open Source services; Technology assessment, strategy, planning and implementation; Professional development courses; Consulting services to address library needs across a wide range of subject areas; Mass Digitization Collaborative Grant funding to support collaborative, technology, preservation and leadership initiatives, with current grants including -Ideas and Insights Series on hot and emerging topics Networking opportunities that bring library professionals together -Training needs assessment and training plan development -Leadership development programs and classes (DEFF) Academic Public Special School Government 48 Leadership Ken Frazier, a former CRKN Advisory Board member, and Director, General Library System, University of Wisconsin-Madison in the USA says of CRKN (CRKN, 2009, p.9), “No country has done a better job of national licensing of research literature than the Canadian Research Knowledge Network. Recently I have been suffering from “Canada envy”. If you want to see how a national information system can be transformed to support research progress and economic development, I say: Watch Canada!” Reflecting CRKN’s credibility and positive reputation in the community at large, CRKN has been invited to speak at over 48 national and international events in order to share expertise and knowledge pertaining to licensing of digital content. Ken Frazier CRKN has also been the recipient of several awards:  2012 IABC Excel Award of Excellence: CRKN was the recipient of an Excel Award of Excellence from the International Association of Business Communicators in the Communication Management category of Electronic and Digital Communications for the development of CRKN's first virtual Annual Report 2010-2011.  2011 IABC Excel Award of Excellence: CRKN won the IABC Excel Award of Excellence award for the Communications Outreach project directed by Dyna Vink, CRKN’s Director of Communications.  2011 CACUL Innovation Achievement Award: CRKN received the award for License Information Module, which provides profiles for all CRKN license agreements, downloadable title lists and contracts for each agreement, and institutional profiles that highlight members’ current and previous license participation.  2010 CACUL New Academic Librarian of the Year: Allison Kelley, Member Services Officer at CRKN was recipient of this award. Ms. Kelley successfully led the development of an innovative web-based license information module that has received accolades from collections managers across the country. Further, she has fostered a network of strong working relationships that unite academic libraries of all sizes from all regions, all the while addressing their diverse needs as well as their common goals.  2010 IABC Gold Quill Award of Excellence: CRKN was the winner of the prestigious 2010 Gold Quill Award for Excellence in business communication, presented by the International Association of Business Communicators (IABC). CRKN captured an Award of Excellence in the Special Events category for its Virtual AGM project. 49  2010 ORION Learning Award: The ORION Awards recognize individuals and groups who have led and championed the use of advanced and collaborative technologies to support research, education and discovery in Ontario or on the global stage.  2002 CACUL Innovation: This award recognizes academic libraries, which, through innovation in ongoing programs/services or in a special event/project, have contributed to the advancement of academic librarianship and library development.  2001 CAUBO Quality and Productivity First Prize: This award was given to the original CNSLP project by Canadian Association of University Business Officers (CAUBO) in recognition of CNSLP's national collaboration, content acquisition strategy and license procurement as an innovative business and service achievement for the Canadian academic community.  2001 CARL/ABRC Award of Merit: The Award was made to Ms. deBruijn in recognition of her outstanding leadership of the Canadian National Site Licensing Project, which provides increased access to information for the Canadian research community. Mechanisms to enhance competitiveness CRKN undertakes numerous outreach activities in order to improve its visibility and reputation with members, including:  NewsBrief: Published 10-12 times per year, the Newsbrief is published on the CRKN website and sent to members and stakeholders. It provides information about the activities of CRKN including licensing activities, new services, governance updates and so on.  OpenLine: The CRKN OpenLine is a teleconference hosted by CRKN to inform members on key topics of interest. It is a virtual event that features discussions with the CRKN Executive Director, Board and Committee members, stakeholders and invited guests.  Annual reports: The annual report provides detailed information about the yearly activities and progress of CRKN. In 2012, CRKN was the recipient of the Excel Award of Excellence from the International Association of Business Communicators for the development of CRKN's first virtual Annual Report 2010-2011 that tripled readership.  Annual General Meetings: Each year, CRKN hosts an Annual General Meeting (AGM) at which members have an opportunity to discuss issues through program sessions. Member feedback indicates the event is highly valued for networking, educational and collaborative purposes. Although librarians tend to know just how much CRKN is contributing to research infrastructure, but it remains challenging to get the word out to senior administrators and to the research community. 3.3 Linkages CRKN maintains formal and informal linkages with related regional, national, and international groups. As discussed earlier, CRKN meets with regional licensing consortia in Canada, and has ongoing informal 50 relationships with other national stakeholders including organizations representing research funders, researchers, and digital infrastructure (see Section 1.2, Roles of Stakeholders, for more details). CRKN also participates in national initiatives aimed at improving its operations and services delivered to members. One example is CRKN participation in the development of the Canadian Access Federation (CAF), a joint initiative of the Canadian University Council of CIOs (CUCCIO) and CANARIE Inc. The CAF was created to coordinate relationships between multiple identity providers (such as universities) and service providers (such as publishers of licensed content). CRKN has served on a “futures committee” to help develop the CAF business plan; encourage publishers of CRKN-licensed content to participate as service providers; and provide opportunities for education and information exchange among universities’ library and information technology communities. CRKN has joined the CAF to formalize its role in ongoing development of these services, and to help advance implementation of Shibboleth within publishers’ platforms and libraries’ access mechanisms. CRKN is also an active member of the International Coalition of Licensing Consortia (ICOLC), an informal group currently comprised of approximately 200 library consortia from around the world. ICOLC provides an important venue for licensing consortia to share best practices and discuss issues of common interest. Formal MOUs signed with other countries and international organizations CRKN has recently entered into its first co-licensing activity at the international level with the Joint Information Systems Committee (JISC Collections) in the UK and the Centre for Research Libraries (CRL) in the US. The ground-breaking agreement will provide access to the Churchill Archive online collection at favourable rates to more than 500 academic institutions in the United Kingdom, Canada and the United States. As part of this unique international arrangement, members of the CRL, JISC Collections, CRKN and select partner consortia will be eligible for discounts on both subscription and perpetual access to the Churchill Archive in 2012. Realized research linkages and characteristics Several studies sponsored by CRKN have found that having access to quality information resources such as those provided via CRKN have enabled Canadian researchers to participate more often in collaborative research projects and engage in multi-disciplinary research. Researchers surveyed in 2004, for instance, reported that having access to electronic journals allowed them to expand their research networks. (The Impact Group, 2004) This was further supported by information gathered through a focus group of researchers convened in 2011. (The Impact Group, 2011) The study showed that having access to online scientific information was helping Canadian researchers to build research networks at home and abroad. The combination of expanded research content combined with enhanced online communications has made it is easier for researchers to identify potential research partners and for partners to find them. In the words of one focus group participant, “Les distances et/ou les frontiers ne sont plus des obstacles aux collaborations” (The Impact Group, 2011, p. 14), which translates to, ‘distance and borders are no longer obstacles to collaboration’. Once partners have been identified and teams built, it is easier for Canadian researchers to keep in touch and to actively manage research. Another focus group contributor noted, “Online 51 access has increased the potential for new collaborations and exposure to more diverse research topics; it is easier to identify and contact new collaborators” (The Impact Group, 2011, p. 14). Focus group researchers overwhelmingly agreed that having access to online scientific information has helped them to build research networks at home and abroad. Once networks are established, participants reported that they increasingly find themselves at the centre of the networks, acting as “information gatekeepers”, because in many instances they have access to a larger body of information than their colleagues at universities in many other parts of the world. CRKN resources also facilitate greater interdisciplinarity, as more publications are monitored, gray literature is included, cross-disciplinary connections are made through keyword searches and links, and literature from unrelated fields of research are accessible. A case study recently collected by CRKN illustrates how digital information can facilitate knowledge transfer across disciplines: Dr. Cristian Suteanu, Associate Professor, Geography Department and Environmental Studies Program, Saint Mary’s University is currently studying river flows. Digital online access has introduced a new way of searching for information: keyword searches. The researcher enters a keyword into a search engine and the engine finds all documents containing that key word. The power of keyword searching is that it potentially yields information from any field of research, not just the primary field of interest. Dr. Suteanu was fascinated by the outcome when he used the digital research content licensed by CRKN at Saint Mary’s library to search for information on fluid flows related to his interest in rivers. His Dr. Cristian Suteanu search yielded a reference to a paper in cardiology - a field of inquiry far removed from his. On further exploration it turned out that the cardiology paper contained a scientific approach that was directly relevant to Suteanu’s research on rivers, which he then incorporated into his own work. In effect, online searching had demonstrated a new form of serendipity in research; an enhanced ability to link diverse fields of research and provided a new context for the term “interdisciplinary”. 3.4 Research outputs Overall research influence and reach CRKN has had a significant impact on the quantity and quality of research in Canada and is an important contributor to Canada’s competitive advantage in research and innovation. A 2004 study of the impact of CNSLP revealed that researchers leveraged these resources to improve their research, which in turn made their institutions more competitive. (The Impact Group, 2004) Over 80% of researchers surveyed in this study reported that e-journal access has had a positive impact on their ability to conduct research. Respondents also indicated that having access to electronic journals has allowed them to expand their research networks, better identify new research opportunities, shorten the development cycle from conception to completion of their research, and conduct better quality research. 52 2011 focus group feedback also showed that access to digital research content is having a significant impact on both the quality and quantity of research conducted in Canada. (The Impact Group, 2011) Since CRKN is not the only source of digital research content in Canada, the findings are correlated to CRKN’s work, not causal. According to the study report, “Canada’s research community reports that it has adapted to and embraced digital research content so seamlessly that it is completely integrated in the national research infrastructure.” (The Impact Group, 2011, p. 21) A 2009 study conducted by RIN in the UK found “Per capita expenditure and use of e-journals is strongly and positively correlated with papers published, numbers of PhD awards, and research grants and contracts income” (RIN, 2009, p. 8). (Figure 20). These correlations were independent of institutional size. Figure 20: Correlation between articles downloaded with number of papers published, number of PhDs awarded and research funding in the United Kingdom. (RIN, 2009, p. 8) The relationship between access to electronic journals and research productivity can also been seen in data gathered from the Canadian context. According to a report produced by the Canadian Council of Academies, Canada’s researchers are very productive in terms of research outputs. The report states, “With less than 0.5 per cent of the world’s population, Canada produces 4.1 per cent of the world’s scientific papers and nearly 5 per cent of the world’s most frequently cited papers. In 2005–2010, Canada produced 59 per cent more papers than in 1999–2004, and was the only G7 country with an increase above the world average.” (The Expert Panel on the State of Science and Technology in Canada, 2012, pg. xii) While it is impossible to make a direct causal link between access to content and research productivity, there is a strong correlation between the launch of the CRKN in 2001 and an increase in research productivity. An analysis of the SCOPUS database7, shows that the publication output of Canadian researchers was relatively stable up until 2002, after which it began increasing dramatically (Figure 18). 7 SCOPUS is the largest abstract and citation database of research literature covering nearly 18,000 titles from more than 5,000 publishers. 53 Although not all disciplines are represented equally in this database, the data is illustrative. For each year from 1996 to 2001, the publication output of Canadian researchers remained steady at about 30,000 papers published per year. After the inception of CNSLP in 2001, publication output has grown each year by 3000-6000 per year until 2011, when the publications output reached over 70,000. Figure 18: Publication output of Canadian researchers (SCOPUS data provided by Elsevier) 2011 2010 2009 2008 2007 2006 2005 2004 2003 2002 2001 2000 1999 1998 1997 Number of publications 70000 60000 50000 40000 30000 20000 The link between access to research papers and research output has been supported by research conducted in the UK by the Research Information Network in 2009. (Tenopir & Volentine, 2011) The study, which takes an in-depth look at the value electronic journals bring to universities and research institutions, found a strong correlation between information consumption and number of publications produced. Also coinciding with the launch of CRKN is an increase in the number of citations of Canadian research papers. Canada is currently ranked sixth in the world in terms of Average Relative Citations (ARC). (The Expert Panel on the State of Science and Technology in Canada, 2012, pg. xii) A citation analysis using Web of Science8 reveals a clear spike in number of citations of authors from Canadian institutions just after the inception of the CNSLP project. As illustrated in Figure 19, from 1996 to 2001, the number of citations per year was consistently around 50,000. From 2002 onwards, the number of citations grew steadily to reach 80,000 in 2009, after which they have levelled off. 8 Includes: Science Citation Index Expanded (SCI-EXPANDED) --1899-present; Social Sciences Citation Index (SSCI) --1898present; Arts & Humanities Citation Index (A&HCI) --1975-present; Conference Proceedings Citation Index- Science (CPCI-S) -1990-present; Conference Proceedings Citation Index- Social Science & Humanities (CPCI-SSH) --1990-present) 54 Figure 19: Number of citations of Canada authors from Web of Science # of citations of Canadian authors 90000 80000 70000 # of citations from Canada 60000 50000 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 40000 CRKN has also collected anecdotal evidence that access to its licensed content has contributed to the productivity of Canada’s researchers. Dr. John Joseph McPhee is Professor and NSERC/Toyota/Maplesoft Industrial Research Chair in Mathematics-based Modelling and Design, Systems Design Engineering at the University of Waterloo. McPhee is helping to develop mathematical models of auto systems. He holds a Research Chair that is jointly supported by a mathematical software company and an auto manufacturer. Dr. McPhee’s research contributes to the development of software that is being used by car manufacturers to design automotive systems. Dr. McPhee reports that CRKN resources have contributed greatly to his research. “I can’t remember the last time I’ve been to the library ... everything is available on my desktop. That represents a huge increase Dr. John McPhee in productivity”. McPhee uses the digital online resources that CRKN has made available to the University of Waterloo library in order to stay up-to-date in his field of research and share information with his research team and students. He is accessing more journals and other materials in his field than before because it is so easy to gain access online. Research opportunities Access to published literature is absolutely critical for researchers across all disciplines. Thousands of researchers use CRKN resources on a daily basis in order to conduct literature reviews, remain up-todate with new developments in their field, and undertake new digital-based research methods. One of the most important early steps in a research project is the literature review. A literature review involves the systematic identification, location, and analysis of documents containing information related to the research problem, and usually entails exhaustive bibliographic searches of all related literature databases. CRKN resources, which represent about 56% of all licensed content in Canadian academic institutions, are absolutely critical for ensuring that Canadian researchers and graduate students can conduct comprehensive literature reviews and remain up-to-date in their fields. 55 Sacha Bailey, Research Assistant and PhD student, Centre for Research on Children and Families, School of Social Work, McGill University, provides a good example of how CRKN resources are used on a daily bases. As a graduate of McGill University’s masters program in social work, Bailey is carrying her interest in parenting children with neurodevelopmental disorders into the second year of a PhD program. Her field, social work, is inherently interdisciplinary, drawing on research from fields as diverse as nursing, paediatrics, psychology and sociology. Sacha Bailey Comprehensive access to electronic databases has been instrumental for her research. To identify relevant literature, Ms. Bailey searches numerous information sources “anything health or psycho-social related” - Medline and various social science and humanities databases. Each month, Bailey and her colleagues run searches of the top 10-15 of the most frequently cited journals to obtain up-to-the-minute research information. Bailey is hoping that by bringing research together from many different fields of health and social science, she and her colleagues will be able to inform practice and to make a difference for parents and families struggling to balance hope for the future with realism about the present. Systematic reviews have also become extremely important for researchers in many fields, especially in the applied health sciences and related fields such as social work. A systematic review begins by scouring many published sources for information for (verifiable) evidence that will help clinicians decide on the best course of treatment for a patient. CRKN resources are central for conducting exhaustive and accurate systematic reviews. The availability of digital content also enables researchers to make new discoveries using large-scale content mining techniques. An article in The Guardian from May 2012 describes the process whereby researchers “need access to tens of thousands of research papers at once, so they (can) use computers to look for unseen patterns and associations across the millions of words in the articles.” (Jha, A, online) The technique, referred to as text-mining, “is a vital 21st-century research method. It uses powerful computers to find links between drugs and side effects, or genes and diseases that are hidden within the vast scientific literature. These are discoveries that a person scouring through papers one by one may never notice.” (Jha, A, online) Several examples of text mining techniques using CRKN literature are described below. Dr. Raymond Siemens, Canada Research Chair in Humanities Computing and Distinguished Professor in the Faculty of Humanities at the University of Victoria, is also highly dependent on access to CRKN resources for his research. Dr. Siemens serves as Chair of the Dr. Raymond Siemens 56 Steering Committee for the Alliance of Digital Humanities Organisations and Vice President, Research Dissemination, of the Canadian Federation of Humanities and Social Sciences. Dr. Siemens has worked with CRKN resources for several years in the context of a number of projects and describes the partnership with CRKN as “positively transformative” for his research. For his most recent project, Dr. Siemens and his collaborators are using technologies to learn more about how scholars read and process information. CRKN has facilitated and structured full access to its licensed materials so that project researchers could assemble a large digital corpus of publications. Using devices such as ipads, ereaders and desktop computers, their research aims to better understand how researchers access and use electronic resources and design better tools and interfaces for discovery and analysis of humanities texts. Heather Piwowar, a post-doctoral researcher at UBC, is tracking how research data is being re-used by other researchers. By following the trail of thousands of datasets into the published literature, she hopes to improve the rates and methods of data re-use in the future, maximizing our governments’ investment in research. Her research will be used towards the development of tools, best practices, and reward structures for investigators who reuse data. The potential benefits of data sharing are impressive: less money spent on duplicate data collection, reduced fraud, diverse Heather Piwowar contributions, better tuned methods, training, tools, and more efficient and effective research progress. However, tracking data reuse is difficult due to inconsistency in attribution practice and ambiguity between attributions describing data submission and data reuse. To do this, Piwowar identifies datasets from selected data repositories and tracks references to those datasets in the published literature. This involves intensive searching the large literature databases made available through CRKN, using key words, dataset accession number, unique identifier, and authors’ names. Geoffrey Rockwell, Professor of Philosophy and Humanities Computing, University of Alberta, discusses the value of the DCI Project for his research. “Since CRKN started rolling out the collections under the DCI project, I have had access to publications that have greatly advanced my research with others. These new collections have allowed me to track early perceptions of the impact that computing had on society dating back to the 1950’s. I and graduate students can compare the public perceptions in the media to the academic discourse. And, with the full-text documents, I am able to use my text analysis tools for concordancing. With research Dr. Geoffrey Rockwell partners at the University of Victoria and Université de Montréal, I am looking at applications of computing in the humanities. CRKN enables us all to access the same materials as we coordinate our research into presentations and papers. We presented on this subject in a paper delivered at a Society for Digital Humanities session at the [CFHSS] 57 Congress earlier this year—outlining our preliminary results. I expect this work will lead to a better sense of the development of research uses of computing and text analysis in the humanities.” (CRKN, 2009) 3.5 Influence of CFI CFI’s investments in CRKN have been instrumental in supporting improved research outcomes in Canada. Large increases in the productivity of Canadian researchers can be seen around the same time as CFI made its initial investment in CRKN. Since 2001, researchers in Canada are more productive and more highly cited. “With less than 0.5 per cent of the world’s population, Canada produces 4.1 per cent of the world’s scientific papers and nearly 5 per cent of the world’s most frequently cited papers.” (The Expert Panel on the State of Science and Technology in Canada, 2012, pg. xii) They are now so well connected to current sources of information that they have become “magnets” for international collaboration and expanded access to online information has raised the profile of their research. While the most well-endowed universities in Canada may have the capacity to license the amount of resources currently available through CRKN (although they would likely pay higher prices), CFI’s investment in CRKN has been absolutely essential for smaller universities in Canada. Small and medium institutions make up 64% of CRKN members. This represents over 300,000 of CRKN’s researcher and student users who would not have access to even a small portion of the content resources they do now through CRKN. As an example, as a researcher at a small university, Dr. Cristian Suteanu was often frustrated because key books and articles he needed for his research were simply not available through Saint Mary’s library. The interlibrary loans system worked well, but it was time consuming and frustratingly slow. Today, the resources available to him are on par with his colleagues at larger universities in Canada or abroad. An even more striking benefit is that online access allows Dr. Suteanu to be a competitive researcher in his field, even though he has no major research grants. With smaller sources of funding from Canada and abroad, he has developed an approach that includes information-mining techniques capable of providing new perspectives and extracting useful information from data previously assessed with classic methods. The Internet gives him access to the data he needs for his work, free. Digital library access provides him with background concepts and methods he uses to process and synthesize the data. So, even a researcher with moderate sources of funding at a small institution can contribute to scientific progress today. 58 4.0 Extrinsic Benefits - Impact on Local, Regional, and National Innovation 4.1 Mechanisms and strategies Impacts The majority of CRKN’s outreach activities are targeted to members and other stakeholders, not the general public. However, with other organizations, CRKN has occasionally participated in national advocacy efforts and consultations. For example, in 2010 CRKN contributed to a brief prepared in conjunction with the Canadian Digital Media Network, the Canadian University Council of CIOs, CANARIE Inc., and Compute Canada (2010) on the federal governments Digital Economy Strategy consultation. In 2012, the CRKN Executive Director attended the Digital Infrastructure Summit that was hosted by CUCCIO (Saskatoon, June 13-14, 2012) on the theme "Putting Researchers First" and in September 2011, CRKN Executive Director attended the 2011 Canadian Research Data Summit which both provided opportunity for CRKN to contribute to these consultative meetings. These activities ensure that CRKN is recognized as a crucial facet of Canada’s digital research infrastructure within the scholarly community. They also raise awareness with policy makers of the role of CRKN and its contribution to research excellence and innovation. 4.2 Intellectual property development and protection IP outputs CRKN resources are used to support patent applications and other technology transfer activities at universities across Canada. A crucial step in any patent application is the prior art search. A prior art search is done in order to ascertain whether an invention is new or not and involves reviewing all the information resources that might relate to an invention. In many cases, the researchers will undertake the prior art search. However, a number of universities have begun to offer these types of services for researchers. The University of Manitoba Technology Transfer Office, for example, undertakes an Intellectual Property Assessment for research outcomes that may have intellectual property, market and commercial potential. The first step is a Priory Art Search and Evaluation, during which a “search of patent and literature databases is undertaken to locate any potential patents and publications in the field that may limit patentability or freedom to operate”. (IP and Marketing Assessments, 2012) 59 Whether undertaken by the researcher or the university, CRKN resources provide an important resource for demonstrating the uniqueness of university inventions. In addition, the original CRKN model license, developed in 2002, was one of the first of its kind and has been used and adapted by other licensing organizations in Canada such as the regional consortia. 4.3 Benefits from knowledge translation and transfer Public access to CRKN content: Many CRKN licensed resources are available to walk-in users at member libraries, in addition to researchers and students. Therefore, CRKN resources are contributing to a better-informed society and a more educated public. Several CRKN member libraries have developed services to support the information needs of nonacademic users. The Irving K. Baker Learning Center at UBC, for example, offers a broad range of programs and services that support teaching and learning, as well as lifelong learning and community engagement, through the development of partnerships across UBC and the wider community. The University of Waterloo Library has also established a service to meet the information needs of small businesses, industry, and other individuals outside the academic community. By utilizing the Library's collections, which are especially strong in the areas of mathematics, science, engineering, economics, and psychology, the Industrial and Business Information Service (IBIS) is provides an accurate, reliable, and timely service to users at a reasonable cost. Two digitization projects are also supported through CRKN using the DCI project funds. One project involves the digitization of microfilmed materials including out-of-copyright newspapers, magazines, and other Canadian material of historical interest as originally gathered and produced by the Canadian Institute for Historical Microreproductions (CIHM). The other project is the digitization of out-ofcopyright books at the University of Toronto. Both of these projects are making the digitized content openly available over the internet to the public. This digitized content, available through Canadiana.org and the Internet Archive websites is highly used and valued by the Canadian public. CRKN supports researchers innovations: In terms of researchers innovations, CRKN supports the work of 99% of academic researchers in Canada, therefore contributing indirectly to the products of their research. For instance, Jeff Dahn, Canada Research Chair at Dalhousie University, uses CRKN publications in support of his research. Dr. Dahn, NSERC/3M Canada Ltd. Industrial Research Chair in Materials for Advanced Batteries and Tier I, is recognized worldwide as Dr. Jeff Dahn one of the pioneering developers of the lithium-ion battery that is now used worldwide in laptop computers, cell-phones and electrified vehicles. He has worked on new electrode materials, improved safety and most recently on improved lifetime for Li- ion batteries. For his work, Dr. Dahn has been awarded both major awards by the Battery Divison of the Electrochemical Society: the “Research Award” in 1996 and the “Technology Award” in 2011. He is a co- 60 author of over 485 refereed journal papers and a co-inventor of 58 inventions with patents issued or filed. As reported in the Canadian Manufacturers and Exporters Magazine 20/20, the global market for lithium-ion batteries is forecast to grow from $11 billion in 2010 to $43 billion by 2020. 3M has commercialized several patented materials based on technology developed by Dr. Dahn and has already begun manufacturing products using these materials on a global scale. According to Mike Irwin, Head of Research, 3M Canada, “Jeff is the most important academic collaborator that 3M has worldwide”. (Canadian Manufacturers and Exporters Magazine 20/20, March/April 2011, p. 35) A review of the reference in Dr. Dahn’s recent articles finds that the majority of the references are to publications that are made available through CRKN licensed content. 4.4 Influence of the CFI CFI’s investments in CRKN have contributed to the development of robust digital economy infrastructure in Canada. From CRKN’s brief submitted to Industry Canada for the Digital Economy Strategy Consultation, “The foundation for a prosperous, sustainable and growing digital economy is a broad base of highly qualified people who leverage an integrated digital environment to engage in activities including basic and exploratory research, development and commercialization activities, and ongoing collaboration across the innovation system.” (Canadian Digital Media Network, et.al, 2010, p.3) CRKN’s content underpins discoveries in healthcare, technological developments, economic growth, social policies, and many other areas. From pioneering stem cell research at the University of Calgary, to voice recognition software developed at the Université de Sherbrooke that’s being adopted worldwide, to genocide intervention policies at Concordia University, Canadian scholars at 75 universities rely on the digital content being provided through CRKN licenses to support their research and innovation activities. While it is difficult to demonstrate the direct impacts of electronic resources on innovations and other downstream outcomes, it is clear that CFI’s investments in CRKN have had an indirect impact. Digital content is a critical component of a digital economy. CFI’s investments in CRKN, therefore, have contributed to the development of a robust national digital infrastructure, and towards fostering a vigorous digital economy in Canada. 61 5.0 Challenges Sustainability issues Maintaining and strengthening a consortium with 75 members, 2 languages, several time zones, that is the sheer size and scope of Canada, and with unique jurisdictional issues, is a major challenge. CRKN must be responsive to diverse priorities and sometimes divergent needs of its members – while managing its core function of negotiating content licenses. As with many research platform initiatives, sustainability has been a challenge for CRKN, both in terms of budgeting and staffing. Members would like to increase the value of their membership in CRKN by acquiring more content through the consortia. However, in the current economic climate, there is very little backing for an increase in membership fees, which is the main source of funding for CRKN’s operations. In the past, CRKN has been able to fund a significant portion of its operating budget using the interest income realized on the funds provided by members for content, but interest rates have declined dramatically and CRKN has been running a deficit for the last 2 years. CRKN has developed a plan to address the deficit, which includes a modest increase of membership fees (2.5%), a reduction in costs (largely achieved through a move to down-sized premises), and draw-down of reserves. However, this approach is not sustainable over the long term which will have an impact on the effectiveness of the organization. Since 2001, CRKN has more than quadrupled the number of licenses it negotiates and manages, but there has not been a corresponding increase in resources. This has led to significant staff turnover due to the high volume and complexity of work. In 2011, 4 of 7 staff members left (representing 20.5 years of experience), and in 2012 there have been another 2 departures (representing 2 years of experience). When the Executive Director leaves later this year, the staff member with the longest organizational memory will have been with the organization for just 3 years. Recruiting new staff members is a very resource intensive activity that further stretches the CRKN office, which is already working at full capacity. In addition, as a result of budget pressures, the Advisory Board was disbanded and the Board of Directors downsized. The loss of the counsel from a broader section of the research community has brought into sharp focus the need to formalize relationships with select organizations to share information and remain up-to-date on current trends and changing information needs. 62 Evolving publisher business models As with other types of media, academic publishers are adapting new business models in the digital environment, requiring that CRKN remain responsive and attentive to the impact of these models on price and access conditions. Some publishers, such as the American Chemical Society, have adopted a pricing approach based on usage that goes clearly against the licensing principles of CRKN, which seeks to encourage greater usage of content rather than limit use. Alternatively, open access policies at funding agencies and institutions, which require that funded researchers make their articles available free of charge to readers, are influencing the business models of academic publishers, many of whom are incrementally moving from subscription-based models to membership models or article process fee approaches. The challenge for CRKN is to keep abreast with these developments and investigate the feasibility of supporting new models at the consortial level. In terms of open access, it is not readily apparent how licensing consortia can support open access models. In 2011, CRKN launched a joint Open Access Working Group (OAWG) with the Canadian Association of Research Libraries (CARL) to further explore its role as open access becomes more prevalent in scholarly publishing. The Working Group presented the preliminary results of its work at the CRKN AGM in October 2012. The report included a number of recommendations for CRKN in terms of how to better support open access in the future. The preliminary report includes (among other things) recommendations for CRKN to:  Work (with other organizations) to develop a national OA sustainability model for Canada’s humanities and social sciences.  Incorporate OA‐related terms and conditions into its licenses.  Negotiation of more favourable OA article processing charges with commercial vendors. Once the report has been finalized, CRKN will consider how best to incorporate the recommendations into the future planning and operations of the organization. Impact measures Accurate usage and impact measures for digital resources would greatly assist CRKN in making better licensing decisions and in demonstrating its impact with members and stakeholders. However, measuring the impact of electronic resources is a resource intensive and problematic issue for CRKN (and other library consortia around the world). Currently CRKN receives usage data from publishers for each institution, but the data are not always COUNTER compliant9, and even if they are COUNTER compliant, they cannot be compared across publishers without great efforts. In addition, this type of usage data (number of downloads and number of queries) is by no means a comprehensive metric for understanding usage patterns even when they are standardized. Improving usage metrics by CRKN has been a priority for members. In the 2012 member survey, 89.4% of respondents indicated that content usage metrics would be valuable to them (Partners Marketing 9 COUNTER stands for Counting Online Usage of Networked Resources. COUNTER compliance reports COUNTER-compliant reports (often just called "COUNTER reports") are usage reports that are formatted exactly as defined in the COUNTER Code of Practice and use defined ways to count usage. For more information, see: http://www.niso.org/workrooms/sushi/faq/general 63 Inc., 2012). This was also identified as an issue of importance for members in the previous 2009 survey (Partners Marketing Inc., 2009). One possible way of gathering more accurate and insightful information about how CRKN resources are being used is to use the MINES for Libraries10 survey methodology. MINES is a survey method that gathers qualitative information from e-content users such as who are the users and for what purpose they are using the resource. MINES data offers a much more detailed portrait of information users than could ever be collected through standard usage statistics such as number of downloads and queries. It can provide a critical link between electronic resources and the value derived by users. The survey also can be extended to assess the relationships between electronic resources with other desired outcomes, such as research quality and productivity. Although implementing this type of survey is an intensive exercise, it is something that CRKN and its members may consider implementing if resources become available in the future. 10 Measuring the Impact of Networked Electronic Services (MINES) is an online, transaction-based survey that collects data on the purpose of use of electronic resources and on the demographics of users. As libraries implement access to electronic resources through portals, collaborations, and consortial arrangements, the MINES for Libraries® protocol offers a convenient way to collect information from users in an environment where they no longer need to physically enter the library in order to access resources. MINES for Libraries® adapts a long-established methodology to account for the use of information resources in the digital environment. The survey is based on methods developed to determine the indirect costs of conducting grantfunded R&D activities, and was adopted as part of ARL’s New Measures program in May 2003. 64 References Association of University and Colleges of Canada. (2011). Trends in Higher Education. Volume 1: Enrolment. Available at: http://www.aucc.ca/media-room/publications/trends-in-higher-educationvolume-1-enrolment Canadian Digital Media Network, Canadian Research Knowledge Network, Canadian University Council of CIOs, CANARIE Inc., Compute Canada. (2010). Canada's Digital Environment for Research, Innovation and Education. Available at: http://www.ic.gc.ca/eic/site/028.nsf/eng/00437.html Canadian Research Knowledge Network. (2006). CRKN Strategic Plan 2007-2009. Available at: http://www.crkn.ca/about/strategic-planning/strategic-plan-2007-2009 Canadian Research Knowledge Network. (2009). CRKN Strategic Plan 2010-2012. Available at: http://www.crkn.ca/about/strategic-planning/strategic-plan-2010-2012 Canadian Research Knowledge Network. (2008-2009). Annual Report Canadian Research Knowledge Network. (2009-2010). Annual Report Canadian Research Knowledge Network. (2010-2011). Annual Report Canadian Research Knowledge Network. (2011-2012). Annual Report Canadian Research Knowledge Network. VITaL Narrative and Dashboards 2010 - 2012 Davidson, C & Kyrillidou, M. (August 2012). The Value of Electronic Resources: Measuring the Impact of Networked Electronic Services (MINES for Libraries®) at the Ontario Council of University Libraries. Research Library Issues, 271. p.41-47. Available at: www.arl.org/bm~doc/rli271-mines.pdf The Expert Panel on the State of Science and Technology in Canada. (2012) The State of Science and Technology in Canada, 2012. Council of Canadian Academies. Available at: http://www.scienceadvice.ca/uploads/eng/assessments%20and%20publications%20and%20news%20re leases/sandt_ii/stateofst2012_fullreporten.pdf The Impact Group. (April 2004). Impact of the Canadian National Site Licensing Project. Available at: http://www.crkn.ca/sites/default/files/about/Impact_Final_Report_Apr04.pdf The Impact Group. (April 2011). CRKN Outputs and Impacts: Findings of Four Focus Groups With the Research Community. Available at: http://www.crkn.ca/sites/default/files/file/CRKN%20Focus%20Group%20Report%20%20Impacts%20and%20Outcomes%202011.pdf Industry Canada. (2012). Building Digital Skills for Tomorrow. The Digital Economy Strategy. Available at: http://www.ic.gc.ca/eic/site/028.nsf/eng/00041.html IP and Marketing Assessments. (2012). University of Manitoba. Available at: http://www.umanitoba.ca/research/tto/researchers/IP_marketing_assessments.html 65 Jha, A. (23 May 2012). “Text mining: what do publishers have against this hi-tech research tool?” The Guardian. Available at: http://www.guardian.co.uk/science/2012/may/23/text-mining-research-toolforbidden McDevitt, N. (March 24, 2011). With Colleen Cook, McGill’s Trenholme Dean of Libraries. McGill Reporter. Available at: http://publications.mcgill.ca/reporter/2011/03/with-colleen-cookmcgill%E2%80%99s-trenholme-dean-of-libraries/ “NSERC boosting 3M Canada’s competitive edge in growing market for lithium-ion batteries”. (March/April, 2011). Canadian Manufacturers and Exporters Magazine 20/20, 6 (1). p. 34-35. Available at: www.2020magazine.ca Partners Marketing Inc. (March 2010). CRKN Communications Outreach: Member e-Survey. Final Report. Available at: http://www.crkn.ca/about/strategic-planning/communications-outreach-2012 Partners Marketing Inc. (August 2012). CRKN Communications Outreach: Member Feedback. Detailed Report. Available at: http://www.crkn.ca/sites/default/files/crkn_member_feedback_report__detailed_report_final_0.pdf Research Information Network. (April 2009). E-journals: their use, value and impact. Available at: http://www.rin.ac.uk/our-work/communicating-and-disseminating-research/e-journals-their-use-valueand-impact Schonfeld, R & Housewright, R. (April 7, 2010). Faculty Survey 2009: Key Strategic Insights for Libraries, Publishers, and Societies. Ithaka S+R. Available at: www.sr.ithaka.org/research-publications/facultysurvey-2009 Statistics Canada. (2010) Table 477-0013 - University enrolments, by registration status, program level, Classification of Instructional Programs, Primary Grouping (CIP_PG) and sex, annual (number), CANSIM (database). Available at: http://www5.statcan.gc.ca/cansim/a05?lang=eng&id=4770013 Tenopir, C & Volentine, R. (February 2011). UK Scholarly Reading and the Value of Library Resources: Summary Results of the Study Conducted Spring 2011. Available at: http://www.jisccollections.ac.uk/Documents/Reports/UK%20Scholarly%20Reading%20and%20the%20Value%20of%20L ibrary%20Resources%20Final%20Report.pdf 66 Appendix 1: Flowchart of Licensing Renewal Process 67 Appendix 2: List of CRKN Content Providers                             Adam Matthew Digital ARTstor Alexander Street Press / Gibson Library Connections American Chemical Society American Mathematical Society CAIRN Cambridge University Press Canadiana.org The Center for Research Libraries Economist Intelligence Unit Elsevier Érudit Gale Cengage Learning Gibson Library Connections Ingram Digital Institute of Physics Publishing InteLex Past Masters JSTOR NRC Research Press (Canadian Science Publishing) Oxford University Press ProQuest Royal Society of Chemistry SAGE Springer SWETS Taylor & Francis Group Thomson Reuters Wiley - Blackwell 68 Appendix 3: CRKN Members Acadia University Algoma University Athabasca University Bishop’s University Brandon University Brock University Cape Breton University Carleton University Concordia University Concordia University College of Alberta Dalhousie University École Polytechnique de Montréal Grant MacEwan University HEC Montréal Kwantlen Polytechnic University Lakehead University Laurentian University McGill University McMaster University Memorial University of Newfoundland Mount Allison University Mount Royal University Mount Saint Vincent University Nipissing University NSCAD University OCAD University Queen’s University Royal Military College of Canada Royal Roads University Ryerson University Saint Mary’s University Simon Fraser University St. Francis Xavier University The King’s University College of Alberta Thompson Rivers University Trent University Trinity Western University Université de Moncton Université de Montréal Université de Sherbrooke Université du Québec (and affiliates listed below)  École de technologie supérieure  École nationale d’administration publique  Institut national de la recherche scientifique  Télé-université  Université du Québec à Chicoutimi  Université du Québec à Montréal  Université du Québec à Rimouski  Université du Québec à Trois-Rivières  Université du Québec en Abitibi-Témiscamingue  Université du Québec en Outaouais Université Laval Université Sainte-Anne University of the Fraser Valley University of Alberta University of British Columbia University of Calgary University of Guelph University of Lethbridge University of Manitoba University of New Brunswick University of Northern British Columbia University of Ontario Institute of Technology University of Ottawa University of Prince Edward Island University of Regina University of Saskatchewan University of Toronto University of Victoria University of Waterloo University of Windsor University of Winnipeg Vancouver Island University Western University Wilfrid Laurier University York University 69 Appendix 4: Flowchart of Governance Review 70 CFI Platform Outcome Measurement Study (POMS) EXPERT PANEL REPORT CANADIAN RESEARCH KNOWLEDGE NETWORK December 4, 2012 Expert Panel Report — Platform Outcome Measurement Study─CRKN, December 2012 1 Contents Lexicon ............................................................................................................................................ 2 Platform Outcome Measurement Study (POMS) ........................................................................... 3 Key findings ..................................................................................................................................... 5 1. Operation of CRKN .................................................................................................................. 7 1.1 1.2 Platform planning process ............................................................................................... 8 1.3 Capital investment value .................................................................................................. 9 1.4 Operation and maintenance (O&M) investments ......................................................... 10 1.5 Platform capabilities and sustainability ......................................................................... 11 1.6 Leadership and competitiveness.................................................................................... 14 1.7 2. Governance, management and advisory structure ......................................................... 7 Linkages .......................................................................................................................... 14 Impacts of CRKN .................................................................................................................... 16 2.1 Foregone costs ............................................................................................................... 16 2.2 Expansion of access and usage ...................................................................................... 17 2.3 Research enabled ........................................................................................................... 19 2.4 Contributions to the training of highly qualified personnel (HQP) ................................ 20 2.5 Cultural, organizational and structural change enabled at stakeholder institutions .... 21 2.6 Extrinsic benefits: Impact on local, regional and national innovation ........................... 23 3. Influence of the CFI and its funding partners ....................................................................... 25 4. Challenges ............................................................................................................................. 26 5. Conclusion ............................................................................................................................. 28 Summary of ratings ....................................................................................................................... 29 Expert Panel Report — Platform Outcome Measurement Study─CRKN, December 2012 1 Lexicon Acronyms used in report APCs article processing charges AUCC Association of Universities and Colleges of Canada CARL Canadian Association of Research Libraries CFI Canada Foundation for Innovation CNSLP Canadian National Site Licensing Project CRKN Canadian Research Knowledge Network EP Expert Panel HQP highly qualified personnel LIM License Information Module O&M operation and maintenance POMS Platform Outcome Measurement Study SMEs small and medium enterprises VITaL Value, Influence, Trends and Leadership Expert Panel Report — Platform Outcome Measurement Study─CRKN, December 2012 2 Platform Outcome Measurement Study (POMS) The CFI is grateful for the support and participation of the Canadian Research Knowledge Network (CRKN) and its members in the Platform Outcome Measurement Study (POMS) and also wishes to thank the Expert Panel (EP) members for their time, expertise and many contributions to this report. In December 2012, the CFI assembled a panel of experts to assess the activities and achievements of CRKN and evaluate the degree to which the investment of the CFI and its partners has had a transformative impact on Canada’s research landscape and is contributing to the CFI’s meeting its objectives. The assessment was based on a new tool in the CFI’s suite of evaluation activities — POMS — developed specifically for large-scale and broad-based multidisciplinary projects that support the Canadian research community. An in-depth report prepared by CRKN captured, with numbers and narrative, the outcomes and impacts of the Network. The confidential self-report, which also highlighted key organizational dimensions (e.g. governance, management, human resources), was provided to the EP. A visit by the group of experts allowed the members to gain additional insights about the activities and outcomes of CRKN and how they relate. On the basis of the self-report and visit discussions, the EP assessed indicators of progress and outcomes (generally using a scale: High, Medium or Low), provided rationale for its decisions and highlighted key contributions and impacts of CRKN. This report summarizes the assessment, findings and conclusions of the EP. Members of the EP Dr. Mark Bisby (Chair) Consultant Ottawa, Ontario, Canada Pam Bjornson Director General, Knowledge Management National Research Council - Canada Institute for Scientific and Technical Information Ottawa, Ontario, Canada Lorraine Estelle Chief Executive Officer JISC Collections London, United Kingdom David Seaman Associate Librarian for Information Management Dartmouth College Hanover, New Hampshire, United States Johannes (Jan) Velterop Chief Executive Officer Academic Concept Knowledge Ltd. Epsom, Surrey, United Kingdom Expert Panel Report — Platform Outcome Measurement Study─CRKN, December 2012 3 The CFI would like to acknowledge the participation of the following representatives of CRKN and its members at the EP visit: Dr. Ronald Bond Interim Chair, CRKN Board of Directors Clare Appavoo Executive Director (incoming) CRKN Sylvie Belzile Director General of Library and Archives Services Université de Sherbrooke Member, VITaL Task Group Deb deBruijn Executive Director (outgoing) CRKN Joyce Garnett University Librarian Western University Chair, VITaL Task Group Dr. Katherine Schultz Former Vice-President Research University of Prince Edward Island Member, VITaL Task Group Kathleen Shearer Consultant - Scholarly Communication, Research Data Management, Digital Repositories Dr. Raymond Siemens Distinguished Professor and Canada Research Chair in Humanities Computing University of Victoria Member, CRKN Board of Directors and VITaL Task Group Dyna Vink Director of Communications CRKN Leslie Weir University Librarian University of Ottawa Member, CRKN Board of Directors Martha Whitehead University Librarian Queen’s University Expert Panel Report — Platform Outcome Measurement Study─CRKN, December 2012 4 Key findings CRKN is among the leading information-enabling organizations worldwide and is recognized as a “game changer” for the Canadian research community.  The number of members and the number of licence agreements have increased through the life of CRKN.  CRKN has enabled universities of all sizes and in all regions to have access to the same licensed content. Similarly, CRKN’s licensing activities support research and training in all disciplines.  CRKN’s activities underpin high-quality university research and teaching, but the Network is largely invisible to its end-users.  CRKN’s model licence agreement has attracted interest from around the world. The investment in CRKN by the CFI and its provincial government partners was essential, timely and catalytic and has been returned many times over.  The support of the CFI and its funding partners stimulated the universities and the existing regional consortia to co-operate in building a national platform with the widest scope and reach.  Contributions made by the CFI and its partners were leveraged several times over with cumulative investments in licensed content of $767 million over the life of CRKN.  Through CRKN’s licence agreements with publishers, savings to members have totalled $1.43 billion since 2001. CRKN has well-developed and efficient operations to address its current mandate; however, recent changes were viewed as potentially detrimental to its future.  CRKN has evolved a mature governance structure, with comprehensive bylaws and operating policies. The planning process and interactions with members are effective, and there have been many management accomplishments. The quality of personnel is high.  Recent changes in the composition of the Board and the abolition of an international Advisory Board as a cost-saving measure will deprive CRKN of a valued and broad-based source of expertise and advice if no alternative sources of engagement are developed. CRKN may not have the resilience with its current level of resources to deal with the ongoing transformation in scholarly communication and the ways in which digital content is used by the research community.  CRKN operates on a budget barely sufficient to fulfill its mandate, with resulting symptoms of organizational stress and risk adversity.  Resource constraints threaten the capacity of CRKN to maintain a leading role as a contemporary research knowledge network. CRKN must 1) embrace innovation and risk to take advantage of emerging opportunities; 2) better convey its value proposition to university stakeholders; and 3) explore potential new sources of revenue. Expert Panel Report — Platform Outcome Measurement Study─CRKN, December 2012 5 Overview of the platform “Collaborating for Value and Impact” — CRKN motto The Canadian Research Knowledge Network (CRKN) is a partnership of Canadian universities dedicated to expanding access to digital content for the academic research enterprise in Canada. It negotiates agreements with publishers to provide the best financial, access and usage terms for digital content made available directly from publishers’ or vendors’ sites. It concentrates its efforts on licensing content that is of broad interest and high need for researchers at member universities. CRKN’s membership has increased from 64 institutions in 2000 to 75 in 2012 and includes the majority of the universities that belong to the Association of Universities and Colleges of Canada (AUCC). CRKN-licensed content represents 56 percent of total spending on digital content by the libraries of member universities. Through these libraries, digital content is available to 99 percent of university researchers and students in Canada. CRKN began as the Canadian National Site Licensing Project (CNSLP) in January 2000, after an award of $20 million from the CFI, which was matched by $20 million from provincial governments and the Atlantic Canada Opportunities Agency plus $10 million from 64 universities, for a total of $50 million over three years. The University of Ottawa served as the project’s host, and a Steering Committee oversaw all aspects of project development. Initially, content was primarily full-text e-journals and citation databases in science, engineering, health and environmental disciplines, as these were where the needs and costs for universities were most acute. On April 1, 2004, CNSLP was renamed CRKN and incorporated as a not-for-profit organization with an independent Board. In 2005, CRKN began a three-phase project that added content in social sciences and humanities and led to a further proposal to the CFI. In February 2007, the CFI awarded $19.1 million from its National Platforms Fund. With matching funds totalling $28.6 million from 67 universities and provincial governments, the Digital Content Infrastructure for the Human and Social Sciences (DCI) Project was launched, and by June 2008, 14 major research collections in social sciences and humanities disciplines had been secured. In 2009, CRKN dedicated the remaining DCI funds to digitization of unique international and Canadian historical materials for open access. CRKN currently operates with a staff of eight. To date, CRKN has negotiated and continues to administer 52 licences with publishers and content vendors. In 2012, total content expenditures amounted to $89 million, with operating costs of just 2 percent of its total budget. While the majority of the licensed content is journals and databases, it increasingly includes e-books, newspapers, videos, images, music and primary source material. In dollar amounts, 65 percent of the content is in science, technology and medicine and 35 percent is in the social sciences and humanities, with 97 percent English content and 3 percent French content. CRKN manages licences with Elsevier, Sage, Springer, Taylor & Francis, Wiley-Blackwell and numerous other content providers. Expert Panel Report — Platform Outcome Measurement Study─CRKN, December 2012 6 1. Operation of CRKN 1.1 Governance, management and advisory structure CRKN has evolved a well-developed and mature governance structure, with comprehensive bylaws and operating policies, a clear organizational chart and standing committees or task forces that address its responsibilities and functions; for example: a Negotiations Resource Team that develops procurement strategies; an Open Access Working Group, jointly with the Canadian Association of Research Libraries (CARL), to explore sustainable open access models in a cost-effective scholarly content ecosystem; and the recent Value, Influence, Trends and Leadership (VITaL) Task Group, working to develop indicators of CRKN performance and impact. In 2010–2011, CRKN undertook a governance review, and following consultation with key academic stakeholders, the Board was restructured in February 2012 to increase the representation of member libraries and reduce the Board size from 16 to 10 members. Among the reasons for this change were cost savings, new federal regulations and good governance practices. While this new composition is more representative of its core client group (university librarians), it is less representative of its broad stakeholder community. In addition, CRKN abandoned its international Advisory Board in March 2011 for cost reduction reasons, depriving it of a valued and broad-based source of advice. The Board is well structured for operational purposes, but whether it is optimum to provide strategic advice is less clear. Its membership is relatively homogeneous and academic, with seven librarians and three university administrators and/or researchers. There are no external stakeholders (e.g. individuals from the financial world, industry and the public sector or international experts) to provide a global context and an expanded vision for its operations. CRKN representatives acknowledged the need to develop other forms of engagement with their wider community. This will be a focus of an upcoming review of CRKN’s committee structure and composition. Stakeholder engagement was described as a “work-in-progress.” The Expert Panel (EP) recommended expanding the scope of the current community by seeking greater input from colleges, private and public sectors and international bodies. Management and staffing of CRKN are lean and efficient, with priority appropriately on sustaining and supporting the membership. Notably, an important management transition occurred immediately prior to the EP review, with the retirement of the founding executive director and the appointment of a successor. Both participated in the EP review. There have been many management accomplishments, such as developing the “made in Canada” licence agreement, with superior terms of access and usage for the academic community; streamlining its licence renewal processes; automating routine office processes; and developing the License Information Module (LIM), an open-source information management system that simplifies access to licence information for all CRKN agreements. The LIM has saved CRKN member libraries time and resources in the management of digital collections and has eliminated duplication of effort among CRKN members. Expert Panel Report — Platform Outcome Measurement Study─CRKN, December 2012 7 Interactions between the CRKN secretariat and its members, focusing on the licence renewal process, were good, with CRKN undertaking member surveys and holding regular conference calls with members to identify problems with vendors and discuss ideas for enhancing CRKN’s value to members. For example, the CRKN Report to the EP showed how it had been able to raise member complaints with a specific content provider and, through the vendor’s cooperation, achieve improved access to and functionality of the licensed content. 1.2 Platform planning process EP rating of the effectiveness of planning and performance monitoring in platform planning High The Expert Panel (EP) recognized that the planning process at CRKN was effective. There had been several iterations of a three-year strategic planning cycle, with the most recent cycle being 2010–2012. The planning cycle for the 2013–2015 Strategic Plan is approaching its conclusion. The planning process is intended to ensure that CRKN’s activities are based on the needs of its members and also take into account major external trends. It is distinguished by a preliminary and exhaustive communications outreach process, including surveys, focus groups and meetings with regional library consortia, to capture members’ and stakeholders’ views about current services and future needs. Notably, CRKN won an international business award for its 2010–2012 Strategic Planning process. Performance measurement was less advanced than the planning function, and there were major gaps in the historical records of usage. CRKN representatives noted that with their limited resources, they have had to collect statistics on an “as needed” basis; for example, before an evaluation, rather than on a continuous basis. However, the formation of the VITaL Task Group has recently helped CRKN improve its monitoring, and since 2010, CRKN has systematically collected information on membership leverage, scholarly content offerings and community engagement, according to a framework developed by the VITaL Task Group. The EP expects that there will be continued improvements in measuring usage of the electronic literature enabled by CRKN, noting that improved statistics would be very valuable to CRKN members in making rational decisions about their journal subscriptions. EP rating of the impact of the platform’s planning process on the evolution of the platform and its user community since the base year High The impact of the planning process was demonstrated by several CRKN successes, particularly the continued increase in membership. Membership of university research institutions is essentially complete, and no members have ever left the Network. In addition, a second application for CFI funding in 2006, resulting from CRKN planning, was successful and has resulted in a quantum increase in access to serials in the social sciences and humanities (see arrow, Figure 2, page 12). Expert Panel Report — Platform Outcome Measurement Study─CRKN, December 2012 8 EP rating of the extent and suitability of stakeholder involvement with the platform Medium With respect to stakeholder involvement, CRKN representatives pointed out that beyond the periodic consultations around strategic planning, the five volunteer task groups are an important vehicle for engaging members and stakeholders in CRKN’s planning activities on a continuous basis. These task groups are commissioned to assist the Board as specific issues arise, some in collaboration with appropriate partner organizations. There is evidence that the task groups reached closure on the issues they explored and achieved their mandates. For example, the Perpetual Access Task Group recently delivered its report to the Board, which is considering the feasibility of implementing its recommendations. There are additional opportunities for interactions with members and stakeholders, including conference calls with members to provide information and answer questions regarding new or renewal licences, ad hoc meetings with regional consortia to identify which licences are in the national interest and CRKN’s Annual General Meeting, which is highly valued for collaboration, educational and networking purposes by the 100 to 200 participants. The “Medium” rating for stakeholder involvement reflects the EP’s concern about the potential for reduced stakeholder involvement following the recent governance changes (Section 1.1), a risk acknowledged by CRKN representatives as requiring attention. The challenge for CRKN now is to seek out additional non-university members and their associated financial support and thereby achieve full coverage of institutions and users of research publications. The EP perceived that there was a lot of good planning going on but less innovation and risktaking: Sometimes risk-taking is needed to accomplish innovation. There are plenty of emerging opportunities for pan-Canadian collaboration in the management of research knowledge, such as collection analysis, collective negotiation of open access fees (see below) and centralized print repositories. 1.3 Capital investment value EP rating of the adequacy of the platform enhancements since the base year in comparison with the initial capital investment (to keep platform offerings up to date) High Interpreting capital investment as the purchase of licences, the leverage of the CFI and partner contributions has been enormous. The funders’ early investments (in 2001–2002) were significantly leveraged over the past decade. During years without ongoing CFI funding, CRKN’s operations and purchases of licences were entirely supported by member contributions (see Figure 1). Even in those years of full operations, when the CFI and partner funds were available, that funding was a minority of the total budget, and additional support by CRKN’s members greatly enhanced the content that CRKN could purchase. Collectively, the funding provided by the CFI and partners, which amounted to $97.7 million across two projects, was leveraged Expert Panel Report — Platform Outcome Measurement Study─CRKN, December 2012 9 nearly eight times, with total cumulative investments of $767 million (adjusted to 2012 dollars) since 2001. Figure 1 CFI Seed Money Leverages Sustained Investments Source: CRKN Report to the EP 1.4 Operation and maintenance (O&M) investments As previously mentioned, CRKN is a shoestring operation, with a staff of eight and an operating budget that was $1.5 million in 2012, representing less than 2 percent of its budget. The major source of the O&M budget is the membership (operations) fees, which must be distinguished from the contributions that institutions make toward the cost of content licences and which are a “flow through” to the publishers (see Table 1). Table 1 CRKN 2012 Budget 2012 Income Licence fees Operations fees Interest and other income Expenditures $93,857,000 $916,000 $197,000 Licence purchases Operating costs Amortization of capital assets Deficit $93,857,000 $1,516,000 $199,000 $602,000 Source: 2012 Annual Report Expert Panel Report — Platform Outcome Measurement Study─CRKN, December 2012 10 Although CRKN has been able to meet the needs and expectations of its membership by maintaining and also improving the efficiency and value of its services — for example, through the development of the License Information Module (see page 7) — one CRKN representative at the site meeting described its operational capability as “stretched since 2007.” This individual noted how difficult it was both to enhance the value for money that members received from licensing agreements and to prepare for the changes that were occurring in the world of data production and scholarly communication. It was also the view of the Expert Panel (EP) that management and staff may be questionably sufficient for current operations and too lean to face the opportunities and challenges ahead. The EP members were amazed when informed of the low cost of membership, which ranges from $3,677 to $31,519 annually, depending on the size of the university. CRKN has been operating at a deficit for the past two years (see Table 1), so it is the opinion of the EP that if CRKN is to survive, the membership fees will have to increase unless new funding opportunities from governments arise or other sources of revenue are sought and secured. 1.5 Platform capabilities and sustainability “To date, CRKN has negotiated and continues to administer 52 licences with total content expenditures of $89 million in 2012. This represents approximately 56 percent of all academic library expenditures on electronic content in Canada.” — CRKN Report to the EP, from CARL statistics 2010–2011 EP rating of the platform capabilities State of the art CRKN does not manage physical infrastructure itself but, rather, negotiates the conditions for access to digital content hosted by scholarly publishers and vendors. During licensing negotiations, CRKN aims to secure all the conditions set out in its model licence in order to provide the most flexible-use environment for researchers and students. From 2001 to 2012, the number of licence agreements managed by CRKN increased more than fivefold (see Figure 2). Currently, CRKN manages 52 licences with publishers and content vendors as well as 2,616 individual agreements across the membership; this is up from 33 licences across 1,487 agreements in 2007, when the CFI and its partners announced their second contribution to the Network (see arrow on Figure 2). In terms of negotiating licences with publishers, the Expert Panel (EP) members stated that CRKN is the equal of other leading international organizations with similar mandates with which they are familiar, and it was the envy of the member from the United States. A survey conducted in 2012 revealed a high level of satisfaction with the services CRKN provided to its members, even higher than that reported in a similar survey conducted in 2009. Expert Panel Report — Platform Outcome Measurement Study─CRKN, December 2012 11 Figure 2 Licence Participation NOTE: Arrow indicates second CFI grant Source: CRKN Report to the EP As emphasized in the CRKN Report to the EP, it is important to recognize that CRKN’s growth has been both quantitative and qualitative, due to the evolution of digital technologies as applied to different media beyond the traditional written word. When it began, the digital information licensed by CRKN was primarily e-journals and citation databases. Newer digital technologies, such as streaming audio and video with interactive Web 2.0 capabilities, now enable new research approaches and increase the complexity of CRKN’s business. EP rating of the capacity and quality of platform personnel Medium The quality of personnel is high, and CRKN’s standards are rigorous. The “Medium” rating is absolutely not a reflection of staff competence but of the stresses under which they work, such as frequent deadlines for negotiations, the amount of member funds involved and a rapidly changing academic publishing environment. All Member Services staff have a master of library science degree, and their jobs are high pressure and subject to constant change. There is a limited training budget but not many opportunities for training, though staff do attend professional conferences. Owing to these job stresses and the competition in the Ottawa region for such highly qualified staff, stability of staffing is an issue. This is doubly important for an organization whose ability to negotiate favourable terms is likely to be helped by the building of trust and familiarity between representatives of the publishers and CRKN staff. The EP recognized that the few CRKN personnel are supplemented by the expertise of its members who volunteer to serve on working groups. For example, the Negotiations Resource Team, which consists of 10 member librarians, assists staff negotiators and monitors the execution of agreed-upon negotiations strategies. Expert Panel Report — Platform Outcome Measurement Study─CRKN, December 2012 12 EP rating of the overall approaches to sustainability of the platform and its related services Medium Sustainability has been an ongoing challenge for CRKN. It has been operating at a deficit for the past two years and, to deal with this, has reduced operating costs (governance review, change in office location) and developed a multi-year financial model to manage the deficit. Considering the remarkable benefits members obtain from the Network, the EP could not understand why the support provided by the CRKN membership was so parsimonious as to compromise its existence. Perhaps this is because, as a CRKN representative noted ruefully, even if CRKN delivers the same savings to institutions year after year, its perceived value gradually decreases because members notice only the annual increases in subscription costs and forget how bad things were before CRKN existed or how much more costly it would be for them if CRKN did not exist. The EP suggests that the value proposition for CRKN may not have been adequately conveyed to stakeholders, above all to the most senior administrators in the nation’s universities. The transition in leadership was viewed as an opportunity for the new executive director to tour the major universities, listen carefully to the expectations university leaders have for profiting from changes in the generation and use of digital research information and remind them of the benefits and potential of CRKN to assist universities in dealing with the challenges of the rapidly evolving business models for scholarly communication, such as open access, discussed in more detail in Section 4, “Challenges.” Through its Finance and Audit Committee, CRKN has explored other sources of revenue and operational efficiencies, as well as ways to further increase its value to members. One interesting idea is the use of futures contracts to minimize exposure to fluctuating currency exchange rates, reducing financial risk to publishers, which helps in negotiating more favourable terms. An enhanced statistical service provided on a for-fee basis might be of value to members. Integrated, standardized measures of access and use are badly needed. Increasing the number of members would also increase operating revenue; however, as discussed in Section 2.2, these would have to be recruited outside the fully participating university sector. It might also be reasonable to charge new members an initiation fee in recognition of the existing members’ previous investment in the establishment of CRKN and its well-functioning operations. CRKN might consider commissioning a thorough analysis of its economic benefits. Following the EP meeting, the author became aware of an economic-impact analysis of a data provision and management service in the United Kingdom1 that could serve as a model process for CRKN. 1 Economic Impact Evaluation of the Economic and Social Data Service. March 2012. Available at: http://www.esrc.ac.uk/_images/ESDS_Economic_Impact_Evaluation_tcm8-22229.pdf. Accessed 2012-12-08. Expert Panel Report — Platform Outcome Measurement Study─CRKN, December 2012 13 1.6 Leadership and competitiveness “No country has done a better job of national licensing of research literature than the Canadian Research Knowledge Network. Recently, I have been suffering from ‘Canada envy.’ If you want to see how a national information system can be transformed to support research progress and economic development, I say: Watch Canada!” — Ken Frazier, Director (retired), General Library System, University of Wisconsin-Madison, in CRKN Report to the EP EP rating of the overall competitiveness of the platform in the international context based on its leadership, reputation and other relevant benchmarks International level CRKN has attracted interest from around the world, and its model licence agreement is also used outside Canada. CRKN and several of its staff have received an impressive series of national and international awards for excellence and innovation. The Expert Panel (EP) members who are knowledgeable about similar platforms internationally considered CRKN on a par with other leading networks in terms of its services and capabilities. The prices for licences that CRKN has negotiated seem to be highly competitive with what universities in similarly research-intensive nations pay. However, it was noted that some networks are more innovative than CRKN appears to be and less severely constrained by resource limitations. Yet the EP recognized that comparisons with like organizations are difficult, because each has a different range of responsibilities for a different range of clients and is embedded in a unique national research environment. 1.7 Linkages EP rating of the impact of the convening and planning activities of the platform Medium The major impact of CRKN’s planning and convening activities has been on its immediate clients and members, the university libraries, and this includes coordination of licence purchases at the national, regional and institutional levels (see below) so that CRKN influences the entire electronic publication acquisition by Canadian universities, not just the content for which it negotiates licences directly. CRKN also participates in national initiatives such as the Canadian Access Federation, managed by CANARIE, and internationally, it is an active participant in the 200-member International Coalition of Library Consortia. The Expert Panel (EP) considered the impact of these activities to be as expected and appropriate for CRKN’s scope and mandate. Expert Panel Report — Platform Outcome Measurement Study─CRKN, December 2012 14 EP rating of the extent to which the platform has established and fostered collaborative relationships High The task forces established by CRKN are an important way of bringing its clients together, and broad representation is made easier by the parity of access to research information fostered by CRKN’s negotiated agreements. There is interaction with peer organizations through attendance of CRKN staff at conferences, and staff participate in the International Coalition of Library Consortia, helping to build a network of colleagues internationally and allowing CRKN to both contribute to and learn from cognate organizations. CRKN recently signed co-licensing agreements with the United Kingdom’s JISC Collections and the Center for Research Libraries in the United States. In addition to the national CRKN, there are four regional library consortia (Conférence des recteurs et des principaux des universités du Québec; Council of Atlantic University Libraries; Council of Prairie and Pacific University Libraries; and Ontario Council of University Libraries), which predate the establishment of CRKN, and all CRKN members are also members of one of the regional library consortia. The EP explored the relationship between CRKN and the regional library consortia. It was explained that this was mutually supportive. For example, in licensing content of regional interest, the regional consortia made use of the CRKN model licence. Dialogue between the regional consortia and CRKN ensured there was no duplication of licensing activity, and collectively, the consortia and CRKN were monitoring which licences would be more appropriately devolved to a regional consortium or escalated to CRKN. There are also ongoing discussions about services that could be better provided at the regional or national level; for example, the possibility of a national network of print repositories. Expert Panel Report — Platform Outcome Measurement Study─CRKN, December 2012 15 2. Impacts of CRKN 2.1 Foregone costs “Since its inception, CRKN has purchased more than $767 million (in 2012 Canadian dollars) of content on behalf of its member institutions. Estimates suggest that if CRKNlicensed content were acquired on an institution-by-institution basis, the cost to CRKN members would be nearly $2.2 billion over the same time frame.” — CRKN Report to the EP EP rating of the benefits of foregone costs to the academic research community and funders High Compelling data were presented in the CRKN Report to the EP, documenting massive savings in licensing costs to institutions. For example, the price negotiated by CRKN for the recent renewal of licences from two important publishers, Swets and Elsevier, was $88.6 million compared with the vendor price of $181.3 million outside the Network. It was estimated generally that if members had to negotiate individually for content licences, the cost would be twofold to threefold greater than the cost to CRKN and that savings to members totalled $1.43 billion over the life of CRKN. This is an exceptional return on the CFI’s and its partners’ contributions, which totalled $97.7 million in the two tranches of funding. The Expert Panel (EP) noted that there are always challenges in estimating the value of foregone costs and asked whether the calculation of foregone costs in the CRKN Report to the EP was based on publishers’ list prices for licences, which nobody pays, resulting in an exaggeration of the benefits of CRKN. The EP was told that these estimates were based on realistic prices the institutions would expect to pay if negotiating the licences individually and were, therefore, reliable estimates of foregone costs. There are also further significant savings to each member institution because it does not have to hire as many staff to conduct its own negotiations or can redirect staff time to other valuable duties. One university representative noted that if his university had to negotiate independently with the publishers, it would require two full-time staff. Another pointed out that her university obtained 30 licences through CRKN and that its operation fee to support CRKN was only $15,000 annually, far less than the salary of the licensing negotiator it would need in the absence of CRKN. In addition, the estimated licensing cost savings for this university were $4 million per year. This represents extraordinary value for money, and it is not surprising that there has never been a defection from CRKN. Some libraries have used the costs saved in the licensing agreements to increase the number of titles in their collections. Thus CRKN is not only about cost savings but also about improving the range of accessible material (see also the example cited in Section 2.3). Expert Panel Report — Platform Outcome Measurement Study─CRKN, December 2012 16 2.2 Expansion of access and usage “Canadian university researchers increasingly find themselves at the centre of [research] networks, acting as ‘information gatekeepers,’ because in many instances, they have access to a larger body of information than their colleagues at universities in many other parts of the world.” — cited in CRKN Report to the EP2 A 2004 survey3 found that CRKN (CNSLP at that time) allowed a 436 percent increase in access to journals. In other words, researchers at a hypothetical institution that previously held 100 titles had gained access to an additional 436 scientific publications, thanks to CRKN. One CRKN representative explained to the Expert Panel (EP) the impact on a leading research university library. In the early 1990s, the library subscribed to about 15,000 serials and had good access to citation databases but no electronic content. By 1999, budget cuts and increased subscription costs had forced the number of serials down to about 10,000 titles, with no capacity to purchase digital content. The collection now numbers 68,000 titles, of which 75 percent are delivered electronically. Admittedly, the acquisitions budget has doubled as a result of the expansion of the university. Nevertheless, a doubling of budget has resulted in a sevenfold increase in titles, thanks to CRKN-negotiated savings in subscriptions that could be applied to the purchase of more serials. EP rating of the suitability of the platform’s access policies and procedures High Securing the best possible access has been a key feature of CRKN’s licensing negotiations and its model agreement. This ensures 24/7 access on or off campus and permits use of content for course materials. The model agreement has also been of benefit to individual institutions and the regional consortia as they negotiate their own licence terms. EP rating of the evolution in the size and distribution of the platform’s potential user community High EP rating of the extent to which the user community supported by the platform is geographically distributed High Over 1.2 million users benefit from CRKN’s services, including students, staff and more than 42,000 full-time faculty members, and there has been steady growth (3 to 5 percent per year) in these numbers during most of CRKN’s existence. Content made accessible through CRKN’s 2 The Impact Group. April 2011. CRKN Outputs and Impacts: Findings of Four Focus Groups With the Research Community. Available at: http://www.crkn.ca/sites/default/files/file/CRKN%20Focus%20Group%20Report%20%20Impacts%20and%20Outcomes%202011.pdf 3 The Impact Group. April 2004. Impact of the Canadian National Site Licensing Project: A Report to Partners and Stakeholders. Available at: http://www.crkn.ca/sites/default/files/about/Impact_Final_Report_Apr04.pdf Expert Panel Report — Platform Outcome Measurement Study─CRKN, December 2012 17 licensing activities now covers all disciplines, with recent emphasis on expansion into the social sciences and humanities. Its 75 members include all the research-intensive and undergraduate universities in Canada, anglophone and francophone alike. EP rating of the extent of utilization of the platform and its services in relation to its capacity and performance targets High CRKN noted that it had difficulty in collecting comprehensive usage data. It compiled statistics from some publishers from 2001 to 2004, but between 2005 and 2008, it did not have the staff to manage the data. It began receiving more comprehensive statistics from all publishers only in 2009. However, available data show that utilization has increased rapidly (a tenfold increase in downloads per researcher for articles published by the largest scientific publisher between 2002 and 2007). Usage may have stabilized in some areas in recent years since all the major serials are now available, the user community has expanded to include virtually all areas of scholarship and all regions with post-secondary institutions are covered. In addition, both research funding and the number of researchers in Canada are not increasing at the same rate as they were during the 2000s. Figure 3 CRKN Virtuous Cycle Improved access to research knowledge More licences secured CRKN’s virtuous cycle Increased research productivity More publications in more journals. More demand for information The platform seems to be fully utilized, relative to its staff capacity, and may be overcapacity, as previously discussed. To some extent, CRKN’s growth has been driven by its own success, since it establishes a virtuous cycle (see Figure 3) that raises expectations for ease of access, scope and scale of licence agreements and augments its workload. Although the university sector membership is essentially complete, there is potential for further growth in membership, with the community colleges and health-care providers unaffiliated with a university as major untapped client groups. Expert Panel Report — Platform Outcome Measurement Study─CRKN, December 2012 18 2.3 Research enabled EP rating of the platform’s impact on the quantity and the quality of research enabled High The work of researchers across the country and in all disciplines has been profoundly impacted by having ready access to a broad range of research information. The quantitative increase in Canadian publications over the years that CRKN and its predecessor, CNSLP, have operated (see Figure 4) is suggestive quantitative evidence of its impact, but multiple other factors, including increased research funding and growth of the research establishment, have combined to produce this increase. Figure 4 Canadian Publications, All Fields Source: SCImago (Scopus data) While it is an intuitive conclusion that access to research literature will improve the quality of Canadian research, it is not easy to demonstrate a specific, generalized improvement that can be attributed to CRKN. A study4 conducted in the United Kingdom and cited in the CRKN Report to the EP found that “use of e-journals is strongly and positively correlated with papers published, numbers of PhD awards and research grants and contracts income.” In its Report to the EP, CRKN offered several examples, across a range of disciplinary and interdisciplinary 4 Research Information Network. April 2009. E-journals: their use, value and impact. Available at: http://www.rin.ac.uk/our-work/communicating-and-disseminating-research/e-journals-their-use-value-and-impact. Accessed 2012-12-09. Expert Panel Report — Platform Outcome Measurement Study─CRKN, December 2012 19 research, pointing to improved quality of research. In addition, the Expert Panel (EP) heard firsthand from the researcher member of the Board a convincing example that helps to validate the intuitive conclusion. Dr. Ray Siemens, Canada Research Chair in Humanities Computing and distinguished professor in the faculty of humanities at the University of Victoria, spoke via phone about the way that CRKN-enabled access to content has revolutionized his work in the digital humanities, leading to innovation in traditional research approaches and allowing his research group and collaborators to address new questions. CRKN helped his team gain full access to its licensed materials and assemble large integrated disciplinary knowledge bases that could be used for data mining. His goal is to understand how researchers access and use electronic resources so that he can design better tools for discovery and analysis of humanities texts. “Canadian researchers are now seen as being so well connected to current sources of information that they have become magnets for international collaboration.” — cited in CRKN Report to the EP5 Even if hard to quantitate, CRKN’s impact on research collaboration can reasonably be inferred. Although Canadian research has always been strongly collaborative, international collaboration has increased significantly over the past decade. In 2001, 30 percent of all Canadian publications were international collaborations, and that rose to 45 percent in 2011 (Scopus data). In summary, CRKN’s activities have leveraged the investments made over the past 15 years by both levels of government in strengthening university research. 2.4 Contributions to the training of highly qualified personnel (HQP) EP rating of the impact of the platform on increasing or sustaining the training of HQP since the base year High EP rating of the impact of the platform on quality of training (e.g. through access to state-of-the-art facilities, data that would not otherwise be available and interactions with peers and users from other institutions) High According to the Association of Universities and Colleges of Canada (AUCC),6 enrolment of undergraduate and graduate students at Canadian universities increased from 700,000 in 2000 5 The Impact Group. April 2011. CRKN Outputs and Impacts: Findings of Four Focus Groups With the Research Community. Available at: http://www.crkn.ca/sites/default/files/file/CRKN%20Focus%20Group%20Report%20%20Impacts%20and%20Outcomes%202011.pdf 6 Association of Universities and Colleges of Canada. 2011. Trends in higher education. Volume 1: Enrolment. Expert Panel Report — Platform Outcome Measurement Study─CRKN, December 2012 20 to over 1.2 million in 2011. Approximately 99 percent of the students enrolled in AUCC member institutions are enrolled in CRKN member universities. The number of graduate students with access to CRKN-licensed resources has also risen steadily since 2001. Currently, over 192,000 of these students are graduate students who require access to the most current research literature as part of their training and thesis production. Results from several surveys conducted during the life of CRKN show that having access to online information improves both the quality of teaching and the range of student learning experiences, including the type of research that graduate students can pursue. CRKN’s model licence permits the reproduction of research articles in coursework, bringing an appreciation of the power and limitations of evidence to enrich the education of legions of undergraduates. Allowing undergraduates to access the original research literature motivates them to become involved in research themselves. CRKN has also increased accessibility to a greater range of publications, particularly in small institutions, and has therefore had a positive effect on the quality and scope of training of HQP at all stages. It is not possible to quantitate the magnitude of this impact. With respect to training within the professional library community, CRKN has set up a Community of Practice focused on licensing issues to educate those who have to negotiate licences on behalf of individual institutions or regional consortia. Through the Community of Practice, expertise in the area of collections and acquisitions has been developed across Canada. 2.5 Cultural, organizational and structural change enabled at stakeholder institutions “CRKN has brought HUGE value to our library. From its very inception, CRKN has allowed us to greatly expand the range and depth of scholarly content that we provide to our academic community.” — from the CRKN Report to the EP, quoting a respondent to the 2012 member satisfaction survey. EP rating of the impact of the platform on activities and services at stakeholder institutions High The Expert Panel (EP) noted that the overall goals of CRKN and its realized impact are well aligned with Canadian values. The licensing agreements allow for parity of access across institutions of all sizes and in all regions of the country, allowing small- and medium-sized universities access to a world-class collection of digital content on a par with the large researchintensive universities, helping all institutions to recruit and retain high-profile researchers. Although the initial focus was on science, technology and medicine, the social sciences and humanities are now also well served. CRKN’s achievement is all the more remarkable because Available at: www.aucc.ca/_pdf/english/publications/trends_2007_vol1_e.pdf Expert Panel Report — Platform Outcome Measurement Study─CRKN, December 2012 21 the Network has established a national platform involving post-secondary institutions that are under provincial jurisdiction and has also accommodated the needs of the two official languages. CRKN has fostered a spirit of co-operation among universities and with and among regional consortia, and the universities have responded to the opportunity to join CRKN because of the evident benefits of membership, as expressed by Tom Sanville of LYRASIS (a U.S. library consortium): “CRKN is providing the necessary means to achieve a level of costeffectiveness, control and expanded information access that individual libraries cannot hope to achieve individually.”7 There is some differentiation in benefits of CRKN membership between large and small universities. While the cost savings to members resulting from CRKN’s negotiation of licences are greater in dollar terms for the large institutions, the benefits in terms of access to research information are proportionately greater for the small institutions, which, thanks to CRKN, have access to a far greater range of serials than would be possible if they had to negotiate individually or even as regional consortia. Katherine Schultz, former vice-president of research at the University of Prince Edward Island (UPEI), pointed out to the EP that CRKN has conquered geographical limitations. The ability of researchers at UPEI to have 24/7 access to the world research literature was an important factor in this geographically isolated university’s being able to increase its research grant and contract income sixfold over the life of CRKN and helped UPEI to welcome the National Research Council Institute for Nutrisciences and Health to its campus. This small university has also been able to attract a Canada Excellence Research Chair, and in Dr. Schultz’s opinion, that would never have happened without the access to the electronic literature that CRKN enabled. While this perception of differential benefit to institutions large and small represents a potential rather than actual strain within CRKN, it must be kept in mind when considering innovations that may be of greater benefit to institutions of a certain size. EP rating of the impact of the platform on how research is done in the discipline(s)/field(s) it supports High EP rating of the impact of the platform on Canada’s international visibility Medium It is impossible to determine CRKN’s impact on the reputation of the Canadian academic research enterprise in isolation, though access to digital content is as vital to research as is electricity or running water. It is important to note that CRKN has done more than lowering the cost of access to e-journals. It has also secured favourable terms that maximize the accessibility and utility of digital information. CRKN has allowed more researchers to access more information more easily and rapidly and to monitor more information within and across more fields more efficiently than ever before. Researchers have access to an expanded range of online information within their disciplines and are using strategies such as preprogrammed 7 Quote by Tom Sanville, director of licensing and special projects at LYRASIS, from CRKN Report to the EP. Expert Panel Report — Platform Outcome Measurement Study─CRKN, December 2012 22 information searches to obtain up-to-the-minute information about research advances in their fields and across other possibly relevant fields that they previously would have been unable to monitor. The CRKN Report to the EP provided an excellent example of how digital information can facilitate knowledge transfer across disciplines. Dr. Cristian Suteanu, associate professor in the geography department and environmental science program at Saint Mary’s University in Halifax, is studying river flows. When he used the digital research content licensed by CRKN to search for information on fluid flows related to rivers, he uncovered a cardiology paper which contained a scientific approach relevant to his own work and which he has now incorporated into his methods. Without this access, he would not have made the connection between river flow and blood flow that has improved his research approach. This immediate access to the 95 percent of research findings obtained outside Canada will have a multiplicative impact in combination with the other significant investments made in research support by federal and provincial governments over the life of CRKN, but neither the EP nor the CRKN team was able to quantitate this impact on international visibility; therefore, the EP was reluctant to categorize it as “High.” 2.6 Extrinsic benefits: Impact on local, regional and national innovation “Digital content is a critical component of a digital economy. CFI’s investments in CRKN, therefore, have contributed to the development of a robust national digital infrastructure and towards fostering a vigorous digital economy in Canada.” — CRKN Report to the EP EP rating of the amount of knowledge translation and transfer catalyzed by the platform EP rating of the importance of socio-economic benefits catalyzed by the platform to Canada and Canadians Medium High The major focus of CRKN’s efforts to secure improved access to the research literature has properly been the university community. But the inclusion of “walk-in” users in CRKN’s model agreement permits access to the research literature by anybody within proximity of a university library. This is an understated but significant CRKN achievement and an important potential contribution to translating knowledge from the academic to the broad public domain. However, the relationships between the platform (licensing content) and the end-users of knowledge derived from university research (health, technology, environment and broad socio-economic benefits) are multi-dimensional, indirect and well beyond CRKN’s direct influence. The Expert Panel (EP) recognized that the impact of CRKN outside academia might be underestimated, especially in regions served by the small universities whose digital collections have been expanded the most. The extent to which member universities publicize this public access opportunity to their local communities is variable. As an example of good practice, the University of Waterloo has Expert Panel Report — Platform Outcome Measurement Study─CRKN, December 2012 23 established the Industrial and Business Information Service to provide an accurate, reliable and timely service at a reasonable cost to meet the information needs of businesses, industry and other individuals outside the academic community. Another example of a deliberate attempt by CRKN members to support knowledge-translation activities is the Irving K. Barber Learning Centre at the University of British Columbia, which offers a broad range of programs and services that support teaching and learning, as well as lifelong learning and community engagement, through the development of partnerships between the university and the wider community. In addition, the two digitization projects supported by CRKN’s Digital Content Infrastructure for the Human and Social Sciences (DCI) Project are open access. The EP discussed with CRKN representatives the opportunities for improving access to the research literature for small and medium enterprises (SMEs). It was pointed out by CRKN that many SMEs retain links to the university where the intellectual property they are exploiting has been developed (e.g. through adjunct appointments), in which case access through the university library is available. However, there are situations where this option is not available, and the high fees demanded by publishers for “pay-per-view” access to single articles (generally in the range of $35) make them unaffordable for struggling SMEs. Institutions in all parts of the country now have better access to a wider range of serials, so if university libraries could provide better access for SMEs, under favourable CRKN licence terms, this could facilitate the work of research and development clusters and support regional economic development. The EP rated the importance of the broader socio-economic benefits resulting from CRKN as “High” because of its essential underpinning of the university research enterprise, as amply discussed in preceding sections. Expert Panel Report — Platform Outcome Measurement Study─CRKN, December 2012 24 3. Influence of the CFI and its funding partners EP rating of the impact of the CFI in enabling acquisition and/or enhancement of the platform and the associated capabilities High Described as a “game changer” by one member of the Expert Panel (EP), CRKN would not have happened without the CFI’s support, which provided sufficient funding to get the partners committed to the Network. The impact of the CFI and partner funding was also a result of perfect timing. It came at the right moment to lower university libraries’ e-journal entry costs and the associated risks of moving to electronic collections and made it easier for libraries to justify an increased university investment in the required electronic information delivery infrastructure. The CFI also persuaded the pre-existing regional consortia (Section 1.) to collaborate in realizing the vision of a single national licensing authority. The most important CFI decision was to recognize that research literature is infrastructure, which is just as vital to high-quality research as are buildings and state-of-the art apparatus. That was a prescient policy decision in the early days of the CFI, one which has been vindicated by the subsequent rise of research approaches that require “big data” and their computerized analysis. The significance of the CFI’s foresight becomes ever more apparent as secondary use of data, through data mining and other associative strategies, is increasingly utilized to generate new knowledge and reveal hidden relationships. Without CFI funding, it is likely that the four regional consortia would now be operating independently and duplicating licence negotiations, large universities would be paying more for their subscriptions and small universities would have far less complete collections. The CFI’s insistence on a national approach to licensing was critical in ensuring that Canadian researchers have the maximum access to information for the minimum cost. Canada is a small nation in terms of its resources for supporting research and scholarship and must avoid duplication of effort. CRKN is a splendid example of the value of a national approach to fundamental research needs. Expert Panel Report — Platform Outcome Measurement Study─CRKN, December 2012 25 4. Challenges “The evidence is compelling: CRKN has done a fantastic job of fulfilling its mission. But it is severely underresourced to deal with the changing technologies of scholarly communication.” — EP member Over the years when electronic publishing emerged and grew, CRKN achieved great success in lowering costs and increasing access to research literature for Canadian institutions, but its future during the next evolutionary phase of scholarly publication appears less certain. As mentioned previously, the Expert Panel (EP) was concerned about a number of organizational stresses, such as high staff turnover, most of which could be traced to its level of funding from member fees, which is inadequate for current operations. Following up on this concern, the EP asked about risk-management processes in CRKN, noting that its Finance and Audit Committee had identified the importance of a risk-mitigation plan in 2009. The response focused on management of financial risks, and the EP was entirely satisfied with CRKN’s day-to-day financial management and accounting practices, as built in to its multiyear financial model. However, the risk-management framework related to matters of reputation, operational effectiveness, business continuity and obsolescence was still a work-inprogress, another victim of inadequate staff time to devote to anything other than the day-today business. One of the emerging challenges for CRKN is the transition from subscription-based journals, where access is restricted to those at subscribing institutions, to “open access” journals, where content is freely available to anyone with an internet connection and publishers obtain their revenue from article processing charges (APCs) and/or institutional and funding organization subsidies. This disruptive change in scholarly publication models was described by one EP member as “a transformation as significant as the move from paper to electronic content” and was discussed at some length. It was pointed out that this involved a shift in payer from the institutions to the granting agencies, whose funds paid for most APCs. The granting agencies should also be involved at an early stage in discussing how to manage this transition. CRKN is clearly aware of the many issues swirling around open access, but it is not so clear that it has the resources to maintain its current subscription licence negotiations while also conducting new negotiations to obtain similarly favourable terms for APCs for authors from member institutions. The EP questioned the resilience of CRKN, as currently resourced, to deal with the many challenges, such as open access, it will encounter as the nature, format and medium of scholarly communication change rapidly, along with the ways in which this content is used by the research community. Given the impact of these exciting but disruptive developments on the performance of the research and innovation system, CRKN needs to benefit more from the vision and foresight of the end-users of research knowledge and to be supported accordingly by its member universities. Clearly, CRKN is aware of the changes in the business models being adopted by publishers and the specific issues they pose. It pointed out, for example, that the Expert Panel Report — Platform Outcome Measurement Study─CRKN, December 2012 26 American Chemical Society has adopted a pricing approach based on usage that contravenes the licensing principles of CRKN. The challenge for CRKN is how to support these new models in a way that continues to provide exceptional value for its members while maintaining its regular licence negotiations. The Board and members of CRKN have to resolve fundamental questions about its future. Does it remain entirely focused on the negotiation of licences, a vital but limiting function of a contemporary research knowledge network? Or will CRKN embrace innovation and continue to play a leading role in enabling Canadian universities to take full advantage of the profound changes in the way research knowledge is acquired and used? Expert Panel Report — Platform Outcome Measurement Study─CRKN, December 2012 27 5. Conclusion “Ten years ago, we were looking only at content. Now we should be looking at new forms of content and scholarly communication: open access, open collaboration. In this new world, CRKN needs the capacity to innovate.” — Joyce Garnett, university librarian, Western University, VITaL Task Group chair. Based on the report provided by CRKN and presentations from supportive CRKN members, Board members and staff, the Expert Panel (EP) was convinced that CRKN is a well-run organization that has delivered exceptional returns for its members year after year and will continue to do so. The returns have come in the form of foregone costs for its members and, for the research community, increased and easier access to a wider range of research literature at universities and their affiliates in all regions. The EP was equally certain that CRKN understands the challenges presented by emerging technologies and new forms of digital scholarship and has the planning and strategic capability to respond, though currently lacking the capacity to do so. Its sustainability is threatened by inadequate support from stakeholders, particularly its membership. This is impeding CRKN from innovating and attempting the necessary ventures, with their associated risks, that will allow Canadian universities to reap the full benefits of the revolution in research publication and the ways it can be used to advance new forms of scholarship. The original support of the CFI and its funding partners was essential, timely and catalytic, stimulating the universities and the existing regional consortia to co-operate in building a national platform with the widest scope and reach. The EP also commended the CFI for extending its Outcome Measurement Study methodology into the area of platform evaluation. National platforms like CRKN have such a pervasive influence on the performance of Canada’s research and innovation system that they must be subject to periodic formative review to help them continue to be well run and to deliver the maximum benefits. Expert Panel Report — Platform Outcome Measurement Study─CRKN, December 2012 28 Summary of ratings Operation of CRKN EP rating of the effectiveness of planning and performance monitoring in platform planning High EP rating of the impact of the platform’s planning process on the evolution of the platform and its user community since the base year High EP rating of the extent and suitability of stakeholder involvement with the platform Medium EP rating of the adequacy of the platform enhancements since the base year in comparison with the initial capital investment (to keep platform offerings up to date) EP rating of the platform capabilities High State of the art EP rating of the capacity and quality of platform personnel Medium EP rating of the overall approaches to sustainability of the platform and its related services Medium EP rating of the overall competitiveness of the platform in the international context based on its leadership, reputation and other relevant benchmarks International level EP rating of the impact of the convening and planning activities of the platform EP rating of the extent to which the platform has established and fostered collaborative relationships Medium High Impacts of CRKN EP rating of the benefits of foregone costs to the academic research community and funders High EP rating of the suitability of the platform’s access policies and procedures High EP rating of the evolution in the size and distribution of the platform’s potential user community High EP rating of the extent to which the user community supported by the platform is geographically distributed High EP rating of the extent of utilization of the platform and its services in relation to its capacity and performance targets High EP rating of the platform’s impact on the quantity and the quality of research enabled High EP rating of the impact of the platform on increasing or sustaining the training of HQP since the base year High EP rating of the impact of the platform on quality of training (e.g. through access to state-of-the-art facilities, data that would not otherwise be available and interactions with peers and users from other institutions) High EP rating of the impact of the platform on activities and services at stakeholder institutions High EP rating of the impact of the platform on how research is done in the discipline(s)/field(s) it supports High Expert Panel Report — Platform Outcome Measurement Study─CRKN, December 2012 29 EP rating of the impact of the platform on Canada’s international visibility Medium EP rating of the amount of knowledge translation and transfer catalyzed by the platform Medium EP rating of the importance of socio-economic benefits catalyzed by the platform to Canada and Canadians High Influence of the CFI and its funding partners EP rating of the impact of the CFI in enabling acquisition and/or enhancement of the platform and the associated capabilities Expert Panel Report — Platform Outcome Measurement Study─CRKN, December 2012 High 30 Expert Panel Report — Platform Outcome Measurement Study─CRKN, December 2012450-230 Queen Street Ottawa ON K1P 5E4 Tel 613.947.7260 Fax 613.943.0227 1 450-230 rue Queen Ottawa ON K1P 5E4 Tél 613.947.7260 Téléc 613.943.0227