The dynamics of ESFRI Projects and ESFRI Landmarks and the needs and targets of the implementation are understood within an overall lifecycle approach, as depicted in Figure 1.

The CONCEPT of a new RI typically emerges bottom-up from the scientific communities clustering around well-identified scientific needs and goals. Such a concept can originate from completely novel approaches to answer scientific questions or to respond to the need of enhanced capacity at pan-European level as well as from new insights in existing RI – e.g. resulting in a plan for major upgrade or merger.

The DESIGN covers the proof of the scientific concept and technical feasibility of the RI, the analysis of the potential user community – both science and Innovation oriented; the outline of a business case and the rationale for the international consortium. The feasibility study can be carried out with institutional, national or international support – e.g. Framework Programme (FP) Design Study grants. The design also includes an initial analysis of its position in the RI landscape, e-Infrastructure requirements and (open)-data management and policy. Importantly, the RI also foresees the financial and political support from governments and funding agencies necessary for the Preparatory Phase.

The PREPARATION – carried out at institutional, national, European or international level – is directed towards developing the RI as a fully-fledged organisation. Completion of preparation for the RIs in the Roadmap is often carried out through a Preparatory Phase contract under FP resulting in a business plan, a legal entity, an agreed role for the RI also in the context of the landscape of existing RIs at European and global level, and secured funding safeguarding the financial sustainability for the Implementation Phase and extending also for the Operation Phase. Some projects face a gap of funding between the end of their Preparatory Phase contract and the final decisions for implementation – legal, funding and construction – which can lead to the establishment of ad hoc interim legal entities and governance to assure appropriate funding to complete the preparation and start construction.

The IMPLEMENTATION is different for single-sited and distributed RIs. In the first case it corresponds to an intense investment period of several years for construction engaging human and financial resources with big impact on the market – suppliers of goods and technologies. Longer-term benefits are generated to the hosting territory: employment, upgrade of services, internationalisation and up-skilling of the population, increased demand on high level services – schools, communication, financial services for international employees – and joint development of novel technologies that remain as a  competitiveness legacy to the procuring firms. In case of distributed RIs, the implementation implies intense negotiations as both the Central Hub and the national nodes require specific commitments. The development of a successful governance and management structure may be of higher complexity than for single-sited RIs. Nevertheless, in several cases distributed RIs have been quite efficient at establishing their legal entities and launching services to the user community.

During their OPERATION, RIs produce frontier research and deliver advanced services for excellent science satisfying the users’ demand, boosting brain circulation of early career scientists and trainees, therefore improving the ranking of their academic and research institutions. RIs can create spin-offs and start-ups and attract corporate partners generating a high potential for innovation. The operational costs of RIs range from 8 to 12% of the initial capital investment per year. A twenty-year operation cycle may develop before major upgrades, requiring new substantial capital investment, are needed. The upgrade cycles in case of HPC and e-Infrastructures are typically much shorter.

The TERMINATION may encompass dissolution of the organisation, dismantling of facilities and related safety aspects and resurrection of the original site but it does not apply in these identical terms in all research domains. The Termination Phase could also result in a new infrastructure development as part of the evolution of the field. Re-orientation of RI sites has already occurred, e.g. in nuclear research or high-energy physics, where outdated RI have been transformed into analytical facilities with new science missions built upon the presence of technological infrastructure, logistics, human resources and organisation.

According to the lifecycle model, the ESFRI Roadmap Dynamics implies the MONITORING OF ESFRI PROJECTS. The ESFRI Projects that entered the Roadmap in 2008 and 2010 underwent monitoring in 2017-2018. In addition, ESFRI carried out a pilot exercise – PILOT PERIODIC REVIEW OF ESFRI LANDMARKS – for collecting elements to build a methodology to be applied in the future for the periodic updating of the ESFRI Landmarks’ state of play.

Both processes were performed in parallel by the Strategy Working Groups (SWGs) in the Energy (ENE), Environment (ENV), Health & Food (H&F), Physical Sciences & Engineering (PSE), Social & Cultural Innovation (SCI) and Data, Computing and Digital Research Infrastructures (DIGIT) domains for the evaluation of the scientific case, and by the Implementation Group (IG) for the assessment of the maturity aspects.

The scheme of the ESFRI Roadmap Dynamics is represented in Figure 2.

The specific goals of the MONITORING OF THE ESFRI PROJECTS is to check the overall progress towards implementation according to the fulfilment of minimal key requirements as defined in the ESFRI Roadmap Guide. The monitoring process leads to conclusions, advices to the Projects and recommendations to the Forum, including the possible promotion of Projects to the status of Landmark. The ten-year rule introduced with the Roadmap 2016 stipulates that Projects have ten years of residency on the Roadmap to reach implementation: those that do not meet this goal are removed from the Roadmap, albeit they can decide to re-apply in full competition with all new proposals. Projects that successfully reach the Implementation Phase can be further considered for the Landmark status.

The MONITORING OF SIX ESFRI PROJECTS 2010 – EU-SOLARIS, MYRRHA, WindScanner (ENE); AnaEE, ISBE, MIRRI (H&F) – was carried out with questionnaires addressing generic and specific aspects of the scientific case and progress towards implementation. The outcomes were presented to the Forum which took note of most common bottlenecks concerning legal and financial issues encountered by the projects.

The MONITORING OF NINE ESFRI PROJECTS 2008 – ECCSEL ERIC (ENE); EISCAT_3D, EPOS, SIOS (ENV); EMBRC ERIC, ERINHA, EU-OPENSCREEN ERIC, Euro-BioImaging (H&F); CTA (PSE) – was meant to verify the achievement of the Implementation Phase as expected by the end of their ten years in the Roadmap. Specific questionnaires were addressed to each of the nine ESFRI Projects 2008, followed by invitation to formal hearing sessions. The SWG and IG Chairs harmonised the results of the monitoring, and the Executive Board (EB) formulated the recommendation to the Forum to award the status of ESFRI Landmark to eight of the 2008 Projects. One project – SIOS – did not reach the pan-European dimension that is required to be an ESFRI Landmark. SIOS is described in the Landscape Analysis but it is not listed anymore as ESFRI RI.

The PILOT PERIODIC REVIEW OF ESFRI LANDMARKS has been carried out to develop and test a methodology for the periodic update of the state of play of the ESFRI Landmarks. Four Landmarks volunteered to be part of the pilot exercise, upon invitation. They were: ELIXIR (H&F), as a distributed RI example dealing with high data-intensiveness and e-needs; SPIRAL2 (PSE), as a single site RI with special implementation conditions; the ESS ERIC (SCI), as a distributed RI with issues connected to the management of social sciences data and e-needs; ICOS ERIC (ENV) as a RI with global outreach. The lessons learnt from the pilot exercise were presented to the EB and to the Forum. The inputs to the future methodology are sketched below.

With the Roadmap 2016, ESFRI performed a broad LANDSCAPE ANALYSIS of the Research Infrastructures accessible to European scientists and industry to structure an overview of the European RI ecosystem in all fields of research. This includes national, regional and international facilities, as well as consortia that offer integrated services and transnational access to state-of-the-art resources for research.

The Landscape Analysis is an indicative reference document and does not represent, in any way, the view and prioritisation of ESFRI, nor any national financial and political commitment.

The goals of the Landscape Analysis are to:

• provide a survey on major transnational RIs offering open access to researchers, students, teachers, support staff, education and research institutions, business, industries and public services in all domains;
• keep track of the developments and trends from thematic roadmaps and strategy papers;
• understand the complementarity and effectiveness of interfaces between RI, also across areas;
• provide an overview of the European RI ecosystem enabling ESFRI to fulfil its strategic and incubator roles;
• enable ESFRI to identify gaps in the European RI landscape and promote inter- and cross- disciplinary aspects;
• update evidence on the overall value and sustainability issues of the operational RIs.

In 2018, the SWGs updated the Landscape Analysis in the ENE, ENV, H&F, PSE, SCI scientific domains and the e-IRG updated the analysis on Data, Computing and Digital Research Infrastructures, with contributions from the DIGIT SWG. The thorough knowledge of the RIs Landscape and of its dynamics is a prerequisite for developing optimal strategies in the field of RIs aimed at strengthening the competitiveness and value – excellence and impact – of European research. The Landscape Analysis is a key ingredient of the ESFRI Methodology.

The selection of new proposals was carried out according to the well established procedure introduced for the Raodmap 2016, whose flow-chart is given in Figure 3.

Member States, Associated Countries and EIROforum members were eligible to submit proposals for the ESFRI Roadmap 2018. Since Roadmap 2016, in order to identify a limited number of projects with a high degree of maturity to ensure maximum likelihood to reach implementation within the ten-year deadline, ESFRI introduced important eligibility conditions: the expression of funding commitment by the lead Member State or Associated Country or a resolution of the Council for EIROforum organisations; the proof of political support by at least two additional MS/AC or EIROforum organisations; and the inter-institutional and multi-lateral agreement signed by the core partners formally involved in the consortium. These requirements strengthened both links with the Governments involved and the research communities, encouraged a closer dialogue among them and support from the very beginning of the RI project resulting in increased likelihood of successful implementation.

The deadline for online submission of new proposals expired on 31 August 2017. In September, the EB checked each proposal for eligibility and proposed attribution to the leading SWG. Proposals carrying a prominent impact across multiple domains were attributed also to a second SWG. In addition, as projects with dominant, or substantial, digital Research Infrastructure character were submitted, ESFRI decided to establish a new Strategy Working Group on Data, Computing and Digital Research Infrastructures – the DIGIT SWG.

The twelve submitted proposals were all eligible and underwent the two parallel and independent evaluation processes run by SWGs and IG, also involving independent external experts. Based on their analysis, the SWGs and IG identified several critical questions about each proposal, and then liaised to achieve a common list of issues. In one case the conclusions were that the proposal did not meet the minimal requirements to be considered as potential candidate for the Roadmap 2018. Eleven proposals were invited to dedicated hearing sessions during which the Coordinators addressed and discussed the issues. After the hearings, the SWGs and IG reached their conclusions and drafted a joint evaluation report for each project.

The harmonisation of the results on scientific excellence and maturity required three dedicated meetings of WG Chairs and EB. According to the ESFRI Roadmap Guide 2018, a threshold score High on both scientific excellence and maturity is needed for considering inclusion in the Roadmap 2018. Nine proposals were harmonized: four passed the criteria for inclusion and five did not meet the minimal requirements. Two projects could not be harmonised as the concerned SWGs and the IG could not reach a conclusive common position on the fate of the proposals. Based on these inputs the EB assessed the strategic value of the four proposals evaluated for inclusion, considering their role in filling gaps and overall reinforcing the Roadmap. The EB also discussed the two non-harmonised proposals, as both met the scientific excellence criterion and bear a high strategic potential value for a timely inclusion in the Roadmap. This led to the recommendation of the six new entries in the ESFRI Projects list that was adopted on 28 June 2018 during the 65^th ESFRI Forum Meeting in Corfu.

Ample discussion in the Forum also led to identifying the high strategic potential areas of research in the field of Social & Cultural Innovation: the area of religious studies and that of digital services for open science, as described in a dedicated page in the Landscape Analysis.

The ESFRI Methodology is continuously adapted to better serve the scope of the Roadmap, as well as to integrate the new mandates that ESFRI receives from the EU Council. In particular, the forthcoming refinements that will be discussed by the Forum will take into account the EU Council Conclusions of 29 May 2018 on Accelerating knowledge circulation in the EUConclusions of the Council of the European Union of 29 May 2018 on Accelerating knowledge circulation in the EU. Doc 9507/18 http://data.consilium.europa.eu/doc/document/ST-9507-2018-INIT/en/pdfthat “Stresses the importance of human resources and training skills as key factors in the success for Research Infrastructures and ACKNOWLEDGES the need for Research Infrastructures to strengthen a service-driven approach; INVITES Member States and the Commission within the framework of ESFRI to develop a common approach for monitoring of their performance and INVITES the Pan-European Research Infrastructures, on a voluntary basis, to include it in their governance and explore options to support this through the use of Key Performance Indicators”. These recommendations inspire the formulation of PERIODIC UPDATES OF LANDMARKS, and the work TOWARDS A FEW GENERIC KEY PERFORMANCE INDICATORS.

The concept of ESFRI Landmarks implies periodic updating of their state of play throughout all the lifecycle phases. ESFRI is aware that the inclusions of Landmarks in its Roadmap often are conditional on (eligibility for) regional, national and European funding and underlines that these inclusions are a result of the robust and transparent methodology. ESFRI will be responsive to the possible requests for guidance indicated by the Research Infrastructures, and will take note of all the internal and external evaluations performed by their stakeholders and users communities, in order to be effectively updated on the state of play of the Landmarks.

The goal of future periodic reviews of ESFRI Landmarks – henceforth referred to as Periodic Updates of ESFRI Landmarks – is to understand the scientific status of each Landmark, considering its specific position in its lifecycle, and enabling ESFRI to fully deploy its strategic role and support the Landmarks. Moreover, periodic updates on ESFRI Landmarks will facilitate meaningful interaction between ESFRI and its Delegations to address the gaps and areas of strategic importance highlighted in the Landscape analyses.

The periodic update of ESFRI Landmarks differs fundamentally from the evaluation and assessment of proposals and from the monitoring of Projects along the following lines.

• It is independent from the ESFRI Roadmap updates and each Landmark is addressed for update periodically, taking into account the specific context of each RI.
• The periodic updates primarily take stock of external and internal evaluations provided by the Landmarks themselves and carried out by their internal bodies, funders and stakeholders. Importantly, the periodic review is directed towards providing a basis for dialogue between ESFRI, Landmarks and other stakeholders, taking into account also national roadmap processes.
• ESFRI will identify a combination of generic KPIs and specific KPIs – suggested by the RI itself – for updating the state of play of the Landmarks. It is important that each KPI is set against certain targets or metrics in terms of self-assessment and quality assurance.
• The updates of the Landmarks are meant to establish a lasting dialogue between ESFRI and the ESFRI Landmarks – and their research communities – and with the involved countries.
• A non-bureaucratic, evidence based, periodic update of Landmarks will reinforce the strategic role and position of ESFRI in supporting these RIs when referring to the EU Council, as well as in allowing for the maintenance of the Landmarks’ list.

TOWARDS A FEW GENERIC KEY PERFORMANCE INDICATORS

To help address the unique aspects of each RI, Landmarks will develop specific KPIs and report on the few generic ones chosen by ESFRI, therefore updating their profile and capturing their own peculiarities. Avoiding one-size-fits-all shortcomings, indicative elements for developing flexible KPIs include:

• scientific excellence outcomes, output and delivery of talent;
• reference role in the disciplinary field (uniqueness in capabilities, in capacity) at European and international level;
• progress in achieving the RI´s milestones along its lifecycle;
• impact, innovation, entrepreneurship;
• establishment and development of the RIs’ user community;
• scientific data management policy, metadata catalogue interfacing EOSC and similar open science initiatives, advanced data services for scientific analysis and for innovation developments;
• enforcement of quality control of access (peer review), data (FAIR and reproducible), and services to research and innovation.

Another important development will concern the ADVANCED COSTS ANALYSIS of the RIs through their lifecycle. The ESFRI Roadmap quotes indicative costs of the Projects and, since 2016 of the Landmarks, under the categories of Capital Value, Construction Cost, Operational Cost, Termination Cost as they are described in the guidelines and obtained from the Coordinators of the Research Infrastructures. These cost values are indicative, but overall turn out to be not accurate as they are interpreted differently by different types of RIs, or quoted inconsistently through the years. As a consequence, ESFRI has taken the initiative to start a study and discussion to improve the economic information on RIs and to facilitate both the RIs to produce reliable data and the users of the Roadmap to retrieve consistent information.