Are you currently developing an MRFF funding application? If so, now is a good time to think about how your application will be assessed.
In March 2023 the MRFF delivered an informative webinar for future applicants 'Assessing MRFF grants: Insights from assessors'.
In this blog we summarise the key points raised by the webinar panel, with a particular focus on what members of Grant Assessment Committees (GACs) look for during the assessment process.
The big question: when is an application determined as 'not for further consideration'?
A similar assessment process is used for all MRFF schemes regardless of being submitted via NHMRC’s Sapphire or the Department of Health and Aged Care’s (DoHAC) business.gov.au hub.
Key assessment criteria for all MRFF applications
The key assessment criteria for all MRFF applications are i) project impact, ii) project methods, iii) capacity, capability and resources, and iv) overall value and risk. We’ve compiled the key issues that assessors consider when determining the strength of your responses to these sections. Considering the questions below may help you leverage your application toward success.
Project impact: alignment with Measures of Success
When assessing project impact, the other documents contributing to this are i) objectives and intended outcomes specified in the guidelines, ii) the MRFF’s initiative or Missions’ Roadmap and Implementation Plan, and iii) your project’s statement against the MRFF Measures of Success.
Notably, the third/final column of your Measures of Success statement indicates the project’s intended outcomes, so ensure you discuss the potential impact that will result from those outcomes.
In the webinar, much attention was paid to the meaningful engagement of consumers/end-users throughout all stages of the project, with the suggestion of embedding a consumer/end-user map. As with any project, it’s very clear to assessors when meaningful collaboration has not been undertaken and where consumers/end-users have not been involved in the conceptualisation of the research idea, prioritising the issues and defining the questions, study design and methodology. The underlying argument of robust engagement is that project impact will be substantially leveraged by meaningful engagement with consumers/end-users across the life of the project: who better to inform what is needed and the best way to do it than those most affected?
Project methods: a robust, richly detailed plan
When assessing methods, the other documents contributing to feasibility and sound methodology include the proposed timeline and milestones, and the team you have assembled.
The webinar panelists discussed what constitutes a sound methodology, with the key points being:
alignment with the Measures of Success statement - the middle column of your Measures of Success statement describes how your project will produce the outcomes
clearly articulated research questions and aims - the project impact section should have set you up to determine your aims -ensure your aims are as specific as possible, and clearly able to be addressed by your proposed study
richly detailed methods with a flawless plan - your research questions should drive the choice of methods - ensure the rationale for your choice of methods is clear - detail every step of the research and avoid any vagueness about the methodology that may attract questions about feasibility - share your methods section with your colleagues to see if they understand what you are proposing and how you’ll do it - present a coherent package, in terms of scope, design and expertise - write succinctly and ensure each element of your methodology fits together strongly - make it clear how your chosen methods enable you to address the aims - use a diagram, if relevant, to illustrate proposed methods
assemble a truly multidisciplinary team to do the work - think carefully about who is on the team and provide evidence about their expertise and complementarity with other team members - clearly identify in the methods who will be undertaking the various aspects of the research, and if junior researchers will do the work, how will they be supported by more senior researchers
early engagement with consumers/end-users or policy makers - it is very obvious when these are a last-minute addition to the proposal - ensure they are meaningfully embedded across all stages of your research - the research journey begins before proposals are developed, so avoid indicating you ‘will’ engage consumers: consumers should have informed the proposal and not be engaged only once funding is allocated - ensure your methods align with best practice principles in consumer engagement.
Capacity, capability and resources: the best team and environment
leadership of the team - why is the named CIA the best one to lead the team? - what is the collaborative leadership capacity of the team? - how, and by who, will the CIA be supported? - what does strong leadership mean for your project
capacity building opportunities - what are the capacity building opportunities (e.g. is there a postdoc built into grant, is there a specific training program, etc)? - who will be mentored, by who and how? - what will be the mentee’s responsibilities in the project and how closely will they work with senior researchers and the CIA? - is there mentoring of juniors, partner organisations or consumer-researchers? - how much time do the named CIs and other senior researchers dedicate to genuine capacity building?
FTE and alignment with methods - has there been enough FTE allocated for CIs to work on this project, including time to upskill and mentor junior researchers? - is the upskilling, mentoring and capacity building feasible within the skillset? - do the CIs have sufficient time given their other commitments and responsibilities?
strengths of the combined team - does everyone have distinct, clearly defined roles and bring the necessary capacity? - is there a previous collaborative track record? - to leverage your project’s feasibility, have you included those with specific expertise such as health economics or biostatisticians?
involvement of consumers/end-users/community in research team - is the research identified by community groups and stakeholders as being needed? - how are consumers involved? (Think broader than just advisory committee meetings.) - is there a shared understanding of genuine, clearly defined leadership roles? - are those with lived experience participating as researchers? - is there a role for consumers/end-users in the project’s governance?
relative to opportunity and career disruptions - will you disadvantage your application by not claiming them?
partnership with organisations - are the meaningful partnerships clear in each section of your application? - have you articulated where the relationships are long-standing to demonstrate commitment?
Overall value and risk: the ‘what if’ scenarios
According to webinar panellists, the assessment of this section involves taking a broad view to determine the ways applicants propose to manage outcomes, risks and the budget in relation to the overall project.
Ensure you write to the overall value and risk category descriptors and provide contingency plans where necessary that will kick in if things go wrong. A panellist on the MRFF webinar stated comments from assessment panels are commonly ‘…OK, great project, great initiative, great recruitment strategy. But look, they haven't really thought about if something goes wrong.’ Not adequately discussing potential risks in this non-weighted section could reduce the overall score of your otherwise well-argued application.
An entwined budget
Demonstrating value for money involves strongly justifying your budget. For example, if you don’t need the full budget and full timeline available to you, then don’t stretch out your project. A more contained project over a shorter period will also likely require a smaller CI team. The alignment of a smaller budget with appropriate methodology still speaks loudly to feasibility and the overall value and risk assessment. Another example: if your project budget is 90% salary costs and 10% project costs, ensure your justification clearly articulates why the salary costs are essential to the research.
Also consider if your budget is:
enough to support successful delivery of your aims
sufficiently detailed and justified
representing value for money to the MRFF
including appropriate costs to enable meaningful consumer engagement.
Embed meaningful involvement of consumers/end-users throughout the research journey
The lack of meaningful involvement of consumers throughout applications is very noticeable to assessors. Think about this phrase, commonly employed in the inclusive research domain: ‘nothing about us without us’. Consumers and consumer organisations can give strength and added value to your project application. Assessors will look for easily understood plain language applications (indicative of consumer involvement in writing applications), strong consumer engagement during conceptualisation that informed and prioritised the research questions and co-designed the overall project, and consumer/end-user involvement in governance.
Demonstrating consumer engagement before and during conceptualisation of your research project could be coupled with results from targeted consumer surveys or results from focus groups demonstrating the importance of consumers/end-users on the research focus. Alternatively, it could be informed by meaningful roles for consumers as researchers on the project. Consumer input could also support an argument about cost benefit and risk, e.g., identifying where novel research involves a considered risk, determined by consumers as risk at an acceptable level given the potential social, economic, or community outcomes.
What is the role of consumers on GACs?
Consumers on GACs bring unique perspectives to the consideration of applications in a way that other assessors may not and might therefore provide exclusive insight to assessment panels as well as inform value for money. For instance, consumer input may be focused on the impact of data collection on participants, appropriate engagement during the early prioritisation of questions, potential (non-clinical) side effects in relation to clinical trials, useability or accessibility of written interventions for population subgroups, among other aspects.
It is important to remember we are all potential consumers of MRFF-funded research. Consumers on GACs may include retired professionals or semi-professionals, individuals from disadvantaged backgrounds, representatives of different cultures, working individuals, and/or those with lived experience of the condition of interest, amongst other inextricably entwined perspectives.
General points
Grantspersonship. It is critical to present a polished application to avoid risking your science due to a poorly constructed argument. Enlist your colleagues, particularly those who don’t know your work or work in your field, to review and comment constructively (and honestly) on your application. Remember, panel members may not be experts in your area, so write for a general audience and avoid jargon.
Think about your assessor. Assessors may likely review grants at the end of the day, potentially when they’re tired. Try to ‘…take the reader by the hand and lead them through your grant. Lead them through your story.’
Write to your category descriptors. During the MRFF webinar, it was suggested applicants have the category descriptors in front of them as they write the proposal. This is because the independent chair is going to bring the panellists and assessors back to those category descriptors during the panel discussion. Self-assess where you are on the assessment matrix; the GAC will focus on the matrix to ensure meaningful discussions around individual projects.
Be involved in an MRFF GAC panel. By contributing to MRFF via a GAC you will attain a greater understanding of the assessment and discussion process including ‘what scores well’ and ‘what to avoid’, thereby leveraging your capacity to self-determine if your own applications align with the category 7 descriptors.
What’s the take home message from the webinar?
Although not all questions raised by the webinar panel are relevant to each application, it’s obvious assessors value richly detailed applications in easily accessible language that explain the ‘why’ and the ‘how’ of the research. Assessors use non-weighted sections during the assessment process, and so will be looking for strong alignment between sections. We strongly encourage you to consider these summarised points from the webinar panel in conjunction with the scheme-specific guidelines.
If you’re interested in watching the MRFF assessment webinar, it can be found here Videos and webinars | Australian Government Department of Health and Aged Care.
Good luck with your MRFF funding applications!
Comments