An inside look at our scientific review process

May 29, 2014

Next week we’ll announce our 2014 Childhood Cancer Research Grants, and I could not be more thrilled about this year’s scientific bounty. The period of time between our application deadline in early March and our big reveal in June represents hours upon hours of work by our Scientific Advisory Committee, culminating in a marathon conference call to determine final recommendations for our Board.

We held that call yesterday – May 28 – and our Board voted a few hours later on the bright minds that Pablove will back in 2014. My to-do list over the next few days involves lots of boring but incredibly essential paperwork as we set up stringent funding conditions and reporting requirements with each grantee.

To break up that task, I’m taking some time to share an insider’s view on our scientific review process and what went down over the past few weeks.

Step One: Reading. Lots and lots of reading.

When our grant deadline closed in March, our Scientific Advisory Chair, Dr. Leo Mascarenhas, surveyed the record-setting field of 28 applications to determine subject-matter, research institution, and the investigator’s academic and clinical background. Here’s an overview of the subjects covered in this year’s batch of proposals:

Our pool of applications was incredibly diverse – spanning three continents, five countries, and 13 states. Proposals focused on everything from biological investigations to clinical treatments to public health and statistical outcomes. Dr. Mascarenhas was faced with the tall task of assigning multiple readers to evaluate each application – good thing our committee is comprised of accomplished physicians and scientists with a wide range of expertise. Occasionally, we bring in experts outside of our committee if an application covers an area our members can’t fully evaluate. (We did that this year in order to review the proposals submitted to investigate Opsoclonus-Myoclonus Syndrome through our designated OMS Fund.) Our committee members spend many hours carefully reading, re-reading, and scoring each application.

Step Two: Fun with Excel.

This would not be science if there was no number crunching at the end of the rainbow. The Pablove Foundation’s funding priorities, in order of importance, are the quality of the science, the project’s ability to address a crucial need, the investigator’s status (we prefer to fund young researchers to encourage careers dedicated to pediatric cancer research, or established investigators trying to get a brand-new idea off the ground), and finally, the application’s focus on a rare childhood cancer. Our reviewers score each application based on five key factors:

      • Significance – Is this project important? Will we learn something valuable or answer a critical question?
      • Background – How strong is the lead researcher’s academic background? Do they have the “right stuff” to see a hypothesis through from ideation to actual results?
      • Innovation – Where does this project fall on the scale of same-old-thing to lightbulbs going off in someone’s head?
      • Approach – Is this a smart, precise plan? Does the project have access to the samples or data necessary to be successful? Is it feasible?
      • Environment – Will the research be conducted in an excellent institution? Is the investigator surrounded by strong leaders who can serve as mentors?

Each of these five factors is rated on a scale of one to nine, with a score of one meaning the grant is exceptional, and a score of nine meaning the project is poorly conceived. This helps each reviewer determine a final overall score. All of this data is then sent to Pablove HQ where we combine the scores to rank the applications.

You’d think this would narrow things down.

It does…to a point.

This year, out of our 28 applications, all but one of them received an overall score of very good or higher. That means 27 of the applications received a score of four or above. This does not mean that our reviewers are lenient. It means the ideas that come across our desks are amazing. Really amazing.

And that is where the marathon conference call comes in.

Step Three: Fourteen calendars and lots of coffee.

At the end of this process, our Scientific Advisory Committee graciously gives us a few more hours of their incredibly valuable time to determine their recommendations to our Board of Directors. (Coordinating a convenient time for everyone to step away from, you know…curing childhood cancer themselves is just as much fun as it sounds).

For me, syncing 14 calendars is totally worth it. I get to be a witness as several of the brightest minds in the pediatric cancer world deliberate science, progress, and impact Pablove is about to make. This is not your average conference call. The enthusiasm streams out of the phone line as we discuss each project.

First, the committee tackles any project where reviewers’ scores differed by more than two points. Having multiple opinions provides for checks and balances, but it is important to come to a final consensus, otherwise the ranking system falls apart. Anyone with a potential conflict of interest abstains from discussions when necessary. For example: Dr. Mascarenhas does not participate in discussions on proposals from the institution where he works. Another of one of our committee members served as a mentor to one of the applicants, so his lips were sealed for that discussion.

Once the committee makes a final decision on any contradictory scores, we move on to a healthy debate on the proposals that were ranked as very good or higher (see above).

Now you see where the coffee comes in.

This final hurdle is the most crucial aspect of the scientific review process. The questions asked, arguments raised, and accolades shared allow the entire group to ensure only the very best research gets funded.

As a donor, a staff member, and a person who has been affected by cancer, I wouldn’t have it any other way.

The committee’s top recommendations are then presented to our Board of Directors for an exciting and final vote on our annual awards.*

And that brings us to now. The days where you watch the calendar, and I process contracts, and we all anxiously await the day where we can share the news of our 2014 Childhood Cancer Research Grants. That day is next Wednesday, June 4. Stay tuned.

– Megan McMillan, Community Affairs Director

*You’re probably asking yourself, “why doesn’t the Board just vote to fund everything if the science is so awesome?” The answer is: we’d love to. That’s where your donations come in. If my calculus classes taught me anything, I know that more donations = more research funded.**

**Calculus was brutal. I learned a little bit more than that…but not by much.

Here’s a breakdown of where our applications came from in 2014:

Comments

Leave a Reply

Your email address will not be published.