Composite measures and indices are used in medical research to represent certain concepts that cannot be measured with one variable. They can be used to predict outcomes or serve as outcomes in trials. The creation of innovative indices is important to increase publications and secure research funding. However, some assumptions and problems are prevalent among indices. We aim to develop a reporting guide and an appraisal tool for indices based on the issues we identified.

We reproduced the three frailty indices from a previous publication and 134,689 principal component-based indices. We reviewed the index assumptions, bias introduced by data processing, relationships between input variables. We interpreted the indices with input variables.

We identified four major issues to be addressed in a reporting guide: constraints imposed by index creation on the input variables; data processing without evidence base; indices poorly linked to input variables; and, relatively inferior predictive power. We demonstrated a flow diagram and a checklist to report and review these four issues related to innovative indices.

A reporting and critical appraisal tool for innovative indices is lacking and needed. These four issues that need to be explicitly considered are previously neglected. This guide is the first attempt to improve the quality and generalizability of innovative indices. This guide can be used to lead further discussion with other experts and review committees.