scholar-evaluation

scholar-evaluation provides structured, repeatable evaluation of scholarly work using a defined rubric. It solves the problem of giving academic and scientific writing consistent, evidence-based feedback rather than ad hoc impressions. It applies to reviewing research papers, proposals, thesis or dissertation chapters, and literature reviews; checking whether claims are supported by cited evidence; evaluating methodology, study design, analysis, and limitations; comparing multiple papers for quality or relevance; and producing structured feedback for revision.

The workflow starts by identifying the artifact type (empirical paper, theoretical paper, technical report, systematic or narrative literature review, research proposal, thesis chapter, or conference abstract) and choosing a scope: comprehensive (all rubric dimensions), targeted (one or two dimensions such as method or citations), or comparative (ranking multiple works). Each applicable dimension is scored 1 to 5, with N/A for dimensions that do not apply, across nine areas: Problem and Research Question, Literature and Context, Methodology, Data and Evidence, Analysis, Results and Interpretation, Limitations and Threats to Validity, Writing and Structure, and Citations. The review process reads the abstract, introduction, figures, and conclusion for the claimed contribution, then methods and results for evidence quality, checks the strongest claims against cited sources, scores each dimension, separates critical blockers from revision suggestions, and ends with concrete next edits. A Markdown output template captures an overall assessment (score, confidence, summary), a dimension score table with evidence and revision priority, and sections for critical issues, recommended revisions, and evidence checks needed.

It targets researchers, reviewers, advisors, and students who need rigorous, structured feedback. The documentation lists explicit pitfalls to avoid: do not use a score as a substitute for concrete feedback, do not penalize a paper for omitting a dimension outside its scope, do not treat citation count, venue, or author reputation as proof of quality, and do not accept unsupported claims just because they appear in the abstract.