• (External) Reviewer: an expert who gives an independent evaluation of a submitted paper
• TPC: Technical Program Chairs: People that oversee the entire review process in a specific year of the conference. They invite ACs, and have the final decision over which papers get accepted or rejected for the conference, based on reviews and AC scores across the entire set of papers.
• GC: General Chairs: People that oversee the entire conference. They invite all the co-chairs, such as the technical program chairs. Other co-chairs are for example the coordinators of the work-in-progress track, or of the video track.
• AC: Associate Chair: Sometimes referred to as meta-reviewer. A more experienced researcher from the field that oversees the review process of a specific set of papers (typically 2 to 5 papers), including inviting external reviewers. We distinguish 1AC and 2AC. Each AC typically serves in both roles on different papers.
• 1AC: An AC that acts as the primary coordinator of the review process of a submission. The TPC assign a 1AC to each paper. The 1AC invites at least 1 external reviewer to assess the paper, and they summarize the reviews. The TPC use the summary of the 1AC, and the underlying reviews, to decide whether to accept or reject a paper for presentation at the conference and associated publication.
• 2AC: Invites one external reviewer to review the paper and writes one review themselves (double-blind).
• Program committee: the set of ACs of a specific conference year.
• Discussion phase: phase during the review process after each reviewer has written an independent review. At this stage, reviewers can see each others’ reviews and scores. They can then discuss it using an internal discussion forum to overcome large differences. This helps the 1AC in reaching a consensus impression on each paper. Although the content of the discussion forum is not publicly visible, the 1AC summarizes the outcome in their meta-review.
• Reviewed process: a process in which submissions are reviewed by independent experts. For example, for AutomotiveUI the paper track is reviewed.
• Juried: a process in which submissions are assessed by a jury to decide who to invite to an opportunity and where other factors (e.g., geographic balance, gender balance) might also form part of the selection criteria. An example of a juried track is the Doctoral Colloquium.
• PCS: Precision conference system, the website used for submitting and reviewing work. See: https://new.precisionconference.com/user/login
• ACM: Association for Computing Machinery - parent organization of SIGCHI. See https://www.acm.org/
• SIGCHI: Special interest group on Computer-Human Interaction of ACM. SIGCHI organizes many conferences, of which AutomotiveUI is one. See https://sigchi.org/

You can volunteer to become a reviewer for AutomotiveUI 2021 via the precision conference system (note: links for various tracks will open gradually). To see which tracks are currently available to volunteer for, select for “Show only” the following: Society - SIGCHI; Conference/journal: AutomotiveUI 2021.
When you volunteer, we ask you to update your profile with the following information via various submenus here:

• Add example papers that you published in the past on this subpage: https://new.precisionconference.com/sample_pubs
• Add keywords relevant to your style of work. Note that typically each person has limited expertise, so do not write that you are an “expert” in all domains, but be selective. Starting with 2021, AutomotiveUI will have its own set of keywords. You can fill out information on this page: https://new.precisionconference.com/expertise under “society” select “SIGCHI” and under conference/journal select “AutomotiveUI of the year that you are reviewing (e.g., “AutomotiveUI 2021”)
• Add your website and other information in your profile here. Ideally, a website that gives an overview of your research experience (e.g., including overview of paper, past projects)

ACs use this information to select the best reviewers that match the work. Keywords and example papers are also used by PCS to semi-automatically suggest suitable reviewers. Update this to have the best matches to your expertise.

You will be invited by ACs via e-mail (automated message from PCS) to review a paper.

• Please respond as soon as possible to the invite, as we are on a strict time limit
• Decide whether a paper is indeed a match to your expertise. If it is not, let the AC know. If you have a suggestion for someone else that we might ask, we always welcome that information, but you are not required to.
• If you suspect you might have a conflict of interest, let the coordinating AC or the person who invited you know.
• Mark your calendar to allocate sufficient time to review the paper.

The general reviewing process of the Papers track is explained in the call of the papers track. Check the section “review process” here: https://www.auto-ui.org/21/authors/papers/

AutomotiveUI strives to accept the best scientific work that is relevant for the AutomotiveUI community. As a reviewer you give an assessment of the paper to inform the Associate Chair (AC), knowing that authors will also read your review. The AC uses your review and that of other reviewers to recommend acceptance of rejection of the paper to the Technical Program Chairs, who take the final decision.

Reviewers are asked to consider for each paper:

• Contribution to the field: Note that AutomotiveUI has a multi-disciplinary audience and that different methods are used. Different methods have different standards and expectations. Authors indicate what method / area their intended contribution is in. Judge the paper by the standards that hold for that specific method.
• Innovation: Does this submission present something new or something that has been done before but in an improved manner? Does it extend what we know?
  Each paper typically brings something new to the table. This does not mean that it needs to be “radically different”. Rather, the paper should build on preceding work and demonstrate what the added value is relative to existing work. As we have a multi-disciplinary community, innovation can take many forms.
• Quality of thought and writing: The write-up should be scientifically sound and valid, and understandable. This can include aspects such as:
  • Are the questions to be examined clearly stated?
  • Does the submission link to previous research to the questions examined? I.e., does it appropriately cite prior work, including relevant Auto-UI research?
  • If the research was experimental, was it rigorously and carefully done? Are there enough details so that someone else can repeat the evaluation (for instance use SAE J2944 definitions)?
  • Is the paper well written (e.g., no grammatical errors, references are formatted properly)? If writing quality is a concern, consider consulting a professional editor. (See acm.internationalscienceediting.com)
  • Paper length: is the contribution of the paper in accordance with its length?

You express your judgment in three ways:

1. Using a rating of the quality of the paper with options:
   •  Definite accept: I would argue strongly for accepting this paper.
   • Probably accept: I would argue for accepting this paper.
   • Borderline: Overall I would neither argue for accepting nor for rejecting this paper.
   • Probably reject: I would argue for rejecting this paper.
   • Definite reject: I would argue strongly for rejecting this paper.
2. Using a rating of your expertise on the domain and/or method at hand. This can be used to weigh your other assessments:
   • Expert
   • Knowledgeable
   • Passing Knowledge
   • No Knowledge
3. Through a qualitative review of the paper that summarizes its strengths and weaknesses, and explains your numerical score.

When reviewing, you can consider ACM SIGCHI’s criteria that reviews should be (a) high quality and (b) fair.

High quality implies that you systematically and clearly motivate your assessment. Even if you think a paper is great, you should motivate why, so the AC and TPC also understand this (not all might be familiar with this specific method or topic).

If you think a paper is weak, you should explain why. Keep in mind that authors are reading this, so your feedback can help them improve this line of work in the future. Especially when the first author is a (PhD) student, your constructive feedback can help them improve this line of work. Don’t just say that something is not good enough, but explain why and use sound argumentation.

Fair can be understood in multiple ways:

• Make sure you are in a position to judge the paper fairly. For example, you should have sufficient expertise on the topic/method, and you should take sufficient time to read and assess it carefully.
• Be constructive in your criticism. Even papers that you think should not be accepted have positives. Make sure to also mention those.
• Judge the paper “as is”, not what it could also be. As AutomotiveUI has contributions from different disciplines, different research methods are applied. Judge it mostly based on the criteria that are applicable to that method, not by the criteria of a different method (for example: don’t judge a qualitative user study by the criteria of a quantitative user study)

• If anything is unclear, or if you are in doubt: reach out to the 1AC or the AC that invited you as a reviewer
• If an AC is not responsive (and you have given sufficient time), reach out to the other AC, or to the technical program chair via papers@auto-ui.org
• Strive for a review that is of high quality and fair.
• Always motivate your judgment
• Always mention at least one positive about the work
• In your review explicitly separate “major” issues/limitations from “minor” issues (e.g., typos). Major issues are more structural / bigger limitations that you identified, and that would often need to be addressed by an author before the paper can be accepted. Note that AutomotiveUI does not have a rebuttal process where authors respond to reviews.
• Consider numbering your major issues, so the AC can easily refer to specific points from your review
• Acknowledge that sometimes the write-up of the work might not be perfect, but the actual work might be sufficient. The authors then need to express their message clearer.
• Once the meta-review is written, check whether the AC indeed understood what you meant. If they misinterpreted your review, clarify it with them. Note that ACs typically do not list all points from all reviewers. You can also use this time to calibrate your own reviewing skills: did you spot the same strengths and weaknesses as other reviewers?

AutomotiveUI is a conference organized by ACM SIGCHI. Some broader policies and tips can therefore be found on other sites. Note that not everything applies to AutomotiveUI:

• Notes on structure the larger decision cycle within AutomotiveUI (reviewer, conference chairs, steering group)
• ACM notes on submissions and review
• Notes from CHI 2021 on reviewing papers
• Notes from CHI 2021 on a successful submission
• Paper by Ken Hinckley on how to write a good review or meta-review (from Mobile HCI)