How we write rebuttals

  1. The reviewers, who have read your paper (to varying degrees), but may have forgotten some of the details or didn’t understand them in the first place.
  2. The AC, who is likely even less familiar with your work, and a good guiding principle is to assume that all they will read is the set of reviews and the rebuttal.
  1. For the reviewers: clarify doubts, answer questions, correct misunderstandings, push back on mischaracterizations, and make a good-faith effort to incorporate feedback and improve your work.
  2. For the AC: convince them that you have made a good-faith effort, present a representative summary of the reviews, help them understand if the reviewer concerns were addressed, call out bad-faith reviewing, and ultimately, help them make a decision.
  1. Itemize reviewer comments. We use a handy spreadsheet to organize individual comments/questions/concerns raised by each reviewer. Laying everything out in the same place helps identify common concerns and keeps us from missing anything accidentally. Do this ASAP to identify any necessary new experiments (if the venue allows it) or analysis early.
  2. Brain dump possible responses. The sheet has a column for each author to reply to each reviewer comment. Dump your thoughts here in rough text without regard for style or length. Being convincing and concise is a subtractive process.
  3. Write a draft rebuttal. Turn your consensus in the sheet into concrete responses in a rebuttal draft. Write concisely but don’t worry about space. Cover every point and trim / prioritize them later.
  4. Review and revise. Reread the initial reviews and your sheet to make sure everything has been addressed. Prioritize major concerns and start working towards meeting space limitations.
  1. “Are these averaged across multiple runs?”:
    Yes, we averaged across 5 random seeds.”
  2. “Are the segmentation masks used during training?”:
    No, they are only used to evaluate our results.”
  3. “So overall, the proposed approach needs more human annotations than the baseline.”:
    Not quite. While the first few iterations of our approach need a human in the loop more often, as we collect more data, our approach relies on humans less than the baseline. For typical dataset sizes, we use fewer annotations overall. E.g., for GNART20 with 10k images, our approach uses 2.4k human annotations while the baseline uses 2.8k”
  4. “Does your baseline match the one reported by Smith et al. at CVPR last year [43]?”:
    Almost. The is a 0.1% difference in performance between [43]’s publicly available code and what is reported in [43]. Our baseline matches the former.”
  5. “Did you evaluate on realistic environments?”:
    We disagree with the question’s premise. While these environments are simulated, they are highly photorealistic.”
  6. “Why did you not compare to GMAP?“
    GMAP is prohibitively expensive in our setting. Our environments have a significantly larger state-space (10k) than BRIE (20) which has been used to evaluate GMAP thus far. Back-of-the-envelope calculations suggest it would take 128 GPUS for 3 months to evaluate GMAP.”



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Devi Parikh

Devi Parikh

Research Director at Facebook AI Research. Associate Professor at Georgia Tech. Generative Artist. Artificial Intelligence.