\textit{Background and Context:} Empirical researchers have a long-standing tradition of explicitly discussing the threats to and limitations of their research. In the past twenty years, these discussions emerged as a standard component of empirical research papers in computing education research (CER) as well.

\textit{Objective:} Our goal was to find out how the CER community talks about threats and limitations (which we refer to collectively as challenges'') and how theyrespond'' to these challenges.

\textit{Method}: Our dataset included a total of 77 papers from four venues: two CER journals, \textit{Computer Science Education} (CSEJ) and \textit{ACM Transactions of Computing Education} (TOCE); one CER conference, \textit{ACM Conference on International Computing Education Research} (ICER); and, for comparison, one mathematics-education journal, \textit{The Journal for Research in Mathematics Education} (JRME). We analyzed the discussions of threats and limitations in these papers using deductive codes drawn from the literature, while also being open to new codes that emerged from the data. We took the papers on their own terms, so where qualitative and quantitative papers discuss challenges differently, we report on both.

\textit{Findings}: We found that the majority of these papers—65 out of 77—did discuss challenges. Most of the challenges that were reported were related to study design, but there also were some related to study implementation, analysis, and interpretation of the results. Depending on the research methodology employed, authors reported challenges that we could map to either internal and external validity, construct validity, and statistical conclusions or to trustworthiness criteria. Almost none of the challenges we found were unique to CER, but were also found in JRME.

We report on the range of challenges found in our dataset, the concrete circumstances that give rise to them, and the responses made to them. Our contributions include describing the broad range of challenges that are discussed, connecting those challenges to concrete circumstances in CER, and describing the range of responses made.

\textit{Implications}: Most immediately, we hope that this paper will broaden the perspective of those sitting down to write a threats and limitations section. It may also be of use at the study-design phase, since some of the challenges can be avoided with the benefit of hindsight. And finally, we hope to start a conversation about the challenges to our research: which can be mitigated, when (and how) a convincing argument can be made that an apparent challenge is not a problem, and which challenges might warrant serious revisions before submission (by the author) or rejection (by reviewers).