Much of the recent discourse within the ML community has been centered around Large Language Models (LLMs), their functionality and potential --- yet not only do we not have a working definition of LLMs, but much of this discourse relies on claims and assumptions that are worth re-examining. We contribute a definition of LLMs, explicate some of the assumptions made regarding their functionality, and outline the existing evidence for and against these assumptions. We conclude with suggestions for future research directions and their framing.