A linear time-invariant system is one where the dependence of the output on the input is: linear: an input produces an output , and…
This note is a scratchpad for investigating the expressivity of the [ transformer ] architecture. In general, one set of intuitions that we…