This talk unveils the transformative potential of achieving sub-10-watt language models (LMs) by drawing inspiration from the brain’s energy efficiency. We introduce a groundbreaking approach to language model design, featuring a matrix-multiplication-free architecture that scales to billions of parameters. To validate this paradigm, we developed custom hardware solutions (FPGA) as well as leveraged pre-existing neuromorphic... Read more →