Soup can fine-tune an 8-billion-parameter model on a 4GB laptop GPU by streaming layers and training small adapters. The project reports about 2.4GB peak GPU memory in its laptop test. The full model still needs space in the computer’s main memory or storage. Transformer figure credit: Vaswani et al., Attention Is All You Need (2017), Google. #AI #Soup #LocalAI