Scaling Sparse Fine-Tuning to Large Language Models
Published in arxiv preprint, 2024
We scale sparse fine-tuning methods for 7b and 13b Llama 2, such that the memory requirements scale linearly with the number of paramerters updated by the fine-tuning.
