Large Language Models (LLM) has recently demonstrated strong capabilities in complextextual tasks including sentiment analysis, text summarization, and text classification.
These achievements would greatly reduce human workloads especially in time-consuming
tasks like summarization, which requires preserving core information and being concise at
the same time. However, most powerful summarization models today are built off large
language models with over a hundred billion parameters, and the training cost is beyond
what small businesses and individuals can afford. Thus in this paper, we explored an
alternative: small language models and their potential in summarization tasks.
We implemented and tested two fine-tuning pipelines that can be supported by a
single NVIDIA 16GB RTX4090 GPU: instruction tuning with LoRA and reinforcement
learning with human feedback over a full spectrum of available small language models
on HuggingFace. For models that exceed memory capacity, we used quantization to
replicate these models with lower precision without losing much model performance.
The dataset used in fine-tuning is a combination of human-annotated CNN/Daily Mails
and a synthetic news dataset generated by GPT-4. We adopted a top-down prompting
techniques to ensure the diversity of the synthetic texts. We compared model performance
under zero-shot prompting, few-shots prompting and instruction tuning to conclude that
fine-tuning is able to improve small language model’s ability to summarize texts. We
found the best model hyper-parameters through cross-validation using ROUGE score as
ii
the evaluation metric and discovered that the T5-small 60M model by Google can achieve
similar performance as our bench-mark LLM GPT-3.5-Turbo on text summarization.
Despite limitations such as context window length, we found that small language
models can have similar performance to LLMs on specific tasks after proper fine-tuning.
Meanwhile, through experimentation, we also gained a deeper understanding on factors
that affect fine-tuning performance. In future research, we seek to apply our fine-tuning
pipelines on more complex tasks such as teaching small language models to perform simple
operations on smartphones and computers.