Master's thesis; Examining Large Language Models for Code Generation

We are looking for a dedicated master’s student to join us in the Connected Intelligence Unit at RISE.

The Connected Intelligence Unit is part of RISE Computer Science in Kista. The current research focus is on devising intelligent autonomous systems for controlling and allocating resources in future computer and communication networks. Among the group's key technologies are the Internet of Things (IoT) and Edge computing. The unit conducts projects together with industry and academic partners from Sweden and across the world.

Background and Purpose
Large language models (LLMs) are trained on vast corpora of text from diverse sources, enabling them to learn patterns, relationships, and context in different languages (including programming languages). LLMs are typically trained as general-purpose models, and they have a large memory footprint. Consequently, performing model inference requires multiple GPUs with large memory, making them less deployable on consumer-grade hardware.

Thesis Description
This thesis aims to facilitate the deployment of LLMs on consumer-grade hardware by building specialized & smaller models. The goal is to (i) examine existing LLMs, (ii) measure their performance for various codegeneration tasks, and (iii) potentially build specialized smaller models for each task via pruning techniques. You will deploy & profile one or more open-source LLMs and analyze the correlation between model weights and the requested prompts to find pruning opportunities to reduce the size of the original model for specific tasks.
Terms:
Start Time: As soon as possible
Scope: 30 hp
Location: RISE Computer Science, Kista, Stockholm. Option to partially work remotely.

Who are you?
We expect you to have a solid knowledge of machine learning theory, good programming skills (especially Python
and C; CUDA programming is a plus), and an interest in computer systems and solving complex problems.

We are looking forward to receiving your application!
To know more, please contact Dejan Kostic (dejan.kostic@ri.se). The thesis will be conducted together with the KTH Networked System Laboratory. Applications should include a brief personal letter, CV, recent grades, and a code excerpt. Candidates are encouraged to send in their application as soon as possible but at the latest by the 15th of January 2025. Suitable applicants will be interviewed as soon as applications are received.

Master thesis, Large Language Models, Systems, Code Generation, Model Pruning, RISE, Stockholm