Georgia Institute of Technology
A public research university in the United States with strong research capabilities in computer science and artificial intelligence.
Timeline (last 90 days)
Research from Georgia Tech found that in agent multi-turn conversation scenarios, the history must be re-tokenized on the CPU every turn, while the GPU's burden actually shrinks because it has cached the content it has already seen.
The research found that, given sufficient CPU resources, the time from sending a request to receiving the first token (TTFT) was shortened by 1.47 to 7.11 times across the various configurations tested.
The Georgia Tech research found that even when the work was split into a separate process and pre-recorded instruction sequences were fed to the chip, the GPU was still undersupplied, indicating that the GIL cannot be the whole reason.
Transformer Explainer is an interactive tool developed by a team at Georgia Tech that uses GPT-2 as a basis to demonstrate the computation flow of each Transformer layer.
Jill Watson is an AI teaching assistant developed in 2016 by Professor Ashok Goel's team at Georgia Tech, initially deployed in the forums of an online master's degree course to automatically answer students' common questions about course syllabi, assignment deadlines, and similar topics
All Facts (5)
Transformer Explainer is an interactive tool developed by a team at Georgia Tech that uses GPT-2 as a basis to demonstrate the computation flow of each Transformer layer.
60%UnverifiedResearch from Georgia Tech found that in agent multi-turn conversation scenarios, the history must be re-tokenized on the CPU every turn, while the GPU's burden actually shrinks because it has cached the content it has already seen.
50%UnverifiedThe research found that, given sufficient CPU resources, the time from sending a request to receiving the first token (TTFT) was shortened by 1.47 to 7.11 times across the various configurations tested.
50%UnverifiedThe Georgia Tech research found that even when the work was split into a separate process and pre-recorded instruction sequences were fed to the chip, the GPU was still undersupplied, indicating that the GIL cannot be the whole reason.
50%UnverifiedJill Watson is an AI teaching assistant developed in 2016 by Professor Ashok Goel's team at Georgia Tech, initially deployed in the forums of an online master's degree course to automatically answer students' common questions about course syllabi, assignment deadlines, and similar topics
50%