Samaya
An organization focused on research into financial AI forecasting, working to build a forecasting framework specialized for finance and to evaluate the capabilities of frontier AI models on financial prediction tasks.
Timeline (last 90 days)
The study's test covered 456 companies, spanning all industry sectors, all with market capitalizations over $5 billion and with at least 8 brokerages providing forecasts.
In the ablation study, when no data is used and only parametric memory is relied on, even the best model performs poorly; after providing early-quarter data, performance improves by about 25 percentage points; but there is still a 12-16 percentage point gap from using the latest data.
After adding expert-guided instructions, the models' search time and context usage increase by about 1.6 to 2.7 times, and forecast error decreases accordingly.
Models perform relatively better on revenue and gross margin than on operating profit and adjusted EPS.
After running five times each on 100 companies, it was found that although there is variation between individual runs, averaging across multiple runs did not bring significant improvement.
The Samaya team says the alpha version of earnings forecasting is already available in the product for users and customers.
All Facts (6)
The study's test covered 456 companies, spanning all industry sectors, all with market capitalizations over $5 billion and with at least 8 brokerages providing forecasts.
50%UnverifiedIn the ablation study, when no data is used and only parametric memory is relied on, even the best model performs poorly; after providing early-quarter data, performance improves by about 25 percentage points; but there is still a 12-16 percentage point gap from using the latest data.
50%UnverifiedAfter adding expert-guided instructions, the models' search time and context usage increase by about 1.6 to 2.7 times, and forecast error decreases accordingly.
50%UnverifiedModels perform relatively better on revenue and gross margin than on operating profit and adjusted EPS.
50%UnverifiedAfter running five times each on 100 companies, it was found that although there is variation between individual runs, averaging across multiple runs did not bring significant improvement.
50%UnverifiedThe Samaya team says the alpha version of earnings forecasting is already available in the product for users and customers.
50%