Somo
An AI math tutoring application that photographs and recognizes a student's handwritten solution steps, marks the first incorrect line with a red circle, and provides explanations in multiple languages along with practice problems from past exams.
Timeline (last 90 days)
Somo is a math tutoring app that takes a photo upload of a handwritten solution process, uses AI to find the first line in the handwritten solution where an error occurs, circles it in red like a teacher, and explains it in English, Swahili, or Spanish.
Somo uses a five-step process: photograph, diagnose, confirm, teach, check; in the diagnosis step, Gemini recognizes the handwritten content and returns structured JSON containing the transcribed text, the tested concept, the first erroneous line, and the reason for the error.
The Somo system makes a separate model call to re-solve the problem without seeing the diagnosis result, and when the two answers disagree it prompts the user that they may have misread something.
After Gemini's free quota ran out midway through development, the author consolidated all calls into one function: try Gemini first, retry once on 503/429, then fall back to Claude, and added a circuit breaker — skipping Gemini for 60 seconds after a quota error.
After adding the circuit breaker and fallback mechanism, diagnosis time dropped from about 116 seconds to about 10 seconds.
The pipeline now clearly labels every problem as a past exam question, an online practice problem, or a Somo-authored problem, and off-topic problems are rejected.
On the three handwritten problems the author tested himself, Somo accurately found the erroneous line in each: one division-by-2 error, one missing negative sign when transposing terms, and one where the two sides of the equation were multiplied by different numbers; the author specifically notes that no real students have used it and that all test problems were written by himself.
Somo's backend uses FastAPI plus Server-Sent Events for streaming output, and the frontend is a single-file mobile interface.
gemini-flash-latest handles the multimodal recognition task, using structured output, with the diagnosis result validated directly against a Pydantic schema; error_step is a real field, and the interface uses it to know which line to circle; Gemini is also responsible for streaming explanations and generating check questions.
Nimble searches the live web for past exam papers on that concept, prioritizing official sites; US students get the NY Regents question bank, and when official sites yield no results it searches the open web; Nimble correctly found the Tanzania Examinations Council's official mathematics papers and examiner reports from 2017 to 2023.
3 more timeline events
All Facts (13)
Somo is a math tutoring app that takes a photo upload of a handwritten solution process, uses AI to find the first line in the handwritten solution where an error occurs, circles it in red like a teacher, and explains it in English, Swahili, or Spanish.
50%UnverifiedSomo uses a five-step process: photograph, diagnose, confirm, teach, check; in the diagnosis step, Gemini recognizes the handwritten content and returns structured JSON containing the transcribed text, the tested concept, the first erroneous line, and the reason for the error.
50%UnverifiedThe Somo system makes a separate model call to re-solve the problem without seeing the diagnosis result, and when the two answers disagree it prompts the user that they may have misread something.
50%UnverifiedAfter Gemini's free quota ran out midway through development, the author consolidated all calls into one function: try Gemini first, retry once on 503/429, then fall back to Claude, and added a circuit breaker — skipping Gemini for 60 seconds after a quota error.
50%UnverifiedAfter adding the circuit breaker and fallback mechanism, diagnosis time dropped from about 116 seconds to about 10 seconds.
50%UnverifiedThe pipeline now clearly labels every problem as a past exam question, an online practice problem, or a Somo-authored problem, and off-topic problems are rejected.
50%UnverifiedOn the three handwritten problems the author tested himself, Somo accurately found the erroneous line in each: one division-by-2 error, one missing negative sign when transposing terms, and one where the two sides of the equation were multiplied by different numbers; the author specifically notes that no real students have used it and that all test problems were written by himself.
50%UnverifiedSomo's backend uses FastAPI plus Server-Sent Events for streaming output, and the frontend is a single-file mobile interface.
50%Unverifiedgemini-flash-latest handles the multimodal recognition task, using structured output, with the diagnosis result validated directly against a Pydantic schema; error_step is a real field, and the interface uses it to know which line to circle; Gemini is also responsible for streaming explanations and generating check questions.
50%UnverifiedNimble searches the live web for past exam papers on that concept, prioritizing official sites; US students get the NY Regents question bank, and when official sites yield no results it searches the open web; Nimble correctly found the Tanzania Examinations Council's official mathematics papers and examiner reports from 2017 to 2023.
50%UnverifiedPDF-type results return only a summary of about 300–700 characters, which contains no problems; Somo treats Nimble as a discoverer: when it encounters short PDF results, it downloads the files and uses pypdf to extract up to 20,000 characters (processing five papers in parallel), then Gemini picks out a problem whose original text appears verbatim and which tests the same concept.
50%UnverifiedThe first real result came from a matrix application problem in the NECTA CSEE 2023 Basic Mathematics paper, displayed with the exam stamp and a source link; results are cached to disk by fuzzy topic matching, and each topic is searched only once.
50%UnverifiedThe Somo project was born at a one-day MLH hackathon; the author had no team and no idea, the deadline was 5 PM, and he froze the features at 3:45.
50%