SLM-CREATOR
A small language model trainer built from scratch that supports a fully hand-written training pipeline. It is developed using a mix of Python and Rust, with an upper limit of 100 million parameters.
Timeline (last 90 days)
SLM-CREATOR is developed with a mix of Python and Rust, with the parameter count capped at 100 million.
The developer chose to hand-write the entire training pipeline in order to truly understand the internal workings of the Transformer.
The developer only has a pure CPU machine on hand, so all GPU-related code paths have only been verified against simulated CLI output.
The script sets CUDA_DEVICE_ORDER=PCI_BUS_ID and then matches the device names reported by the vendor tools against the values returned by torch.cuda.get_device_name() to align device indices.
The performance test first runs about 12 seconds of warm-up to bring the GPU to a thermally stable state, then times a 4096×4096 matrix multiplication.
The developer hopes users with NVIDIA or AMD graphics cards will run the hardware detection script and post the output or errors to Issue #1 or the thread's comment section.
All Facts (6)
SLM-CREATOR is developed with a mix of Python and Rust, with the parameter count capped at 100 million.
50%UnverifiedThe developer chose to hand-write the entire training pipeline in order to truly understand the internal workings of the Transformer.
50%UnverifiedThe developer only has a pure CPU machine on hand, so all GPU-related code paths have only been verified against simulated CLI output.
50%UnverifiedThe script sets CUDA_DEVICE_ORDER=PCI_BUS_ID and then matches the device names reported by the vendor tools against the values returned by torch.cuda.get_device_name() to align device indices.
50%UnverifiedThe performance test first runs about 12 seconds of warm-up to bring the GPU to a thermally stable state, then times a 4096×4096 matrix multiplication.
50%UnverifiedThe developer hopes users with NVIDIA or AMD graphics cards will run the hardware detection script and post the output or errors to Issue #1 or the thread's comment section.
50%