TorchCodec
An official media decoding/encoding library from PyTorch that unifies the I/O entry points for images, video, and audio. It relies on frameworks such as FFmpeg at its lower layers, supports GPU decoding, works with DataLoader multiprocessing, and minimizes data copies between Python and C++.
Timeline (last 90 days)
PyTorch officially recommends using TorchCodec when media (images, video, audio) needs to be decoded or encoded.
TorchCodec does not implement codecs from scratch at the underlying layer; instead, it relies on mature multimedia frameworks such as FFmpeg to perform demultiplexing and decoding, and then feeds the results into PyTorch tensors through custom operators.
TorchCodec supports placing decoding work on the GPU and using it together with DataLoader's multi-process workers to alleviate the CPU bottleneck common in video decoding.
TorchCodec aims to converge the decoding and encoding of images, video, and audio into a single library, reducing fragmented dependencies on multiple low-level bindings.
TorchCodec handles compatibility with different versions of underlying libraries, and compared with directly calling PyAV or FFmpeg's Python bindings, it provides an API closer to PyTorch usage conventions.
If code uses TorchVision to read video files directly, it may in the future need to migrate to TorchCodec for the I/O portion.
Separating the responsibilities of decoding/encoding from transforms allows the underlying codec dependencies and the upper-layer data augmentation logic to evolve independently, avoiding one library bearing two different sets of engineering burdens at the same time.
All Facts (7)
TorchCodec aims to converge the decoding and encoding of images, video, and audio into a single library, reducing fragmented dependencies on multiple low-level bindings.
50%UnverifiedTorchCodec does not implement codecs from scratch at the underlying layer; instead, it relies on mature multimedia frameworks such as FFmpeg to perform demultiplexing and decoding, and then feeds the results into PyTorch tensors through custom operators.
50%UnverifiedTorchCodec supports placing decoding work on the GPU and using it together with DataLoader's multi-process workers to alleviate the CPU bottleneck common in video decoding.
50%UnverifiedPyTorch officially recommends using TorchCodec when media (images, video, audio) needs to be decoded or encoded.
50%UnverifiedTorchCodec handles compatibility with different versions of underlying libraries, and compared with directly calling PyAV or FFmpeg's Python bindings, it provides an API closer to PyTorch usage conventions.
50%UnverifiedSeparating the responsibilities of decoding/encoding from transforms allows the underlying codec dependencies and the upper-layer data augmentation logic to evolve independently, avoiding one library bearing two different sets of engineering burdens at the same time.
50%UnverifiedIf code uses TorchVision to read video files directly, it may in the future need to migrate to TorchCodec for the I/O portion.
50%