Get the latest tech news
Transposing Tensor Files
Where does file metadata belong?
This article describes the safetensors file structure, points out its minor design flaws, and explains how changing the metadata location can address them. For example, png encodes various aspects of the image as separate chunks, and the WebAssembly binary format represents a module as a collection of sections (types, imports, memory, etc. This approach guarantees that the file will stay consistent even if the writing process crashes before completing the update, but it can lead to extra space usage.
Or read this on Hacker News