Get the latest tech news

UIT – performant, modular, low-memory file processing at scale, in the Cloud


Library for performant, modular, low-memory file processing at scale, in the Cloud - janwilmake/uit

Thus, as features and complexity grew the need was born to create a more modular extensible architecture with good serverless practices in mind. Although FormData does not support stream-reading directly from Request and other Web Standards yet, UIT leverages the fact that intermediate results can be read using the Streams API using multipart-formdata-stream-js. uithub.ingestzip- Ingests and processes ZIP files into normalized formdata format uithub.merge- Combines multiple formdata streams into a single unified stream uithub.outputmd- Transforms and outputs data as markdown files uithub.outputzip- Packages processed data into downloadable ZIP archives uithub.search- Provides search capabilities across file hierarchies uithub.ziptree- Highly performant zip file-hierarchy extractor uithub.otp- Source proxy that generates an OTP to minimize secret exposure to other modules.

Get the Android app

Or read this on Hacker News

Read more on:

Photo of cloud

cloud

Photo of scale

scale

Related news:

News photo

OpenAI wants its ‘open’ AI model to call models in the cloud for help

News photo

SCALE 1.3 Adds BFloat16 & Other New Features For Compiling CUDA Apps On AMD GPUs

News photo

Electric trains in California cut 89% of toxic air pollution, study surprises | What made this transition unique was not just its scale, but its speed—and the immediate impact it had on air quality.