Get the latest tech news
S5cmd: Parallel S3 and local filesystem execution tool
Parallel S3 and local filesystem execution tool. Contribute to peak/s5cmd development by creating an account on GitHub.
s5cmd supports wide range of object management tasks both for cloud storage services and local filesystems. Thousands of S3 and filesystem commands are declared in a file (or simply piped in from another process) and they are executed using multiple parallel workers. For a more practical scenario, let's say we have an avocado prices dataset, and we would like to take a peek at the few lines of the data by fetching only the necessary bytes.
Or read this on Hacker News