Image
Top
Navigation
March 17, 2014

Nodejs Streaming S3 Module

The ultimate nodeJS s3 data streaming module. Seriously!

After the release of the amazing google spreadsheet module comes another exciting release by the team at Mindblaze.

Lo and behold the Streaming S3 module that contains tons of exciting features for budding nodeJS programmers who would like to play with Amazon S3. We were not happy with what was out there so we made our own!

Source

Head-on over to GitHub at http://mindblaze.github.io/streaming-s3/

Benefits & Features

  • Super fast and super easy to use
  • Low memory usage
  • Nothing is written to disk during upload
  • Parallel part uploading
  • No need to know total size of the object
  • Implicit retry mechanism for failed part uploads
  • Tons of configurable options
  • Simple interface (Asynchronous and evented)
  • Downloading and uploading statistics (U/D speed and U/D time)
  • Proper usage of streams and graceful error handling
  • Production ready (Used and tested on production environments, uploading gigabytes of files to S3)
  • Uses official AWS SDK

Defaults and Configurables

  • concurrentParts (Default: 5) – Parts that are uploaded simultaneously.
  • waitTime (Default: 1 min (60000 ms)) – Time to wait for verification from S3 after uploading parts.
  • retries (Default: 5) – Number of times to retry uploading a part, before failing.
  • maxPartSize (Default: 5 MB) – Maximum size of each part.

Example 1: Uploading local file with callback.

Example 2: Uploading remote file without callback and options

 

Comments

  1. Hitu

    Hello

    I am using above cod.e, but it gives socket hangup error while doing multiple request same time

  2. Please visit http://mindblaze.github.io/streaming-s3/ and file a bug there. We will investigate and patch if needed.

  3. Thank you very much for posting and sharing this great Blog..

Submit a Comment

Posted By

Categories

Node.js, Tech