Read contents of Node.js stream into a string variableconvert streamed buffers to utf8-stringNode.js - How to get stream into stringwrapping nodejs stream in JSON objectHow do I read / convert an InputStream into a String in Java?How to replace all occurrences of a string in JavaScriptHow to check whether a string contains a substring in JavaScript?How do I debug Node.js applications?Writing files in Node.jsCheck if a variable is a string in JavaScriptHow do I pass command line arguments to a Node.js program?Read environment variables in Node.jsHow to decide when to use Node.js?How to exit in Node.js

What to wear for invited talk in Canada

How would photo IDs work for shapeshifters?

Need help identifying/translating a plaque in Tangier, Morocco

What does 'script /dev/null' do?

Why is the design of haulage companies so “special”?

Information to fellow intern about hiring?

Could a US political party gain complete control over the government by removing checks & balances?

Landlord wants to switch my lease to a "Land contract" to "get back at the city"

Pristine Bit Checking

Why do we use polarized capacitors?

Is domain driven design an anti-SQL pattern?

Can I legally use front facing blue light in the UK?

Finding files for which a command fails

Why do UK politicians seemingly ignore opinion polls on Brexit?

Why did the Germans forbid the possession of pet pigeons in Rostov-on-Don in 1941?

Is it wise to hold on to stock that has plummeted and then stabilized?

Add an angle to a sphere

Is there any use for defining additional entity types in a SOQL FROM clause?

Why doesn't a const reference extend the life of a temporary object passed via a function?

What is the command to reset a PC without deleting any files

How can I fix this gap between bookcases I made?

Could Giant Ground Sloths have been a good pack animal for the ancient Mayans?

Is Social Media Science Fiction?

Is there a way to make member function NOT callable from constructor?



Read contents of Node.js stream into a string variable


convert streamed buffers to utf8-stringNode.js - How to get stream into stringwrapping nodejs stream in JSON objectHow do I read / convert an InputStream into a String in Java?How to replace all occurrences of a string in JavaScriptHow to check whether a string contains a substring in JavaScript?How do I debug Node.js applications?Writing files in Node.jsCheck if a variable is a string in JavaScriptHow do I pass command line arguments to a Node.js program?Read environment variables in Node.jsHow to decide when to use Node.js?How to exit in Node.js






.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty height:90px;width:728px;box-sizing:border-box;








64















I'm hacking on a node.js program that captures SMTP mails and acts on the mail data. The 'smtp-protocol' node library provides the mail data live as a stream, and as a node.js newbie I'm not sure how to write that stream into a string variable. I currently have it writing to stdout using the line:



stream.pipe(process.stdout, end : false );


As I said, I need to get this stream data writing to a string variable instead, which I will then use once the stream has ended.



Help much appreciated!










share|improve this question
























  • You should copy the stream or flag it with (autoClose: false). It is bad practice to pollute the memory.

    – Kenan Sulayman
    Jun 1 '13 at 17:42

















64















I'm hacking on a node.js program that captures SMTP mails and acts on the mail data. The 'smtp-protocol' node library provides the mail data live as a stream, and as a node.js newbie I'm not sure how to write that stream into a string variable. I currently have it writing to stdout using the line:



stream.pipe(process.stdout, end : false );


As I said, I need to get this stream data writing to a string variable instead, which I will then use once the stream has ended.



Help much appreciated!










share|improve this question
























  • You should copy the stream or flag it with (autoClose: false). It is bad practice to pollute the memory.

    – Kenan Sulayman
    Jun 1 '13 at 17:42













64












64








64


8






I'm hacking on a node.js program that captures SMTP mails and acts on the mail data. The 'smtp-protocol' node library provides the mail data live as a stream, and as a node.js newbie I'm not sure how to write that stream into a string variable. I currently have it writing to stdout using the line:



stream.pipe(process.stdout, end : false );


As I said, I need to get this stream data writing to a string variable instead, which I will then use once the stream has ended.



Help much appreciated!










share|improve this question
















I'm hacking on a node.js program that captures SMTP mails and acts on the mail data. The 'smtp-protocol' node library provides the mail data live as a stream, and as a node.js newbie I'm not sure how to write that stream into a string variable. I currently have it writing to stdout using the line:



stream.pipe(process.stdout, end : false );


As I said, I need to get this stream data writing to a string variable instead, which I will then use once the stream has ended.



Help much appreciated!







javascript node.js stream






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Mar 22 at 1:37









Dan Dascalescu

69.8k21208280




69.8k21208280










asked May 16 '12 at 17:42









obrienmdobrienmd

515258




515258












  • You should copy the stream or flag it with (autoClose: false). It is bad practice to pollute the memory.

    – Kenan Sulayman
    Jun 1 '13 at 17:42

















  • You should copy the stream or flag it with (autoClose: false). It is bad practice to pollute the memory.

    – Kenan Sulayman
    Jun 1 '13 at 17:42
















You should copy the stream or flag it with (autoClose: false). It is bad practice to pollute the memory.

– Kenan Sulayman
Jun 1 '13 at 17:42





You should copy the stream or flag it with (autoClose: false). It is bad practice to pollute the memory.

– Kenan Sulayman
Jun 1 '13 at 17:42












14 Answers
14






active

oldest

votes


















27














The key is to use these two Stream events:



  • Event: 'data'

  • Event: 'end'

For stream.on('data', ...) you should collect your data data into either a Buffer (if it is binary) or into a string.



For on('end', ...) you should call a callback with you completed buffer, or if you can inline it and use return using a Promises library.






share|improve this answer

























  • That worked perfectly. Thanks!

    – obrienmd
    May 16 '12 at 18:10






  • 112





    A couple of lines of code illustrating the answer is preferable to just pointing a link at the API. Don't disagree with the answer, just don't believe it is complete enough.

    – arcseldon
    Jun 4 '14 at 5:14






  • 2





    With newer node.js versions, this is cleaner: stackoverflow.com/a/35530615/271961

    – Simon A. Eugster
    Dec 7 '16 at 11:28











  • The answer should be updated to not recommend using a Promises library, but use native Promises.

    – Dan Dascalescu
    Mar 22 at 1:27











  • @DanDascalescu I agree with you. The problem is that I wrote this answer 7 years ago, and I haven't kept up with node.js . If you are someone else would like to update it, that would be great. Or I could simply delete it, as there seems to be a better answer already. What would you recommend?

    – ControlAltDel
    Mar 25 at 13:30


















51














Hope this is more useful than the above answer (which now has a broken link).



Also note that string concatenation is not an efficient way to collect the string parts, but it is used for simplicity (and perhaps your code does not care about efficiency)



var string = ''
stream.on('readable',function(buffer)
var part = buffer.read().toString();
string += part;
console.log('stream data ' + part);
);


stream.on('end',function()
console.log('final output ' + string);
);





share|improve this answer


















  • 3





    What would be a more efficient way to collect string parts? TY

    – sean2078
    Aug 27 '15 at 13:34






  • 2





    you could use a buffer docs.nodejitsu.com/articles/advanced/buffers/how-to-use-buffers but it really depends on your use.

    – Tom Carchrae
    Aug 27 '15 at 13:43






  • 2





    Use an array of strings where you append each new chunk to the array and call join("") on the array at the end.

    – Valeriu Paloş
    Mar 8 '16 at 7:22







  • 6





    This isn't right. If buffer is halfway through a multi-byte code point then the toString() will receive malformed utf-8 and you will end up with a bunch of � in your string.

    – alextgordon
    Oct 16 '16 at 22:03











  • i'm not sure what you mean by a 'mutli-byte code point' but if you want to convert the encoding of the stream you can pass an encoding parameter like this toString('utf8') - but by default string encoding is utf8 so i suspect that your stream may not be utf8 @alextgordon - see stackoverflow.com/questions/12121775/… for more

    – Tom Carchrae
    Oct 18 '16 at 13:41



















41














None of the above worked for me. I needed to use the Buffer object:



 const chunks = [];

readStream.on("data", function (chunk)
chunks.push(chunk);
);

// Send the buffer or you can put it into a var
readStream.on("end", function ()
res.send(Buffer.concat(chunks));
);





share|improve this answer


















  • 3





    this is actually the cleanest way of doing it ;)

    – Ivo
    Oct 28 '16 at 9:58











  • Clean, efficient and works well with binary data.

    – giosh94mhz
    Jul 28 '17 at 7:11






  • 2





    Works great. Just a note: if you want a proper string type, you will need to call .toString() on the resulting Buffer object from concat() call

    – Bryan Johnson
    Sep 25 '17 at 14:30


















24














Another way would be to convert the stream to a promise (refer to the example below) and use then (or await) to assign the resolved value to a variable.



function streamToString (stream) 
const chunks = []
return new Promise((resolve, reject) =>
stream.on('data', chunk => chunks.push(chunk))
stream.on('error', reject)
stream.on('end', () => resolve(Buffer.concat(chunks).toString('utf8')))
)


const result = await streamToString(stream)





share|improve this answer
































    17














    I'm using usually this simple function to transform a stream into a string:



    function streamToString(stream, cb) 
    const chunks = [];
    stream.on('data', (chunk) =>
    chunks.push(chunk.toString());
    );
    stream.on('end', () =>
    cb(chunks.join(''));
    );



    Usage example:



    let stream = fs.createReadStream('./myFile.foo');
    streamToString(stream, (data) =>
    console.log(data); // data is now my string variable
    );





    share|improve this answer

























    • Useful answer but it looks like each chunk must be converted to a string before it is pushed in the array: chunks.push(chunk.toString());

      – Nicolas Le Thierry d'Ennequin
      Jun 7 '16 at 10:06












    • Your are right! I Change it!

      – dreampulse
      Jun 13 '16 at 8:41











    • don t forget to bind error event!

      – mh-cbon
      Mar 13 '17 at 21:47


















    7














    From the nodejs documentation you should do this - always remember a string without knowing the encoding is just a bunch of bytes:



    var readable = getReadableStreamSomehow();
    readable.setEncoding('utf8');
    readable.on('data', function(chunk)
    assert.equal(typeof chunk, 'string');
    console.log('got %d characters of string data', chunk.length);
    )





    share|improve this answer






























      6














      Streams don't have a simple .toString() function (which I understand) nor something like a .toStringAsync(cb) function (which I don't understand).



      So I created my own helper function:



      var streamToString = function(stream, callback) 
      var str = '';
      stream.on('data', function(chunk)
      str += chunk;
      );
      stream.on('end', function()
      callback(str);
      );


      // how to use:
      streamToString(myStream, function(myStr)
      console.log(myStr);
      );





      share|improve this answer






























        3














        I had more luck using like that :



        let string = '';
        readstream
        .on('data', (buf) => string += buf.toString())
        .on('end', () => console.log(string));


        I use node v9.11.1 and the readstream is the response from a http.get callback.






        share|improve this answer






























          3














          Easy way with the popular (over 5m weekly downloads) and lightweight get-stream library:



          https://www.npmjs.com/package/get-stream



          const fs = require('fs');
          const getStream = require('get-stream');

          (async () =>
          const stream = fs.createReadStream('unicorn.txt');
          console.log(await getStream(stream)); //output is string
          )();





          share|improve this answer






























            2














            What about something like a stream reducer ?



            Here is an example using ES6 classes how to use one.



            var stream = require('stream')

            class StreamReducer extends stream.Writable
            constructor(chunkReducer, initialvalue, cb)
            super();
            this.reducer = chunkReducer;
            this.accumulator = initialvalue;
            this.cb = cb;

            _write(chunk, enc, next)
            this.accumulator = this.reducer(this.accumulator, chunk);
            next();

            end()
            this.cb(null, this.accumulator)



            // just a test stream
            class EmitterStream extends stream.Readable
            constructor(chunks)
            super();
            this.chunks = chunks;

            _read()
            this.chunks.forEach(function (chunk)
            this.push(chunk);
            .bind(this));
            this.push(null);



            // just transform the strings into buffer as we would get from fs stream or http request stream
            (new EmitterStream(
            ["hello ", "world !"]
            .map(function(str)
            return Buffer.from(str, 'utf8');
            )
            )).pipe(new StreamReducer(
            function (acc, v)
            acc.push(v);
            return acc;
            ,
            [],
            function(err, chunks)
            console.log(Buffer.concat(chunks).toString('utf8'));
            )
            );





            share|improve this answer






























              2














              The cleanest solution may be to use the "string-stream" package, which converts a stream to a string with a promise.



              const streamString = require('stream-string')

              streamString(myStream).then(string_variable =>
              // myStream was converted to a string, and that string is stored in string_variable
              console.log(string_variable)

              ).catch(err =>
              // myStream emitted an error event (err), so the promise from stream-string was rejected
              throw err
              )





              share|improve this answer






























                0














                This worked for me and is based on Node v6.7.0 docs:



                let output = '';
                stream.on('readable', function()
                let read = stream.read();
                if (read !== null)
                // New stream data is available
                output += read.toString();
                else
                // Stream is now finished when read is null.
                // You can callback here e.g.:
                callback(null, output);

                );

                stream.on('error', function(err)
                callback(err, null);
                )





                share|improve this answer






























                  0














                  Using the quite popular stream-buffers package which you probably already have in your project dependencies, this is pretty straightforward:



                  // imports
                  const WritableStreamBuffer = require('stream-buffers');
                  const promisify = require('util');
                  const createReadStream = require('fs');
                  const pipeline = promisify(require('stream').pipeline);

                  // sample stream
                  let stream = createReadStream('/etc/hosts');

                  // pipeline the stream into a buffer, and print the contents when done
                  let buf = new WritableStreamBuffer();
                  pipeline(stream, buf).then(() => console.log(buf.getContents().toString()));





                  share|improve this answer






























                    0














                    setEncoding('utf8');



                    Well done Sebastian J above.



                    I had the "buffer problem" with a few lines of test code I had, and added the encoding information and it solved it, see below.



                    Demonstrate the problem



                    software



                    // process.stdin.setEncoding('utf8');
                    process.stdin.on('data', (data) =>
                    console.log(typeof(data), data);
                    );


                    input



                    hello world


                    output



                    object <Buffer 68 65 6c 6c 6f 20 77 6f 72 6c 64 0d 0a>


                    Demonstrate the solution



                    software



                    process.stdin.setEncoding('utf8'); // <- Activate!
                    process.stdin.on('data', (data) =>
                    console.log(typeof(data), data);
                    );


                    input



                    hello world


                    output



                    string hello world





                    share|improve this answer























                      Your Answer






                      StackExchange.ifUsing("editor", function ()
                      StackExchange.using("externalEditor", function ()
                      StackExchange.using("snippets", function ()
                      StackExchange.snippets.init();
                      );
                      );
                      , "code-snippets");

                      StackExchange.ready(function()
                      var channelOptions =
                      tags: "".split(" "),
                      id: "1"
                      ;
                      initTagRenderer("".split(" "), "".split(" "), channelOptions);

                      StackExchange.using("externalEditor", function()
                      // Have to fire editor after snippets, if snippets enabled
                      if (StackExchange.settings.snippets.snippetsEnabled)
                      StackExchange.using("snippets", function()
                      createEditor();
                      );

                      else
                      createEditor();

                      );

                      function createEditor()
                      StackExchange.prepareEditor(
                      heartbeatType: 'answer',
                      autoActivateHeartbeat: false,
                      convertImagesToLinks: true,
                      noModals: true,
                      showLowRepImageUploadWarning: true,
                      reputationToPostImages: 10,
                      bindNavPrevention: true,
                      postfix: "",
                      imageUploader:
                      brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
                      contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
                      allowUrls: true
                      ,
                      onDemand: true,
                      discardSelector: ".discard-answer"
                      ,immediatelyShowMarkdownHelp:true
                      );



                      );













                      draft saved

                      draft discarded


















                      StackExchange.ready(
                      function ()
                      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f10623798%2fread-contents-of-node-js-stream-into-a-string-variable%23new-answer', 'question_page');

                      );

                      Post as a guest















                      Required, but never shown

























                      14 Answers
                      14






                      active

                      oldest

                      votes








                      14 Answers
                      14






                      active

                      oldest

                      votes









                      active

                      oldest

                      votes






                      active

                      oldest

                      votes









                      27














                      The key is to use these two Stream events:



                      • Event: 'data'

                      • Event: 'end'

                      For stream.on('data', ...) you should collect your data data into either a Buffer (if it is binary) or into a string.



                      For on('end', ...) you should call a callback with you completed buffer, or if you can inline it and use return using a Promises library.






                      share|improve this answer

























                      • That worked perfectly. Thanks!

                        – obrienmd
                        May 16 '12 at 18:10






                      • 112





                        A couple of lines of code illustrating the answer is preferable to just pointing a link at the API. Don't disagree with the answer, just don't believe it is complete enough.

                        – arcseldon
                        Jun 4 '14 at 5:14






                      • 2





                        With newer node.js versions, this is cleaner: stackoverflow.com/a/35530615/271961

                        – Simon A. Eugster
                        Dec 7 '16 at 11:28











                      • The answer should be updated to not recommend using a Promises library, but use native Promises.

                        – Dan Dascalescu
                        Mar 22 at 1:27











                      • @DanDascalescu I agree with you. The problem is that I wrote this answer 7 years ago, and I haven't kept up with node.js . If you are someone else would like to update it, that would be great. Or I could simply delete it, as there seems to be a better answer already. What would you recommend?

                        – ControlAltDel
                        Mar 25 at 13:30















                      27














                      The key is to use these two Stream events:



                      • Event: 'data'

                      • Event: 'end'

                      For stream.on('data', ...) you should collect your data data into either a Buffer (if it is binary) or into a string.



                      For on('end', ...) you should call a callback with you completed buffer, or if you can inline it and use return using a Promises library.






                      share|improve this answer

























                      • That worked perfectly. Thanks!

                        – obrienmd
                        May 16 '12 at 18:10






                      • 112





                        A couple of lines of code illustrating the answer is preferable to just pointing a link at the API. Don't disagree with the answer, just don't believe it is complete enough.

                        – arcseldon
                        Jun 4 '14 at 5:14






                      • 2





                        With newer node.js versions, this is cleaner: stackoverflow.com/a/35530615/271961

                        – Simon A. Eugster
                        Dec 7 '16 at 11:28











                      • The answer should be updated to not recommend using a Promises library, but use native Promises.

                        – Dan Dascalescu
                        Mar 22 at 1:27











                      • @DanDascalescu I agree with you. The problem is that I wrote this answer 7 years ago, and I haven't kept up with node.js . If you are someone else would like to update it, that would be great. Or I could simply delete it, as there seems to be a better answer already. What would you recommend?

                        – ControlAltDel
                        Mar 25 at 13:30













                      27












                      27








                      27







                      The key is to use these two Stream events:



                      • Event: 'data'

                      • Event: 'end'

                      For stream.on('data', ...) you should collect your data data into either a Buffer (if it is binary) or into a string.



                      For on('end', ...) you should call a callback with you completed buffer, or if you can inline it and use return using a Promises library.






                      share|improve this answer















                      The key is to use these two Stream events:



                      • Event: 'data'

                      • Event: 'end'

                      For stream.on('data', ...) you should collect your data data into either a Buffer (if it is binary) or into a string.



                      For on('end', ...) you should call a callback with you completed buffer, or if you can inline it and use return using a Promises library.







                      share|improve this answer














                      share|improve this answer



                      share|improve this answer








                      edited May 16 '12 at 17:54









                      Jordan Running

                      76.4k11135139




                      76.4k11135139










                      answered May 16 '12 at 17:51









                      ControlAltDelControlAltDel

                      21.7k53560




                      21.7k53560












                      • That worked perfectly. Thanks!

                        – obrienmd
                        May 16 '12 at 18:10






                      • 112





                        A couple of lines of code illustrating the answer is preferable to just pointing a link at the API. Don't disagree with the answer, just don't believe it is complete enough.

                        – arcseldon
                        Jun 4 '14 at 5:14






                      • 2





                        With newer node.js versions, this is cleaner: stackoverflow.com/a/35530615/271961

                        – Simon A. Eugster
                        Dec 7 '16 at 11:28











                      • The answer should be updated to not recommend using a Promises library, but use native Promises.

                        – Dan Dascalescu
                        Mar 22 at 1:27











                      • @DanDascalescu I agree with you. The problem is that I wrote this answer 7 years ago, and I haven't kept up with node.js . If you are someone else would like to update it, that would be great. Or I could simply delete it, as there seems to be a better answer already. What would you recommend?

                        – ControlAltDel
                        Mar 25 at 13:30

















                      • That worked perfectly. Thanks!

                        – obrienmd
                        May 16 '12 at 18:10






                      • 112





                        A couple of lines of code illustrating the answer is preferable to just pointing a link at the API. Don't disagree with the answer, just don't believe it is complete enough.

                        – arcseldon
                        Jun 4 '14 at 5:14






                      • 2





                        With newer node.js versions, this is cleaner: stackoverflow.com/a/35530615/271961

                        – Simon A. Eugster
                        Dec 7 '16 at 11:28











                      • The answer should be updated to not recommend using a Promises library, but use native Promises.

                        – Dan Dascalescu
                        Mar 22 at 1:27











                      • @DanDascalescu I agree with you. The problem is that I wrote this answer 7 years ago, and I haven't kept up with node.js . If you are someone else would like to update it, that would be great. Or I could simply delete it, as there seems to be a better answer already. What would you recommend?

                        – ControlAltDel
                        Mar 25 at 13:30
















                      That worked perfectly. Thanks!

                      – obrienmd
                      May 16 '12 at 18:10





                      That worked perfectly. Thanks!

                      – obrienmd
                      May 16 '12 at 18:10




                      112




                      112





                      A couple of lines of code illustrating the answer is preferable to just pointing a link at the API. Don't disagree with the answer, just don't believe it is complete enough.

                      – arcseldon
                      Jun 4 '14 at 5:14





                      A couple of lines of code illustrating the answer is preferable to just pointing a link at the API. Don't disagree with the answer, just don't believe it is complete enough.

                      – arcseldon
                      Jun 4 '14 at 5:14




                      2




                      2





                      With newer node.js versions, this is cleaner: stackoverflow.com/a/35530615/271961

                      – Simon A. Eugster
                      Dec 7 '16 at 11:28





                      With newer node.js versions, this is cleaner: stackoverflow.com/a/35530615/271961

                      – Simon A. Eugster
                      Dec 7 '16 at 11:28













                      The answer should be updated to not recommend using a Promises library, but use native Promises.

                      – Dan Dascalescu
                      Mar 22 at 1:27





                      The answer should be updated to not recommend using a Promises library, but use native Promises.

                      – Dan Dascalescu
                      Mar 22 at 1:27













                      @DanDascalescu I agree with you. The problem is that I wrote this answer 7 years ago, and I haven't kept up with node.js . If you are someone else would like to update it, that would be great. Or I could simply delete it, as there seems to be a better answer already. What would you recommend?

                      – ControlAltDel
                      Mar 25 at 13:30





                      @DanDascalescu I agree with you. The problem is that I wrote this answer 7 years ago, and I haven't kept up with node.js . If you are someone else would like to update it, that would be great. Or I could simply delete it, as there seems to be a better answer already. What would you recommend?

                      – ControlAltDel
                      Mar 25 at 13:30













                      51














                      Hope this is more useful than the above answer (which now has a broken link).



                      Also note that string concatenation is not an efficient way to collect the string parts, but it is used for simplicity (and perhaps your code does not care about efficiency)



                      var string = ''
                      stream.on('readable',function(buffer)
                      var part = buffer.read().toString();
                      string += part;
                      console.log('stream data ' + part);
                      );


                      stream.on('end',function()
                      console.log('final output ' + string);
                      );





                      share|improve this answer


















                      • 3





                        What would be a more efficient way to collect string parts? TY

                        – sean2078
                        Aug 27 '15 at 13:34






                      • 2





                        you could use a buffer docs.nodejitsu.com/articles/advanced/buffers/how-to-use-buffers but it really depends on your use.

                        – Tom Carchrae
                        Aug 27 '15 at 13:43






                      • 2





                        Use an array of strings where you append each new chunk to the array and call join("") on the array at the end.

                        – Valeriu Paloş
                        Mar 8 '16 at 7:22







                      • 6





                        This isn't right. If buffer is halfway through a multi-byte code point then the toString() will receive malformed utf-8 and you will end up with a bunch of � in your string.

                        – alextgordon
                        Oct 16 '16 at 22:03











                      • i'm not sure what you mean by a 'mutli-byte code point' but if you want to convert the encoding of the stream you can pass an encoding parameter like this toString('utf8') - but by default string encoding is utf8 so i suspect that your stream may not be utf8 @alextgordon - see stackoverflow.com/questions/12121775/… for more

                        – Tom Carchrae
                        Oct 18 '16 at 13:41
















                      51














                      Hope this is more useful than the above answer (which now has a broken link).



                      Also note that string concatenation is not an efficient way to collect the string parts, but it is used for simplicity (and perhaps your code does not care about efficiency)



                      var string = ''
                      stream.on('readable',function(buffer)
                      var part = buffer.read().toString();
                      string += part;
                      console.log('stream data ' + part);
                      );


                      stream.on('end',function()
                      console.log('final output ' + string);
                      );





                      share|improve this answer


















                      • 3





                        What would be a more efficient way to collect string parts? TY

                        – sean2078
                        Aug 27 '15 at 13:34






                      • 2





                        you could use a buffer docs.nodejitsu.com/articles/advanced/buffers/how-to-use-buffers but it really depends on your use.

                        – Tom Carchrae
                        Aug 27 '15 at 13:43






                      • 2





                        Use an array of strings where you append each new chunk to the array and call join("") on the array at the end.

                        – Valeriu Paloş
                        Mar 8 '16 at 7:22







                      • 6





                        This isn't right. If buffer is halfway through a multi-byte code point then the toString() will receive malformed utf-8 and you will end up with a bunch of � in your string.

                        – alextgordon
                        Oct 16 '16 at 22:03











                      • i'm not sure what you mean by a 'mutli-byte code point' but if you want to convert the encoding of the stream you can pass an encoding parameter like this toString('utf8') - but by default string encoding is utf8 so i suspect that your stream may not be utf8 @alextgordon - see stackoverflow.com/questions/12121775/… for more

                        – Tom Carchrae
                        Oct 18 '16 at 13:41














                      51












                      51








                      51







                      Hope this is more useful than the above answer (which now has a broken link).



                      Also note that string concatenation is not an efficient way to collect the string parts, but it is used for simplicity (and perhaps your code does not care about efficiency)



                      var string = ''
                      stream.on('readable',function(buffer)
                      var part = buffer.read().toString();
                      string += part;
                      console.log('stream data ' + part);
                      );


                      stream.on('end',function()
                      console.log('final output ' + string);
                      );





                      share|improve this answer













                      Hope this is more useful than the above answer (which now has a broken link).



                      Also note that string concatenation is not an efficient way to collect the string parts, but it is used for simplicity (and perhaps your code does not care about efficiency)



                      var string = ''
                      stream.on('readable',function(buffer)
                      var part = buffer.read().toString();
                      string += part;
                      console.log('stream data ' + part);
                      );


                      stream.on('end',function()
                      console.log('final output ' + string);
                      );






                      share|improve this answer












                      share|improve this answer



                      share|improve this answer










                      answered Sep 27 '14 at 15:08









                      Tom CarchraeTom Carchrae

                      5,47722630




                      5,47722630







                      • 3





                        What would be a more efficient way to collect string parts? TY

                        – sean2078
                        Aug 27 '15 at 13:34






                      • 2





                        you could use a buffer docs.nodejitsu.com/articles/advanced/buffers/how-to-use-buffers but it really depends on your use.

                        – Tom Carchrae
                        Aug 27 '15 at 13:43






                      • 2





                        Use an array of strings where you append each new chunk to the array and call join("") on the array at the end.

                        – Valeriu Paloş
                        Mar 8 '16 at 7:22







                      • 6





                        This isn't right. If buffer is halfway through a multi-byte code point then the toString() will receive malformed utf-8 and you will end up with a bunch of � in your string.

                        – alextgordon
                        Oct 16 '16 at 22:03











                      • i'm not sure what you mean by a 'mutli-byte code point' but if you want to convert the encoding of the stream you can pass an encoding parameter like this toString('utf8') - but by default string encoding is utf8 so i suspect that your stream may not be utf8 @alextgordon - see stackoverflow.com/questions/12121775/… for more

                        – Tom Carchrae
                        Oct 18 '16 at 13:41













                      • 3





                        What would be a more efficient way to collect string parts? TY

                        – sean2078
                        Aug 27 '15 at 13:34






                      • 2





                        you could use a buffer docs.nodejitsu.com/articles/advanced/buffers/how-to-use-buffers but it really depends on your use.

                        – Tom Carchrae
                        Aug 27 '15 at 13:43






                      • 2





                        Use an array of strings where you append each new chunk to the array and call join("") on the array at the end.

                        – Valeriu Paloş
                        Mar 8 '16 at 7:22







                      • 6





                        This isn't right. If buffer is halfway through a multi-byte code point then the toString() will receive malformed utf-8 and you will end up with a bunch of � in your string.

                        – alextgordon
                        Oct 16 '16 at 22:03











                      • i'm not sure what you mean by a 'mutli-byte code point' but if you want to convert the encoding of the stream you can pass an encoding parameter like this toString('utf8') - but by default string encoding is utf8 so i suspect that your stream may not be utf8 @alextgordon - see stackoverflow.com/questions/12121775/… for more

                        – Tom Carchrae
                        Oct 18 '16 at 13:41








                      3




                      3





                      What would be a more efficient way to collect string parts? TY

                      – sean2078
                      Aug 27 '15 at 13:34





                      What would be a more efficient way to collect string parts? TY

                      – sean2078
                      Aug 27 '15 at 13:34




                      2




                      2





                      you could use a buffer docs.nodejitsu.com/articles/advanced/buffers/how-to-use-buffers but it really depends on your use.

                      – Tom Carchrae
                      Aug 27 '15 at 13:43





                      you could use a buffer docs.nodejitsu.com/articles/advanced/buffers/how-to-use-buffers but it really depends on your use.

                      – Tom Carchrae
                      Aug 27 '15 at 13:43




                      2




                      2





                      Use an array of strings where you append each new chunk to the array and call join("") on the array at the end.

                      – Valeriu Paloş
                      Mar 8 '16 at 7:22






                      Use an array of strings where you append each new chunk to the array and call join("") on the array at the end.

                      – Valeriu Paloş
                      Mar 8 '16 at 7:22





                      6




                      6





                      This isn't right. If buffer is halfway through a multi-byte code point then the toString() will receive malformed utf-8 and you will end up with a bunch of � in your string.

                      – alextgordon
                      Oct 16 '16 at 22:03





                      This isn't right. If buffer is halfway through a multi-byte code point then the toString() will receive malformed utf-8 and you will end up with a bunch of � in your string.

                      – alextgordon
                      Oct 16 '16 at 22:03













                      i'm not sure what you mean by a 'mutli-byte code point' but if you want to convert the encoding of the stream you can pass an encoding parameter like this toString('utf8') - but by default string encoding is utf8 so i suspect that your stream may not be utf8 @alextgordon - see stackoverflow.com/questions/12121775/… for more

                      – Tom Carchrae
                      Oct 18 '16 at 13:41






                      i'm not sure what you mean by a 'mutli-byte code point' but if you want to convert the encoding of the stream you can pass an encoding parameter like this toString('utf8') - but by default string encoding is utf8 so i suspect that your stream may not be utf8 @alextgordon - see stackoverflow.com/questions/12121775/… for more

                      – Tom Carchrae
                      Oct 18 '16 at 13:41












                      41














                      None of the above worked for me. I needed to use the Buffer object:



                       const chunks = [];

                      readStream.on("data", function (chunk)
                      chunks.push(chunk);
                      );

                      // Send the buffer or you can put it into a var
                      readStream.on("end", function ()
                      res.send(Buffer.concat(chunks));
                      );





                      share|improve this answer


















                      • 3





                        this is actually the cleanest way of doing it ;)

                        – Ivo
                        Oct 28 '16 at 9:58











                      • Clean, efficient and works well with binary data.

                        – giosh94mhz
                        Jul 28 '17 at 7:11






                      • 2





                        Works great. Just a note: if you want a proper string type, you will need to call .toString() on the resulting Buffer object from concat() call

                        – Bryan Johnson
                        Sep 25 '17 at 14:30















                      41














                      None of the above worked for me. I needed to use the Buffer object:



                       const chunks = [];

                      readStream.on("data", function (chunk)
                      chunks.push(chunk);
                      );

                      // Send the buffer or you can put it into a var
                      readStream.on("end", function ()
                      res.send(Buffer.concat(chunks));
                      );





                      share|improve this answer


















                      • 3





                        this is actually the cleanest way of doing it ;)

                        – Ivo
                        Oct 28 '16 at 9:58











                      • Clean, efficient and works well with binary data.

                        – giosh94mhz
                        Jul 28 '17 at 7:11






                      • 2





                        Works great. Just a note: if you want a proper string type, you will need to call .toString() on the resulting Buffer object from concat() call

                        – Bryan Johnson
                        Sep 25 '17 at 14:30













                      41












                      41








                      41







                      None of the above worked for me. I needed to use the Buffer object:



                       const chunks = [];

                      readStream.on("data", function (chunk)
                      chunks.push(chunk);
                      );

                      // Send the buffer or you can put it into a var
                      readStream.on("end", function ()
                      res.send(Buffer.concat(chunks));
                      );





                      share|improve this answer













                      None of the above worked for me. I needed to use the Buffer object:



                       const chunks = [];

                      readStream.on("data", function (chunk)
                      chunks.push(chunk);
                      );

                      // Send the buffer or you can put it into a var
                      readStream.on("end", function ()
                      res.send(Buffer.concat(chunks));
                      );






                      share|improve this answer












                      share|improve this answer



                      share|improve this answer










                      answered Feb 21 '16 at 0:02









                      RickyRicky

                      15.4k43126




                      15.4k43126







                      • 3





                        this is actually the cleanest way of doing it ;)

                        – Ivo
                        Oct 28 '16 at 9:58











                      • Clean, efficient and works well with binary data.

                        – giosh94mhz
                        Jul 28 '17 at 7:11






                      • 2





                        Works great. Just a note: if you want a proper string type, you will need to call .toString() on the resulting Buffer object from concat() call

                        – Bryan Johnson
                        Sep 25 '17 at 14:30












                      • 3





                        this is actually the cleanest way of doing it ;)

                        – Ivo
                        Oct 28 '16 at 9:58











                      • Clean, efficient and works well with binary data.

                        – giosh94mhz
                        Jul 28 '17 at 7:11






                      • 2





                        Works great. Just a note: if you want a proper string type, you will need to call .toString() on the resulting Buffer object from concat() call

                        – Bryan Johnson
                        Sep 25 '17 at 14:30







                      3




                      3





                      this is actually the cleanest way of doing it ;)

                      – Ivo
                      Oct 28 '16 at 9:58





                      this is actually the cleanest way of doing it ;)

                      – Ivo
                      Oct 28 '16 at 9:58













                      Clean, efficient and works well with binary data.

                      – giosh94mhz
                      Jul 28 '17 at 7:11





                      Clean, efficient and works well with binary data.

                      – giosh94mhz
                      Jul 28 '17 at 7:11




                      2




                      2





                      Works great. Just a note: if you want a proper string type, you will need to call .toString() on the resulting Buffer object from concat() call

                      – Bryan Johnson
                      Sep 25 '17 at 14:30





                      Works great. Just a note: if you want a proper string type, you will need to call .toString() on the resulting Buffer object from concat() call

                      – Bryan Johnson
                      Sep 25 '17 at 14:30











                      24














                      Another way would be to convert the stream to a promise (refer to the example below) and use then (or await) to assign the resolved value to a variable.



                      function streamToString (stream) 
                      const chunks = []
                      return new Promise((resolve, reject) =>
                      stream.on('data', chunk => chunks.push(chunk))
                      stream.on('error', reject)
                      stream.on('end', () => resolve(Buffer.concat(chunks).toString('utf8')))
                      )


                      const result = await streamToString(stream)





                      share|improve this answer





























                        24














                        Another way would be to convert the stream to a promise (refer to the example below) and use then (or await) to assign the resolved value to a variable.



                        function streamToString (stream) 
                        const chunks = []
                        return new Promise((resolve, reject) =>
                        stream.on('data', chunk => chunks.push(chunk))
                        stream.on('error', reject)
                        stream.on('end', () => resolve(Buffer.concat(chunks).toString('utf8')))
                        )


                        const result = await streamToString(stream)





                        share|improve this answer



























                          24












                          24








                          24







                          Another way would be to convert the stream to a promise (refer to the example below) and use then (or await) to assign the resolved value to a variable.



                          function streamToString (stream) 
                          const chunks = []
                          return new Promise((resolve, reject) =>
                          stream.on('data', chunk => chunks.push(chunk))
                          stream.on('error', reject)
                          stream.on('end', () => resolve(Buffer.concat(chunks).toString('utf8')))
                          )


                          const result = await streamToString(stream)





                          share|improve this answer















                          Another way would be to convert the stream to a promise (refer to the example below) and use then (or await) to assign the resolved value to a variable.



                          function streamToString (stream) 
                          const chunks = []
                          return new Promise((resolve, reject) =>
                          stream.on('data', chunk => chunks.push(chunk))
                          stream.on('error', reject)
                          stream.on('end', () => resolve(Buffer.concat(chunks).toString('utf8')))
                          )


                          const result = await streamToString(stream)






                          share|improve this answer














                          share|improve this answer



                          share|improve this answer








                          edited Oct 19 '18 at 17:50









                          Dave Dunkin

                          1,011713




                          1,011713










                          answered Mar 22 '18 at 12:16









                          Marlon BernardesMarlon Bernardes

                          6,79442743




                          6,79442743





















                              17














                              I'm using usually this simple function to transform a stream into a string:



                              function streamToString(stream, cb) 
                              const chunks = [];
                              stream.on('data', (chunk) =>
                              chunks.push(chunk.toString());
                              );
                              stream.on('end', () =>
                              cb(chunks.join(''));
                              );



                              Usage example:



                              let stream = fs.createReadStream('./myFile.foo');
                              streamToString(stream, (data) =>
                              console.log(data); // data is now my string variable
                              );





                              share|improve this answer

























                              • Useful answer but it looks like each chunk must be converted to a string before it is pushed in the array: chunks.push(chunk.toString());

                                – Nicolas Le Thierry d'Ennequin
                                Jun 7 '16 at 10:06












                              • Your are right! I Change it!

                                – dreampulse
                                Jun 13 '16 at 8:41











                              • don t forget to bind error event!

                                – mh-cbon
                                Mar 13 '17 at 21:47















                              17














                              I'm using usually this simple function to transform a stream into a string:



                              function streamToString(stream, cb) 
                              const chunks = [];
                              stream.on('data', (chunk) =>
                              chunks.push(chunk.toString());
                              );
                              stream.on('end', () =>
                              cb(chunks.join(''));
                              );



                              Usage example:



                              let stream = fs.createReadStream('./myFile.foo');
                              streamToString(stream, (data) =>
                              console.log(data); // data is now my string variable
                              );





                              share|improve this answer

























                              • Useful answer but it looks like each chunk must be converted to a string before it is pushed in the array: chunks.push(chunk.toString());

                                – Nicolas Le Thierry d'Ennequin
                                Jun 7 '16 at 10:06












                              • Your are right! I Change it!

                                – dreampulse
                                Jun 13 '16 at 8:41











                              • don t forget to bind error event!

                                – mh-cbon
                                Mar 13 '17 at 21:47













                              17












                              17








                              17







                              I'm using usually this simple function to transform a stream into a string:



                              function streamToString(stream, cb) 
                              const chunks = [];
                              stream.on('data', (chunk) =>
                              chunks.push(chunk.toString());
                              );
                              stream.on('end', () =>
                              cb(chunks.join(''));
                              );



                              Usage example:



                              let stream = fs.createReadStream('./myFile.foo');
                              streamToString(stream, (data) =>
                              console.log(data); // data is now my string variable
                              );





                              share|improve this answer















                              I'm using usually this simple function to transform a stream into a string:



                              function streamToString(stream, cb) 
                              const chunks = [];
                              stream.on('data', (chunk) =>
                              chunks.push(chunk.toString());
                              );
                              stream.on('end', () =>
                              cb(chunks.join(''));
                              );



                              Usage example:



                              let stream = fs.createReadStream('./myFile.foo');
                              streamToString(stream, (data) =>
                              console.log(data); // data is now my string variable
                              );






                              share|improve this answer














                              share|improve this answer



                              share|improve this answer








                              edited Jun 13 '16 at 8:41

























                              answered Sep 14 '15 at 12:59









                              dreampulsedreampulse

                              60767




                              60767












                              • Useful answer but it looks like each chunk must be converted to a string before it is pushed in the array: chunks.push(chunk.toString());

                                – Nicolas Le Thierry d'Ennequin
                                Jun 7 '16 at 10:06












                              • Your are right! I Change it!

                                – dreampulse
                                Jun 13 '16 at 8:41











                              • don t forget to bind error event!

                                – mh-cbon
                                Mar 13 '17 at 21:47

















                              • Useful answer but it looks like each chunk must be converted to a string before it is pushed in the array: chunks.push(chunk.toString());

                                – Nicolas Le Thierry d'Ennequin
                                Jun 7 '16 at 10:06












                              • Your are right! I Change it!

                                – dreampulse
                                Jun 13 '16 at 8:41











                              • don t forget to bind error event!

                                – mh-cbon
                                Mar 13 '17 at 21:47
















                              Useful answer but it looks like each chunk must be converted to a string before it is pushed in the array: chunks.push(chunk.toString());

                              – Nicolas Le Thierry d'Ennequin
                              Jun 7 '16 at 10:06






                              Useful answer but it looks like each chunk must be converted to a string before it is pushed in the array: chunks.push(chunk.toString());

                              – Nicolas Le Thierry d'Ennequin
                              Jun 7 '16 at 10:06














                              Your are right! I Change it!

                              – dreampulse
                              Jun 13 '16 at 8:41





                              Your are right! I Change it!

                              – dreampulse
                              Jun 13 '16 at 8:41













                              don t forget to bind error event!

                              – mh-cbon
                              Mar 13 '17 at 21:47





                              don t forget to bind error event!

                              – mh-cbon
                              Mar 13 '17 at 21:47











                              7














                              From the nodejs documentation you should do this - always remember a string without knowing the encoding is just a bunch of bytes:



                              var readable = getReadableStreamSomehow();
                              readable.setEncoding('utf8');
                              readable.on('data', function(chunk)
                              assert.equal(typeof chunk, 'string');
                              console.log('got %d characters of string data', chunk.length);
                              )





                              share|improve this answer



























                                7














                                From the nodejs documentation you should do this - always remember a string without knowing the encoding is just a bunch of bytes:



                                var readable = getReadableStreamSomehow();
                                readable.setEncoding('utf8');
                                readable.on('data', function(chunk)
                                assert.equal(typeof chunk, 'string');
                                console.log('got %d characters of string data', chunk.length);
                                )





                                share|improve this answer

























                                  7












                                  7








                                  7







                                  From the nodejs documentation you should do this - always remember a string without knowing the encoding is just a bunch of bytes:



                                  var readable = getReadableStreamSomehow();
                                  readable.setEncoding('utf8');
                                  readable.on('data', function(chunk)
                                  assert.equal(typeof chunk, 'string');
                                  console.log('got %d characters of string data', chunk.length);
                                  )





                                  share|improve this answer













                                  From the nodejs documentation you should do this - always remember a string without knowing the encoding is just a bunch of bytes:



                                  var readable = getReadableStreamSomehow();
                                  readable.setEncoding('utf8');
                                  readable.on('data', function(chunk)
                                  assert.equal(typeof chunk, 'string');
                                  console.log('got %d characters of string data', chunk.length);
                                  )






                                  share|improve this answer












                                  share|improve this answer



                                  share|improve this answer










                                  answered Nov 14 '14 at 15:21









                                  Sebastian J.Sebastian J.

                                  607824




                                  607824





















                                      6














                                      Streams don't have a simple .toString() function (which I understand) nor something like a .toStringAsync(cb) function (which I don't understand).



                                      So I created my own helper function:



                                      var streamToString = function(stream, callback) 
                                      var str = '';
                                      stream.on('data', function(chunk)
                                      str += chunk;
                                      );
                                      stream.on('end', function()
                                      callback(str);
                                      );


                                      // how to use:
                                      streamToString(myStream, function(myStr)
                                      console.log(myStr);
                                      );





                                      share|improve this answer



























                                        6














                                        Streams don't have a simple .toString() function (which I understand) nor something like a .toStringAsync(cb) function (which I don't understand).



                                        So I created my own helper function:



                                        var streamToString = function(stream, callback) 
                                        var str = '';
                                        stream.on('data', function(chunk)
                                        str += chunk;
                                        );
                                        stream.on('end', function()
                                        callback(str);
                                        );


                                        // how to use:
                                        streamToString(myStream, function(myStr)
                                        console.log(myStr);
                                        );





                                        share|improve this answer

























                                          6












                                          6








                                          6







                                          Streams don't have a simple .toString() function (which I understand) nor something like a .toStringAsync(cb) function (which I don't understand).



                                          So I created my own helper function:



                                          var streamToString = function(stream, callback) 
                                          var str = '';
                                          stream.on('data', function(chunk)
                                          str += chunk;
                                          );
                                          stream.on('end', function()
                                          callback(str);
                                          );


                                          // how to use:
                                          streamToString(myStream, function(myStr)
                                          console.log(myStr);
                                          );





                                          share|improve this answer













                                          Streams don't have a simple .toString() function (which I understand) nor something like a .toStringAsync(cb) function (which I don't understand).



                                          So I created my own helper function:



                                          var streamToString = function(stream, callback) 
                                          var str = '';
                                          stream.on('data', function(chunk)
                                          str += chunk;
                                          );
                                          stream.on('end', function()
                                          callback(str);
                                          );


                                          // how to use:
                                          streamToString(myStream, function(myStr)
                                          console.log(myStr);
                                          );






                                          share|improve this answer












                                          share|improve this answer



                                          share|improve this answer










                                          answered Mar 9 '16 at 13:37









                                          floriflori

                                          6,54113344




                                          6,54113344





















                                              3














                                              I had more luck using like that :



                                              let string = '';
                                              readstream
                                              .on('data', (buf) => string += buf.toString())
                                              .on('end', () => console.log(string));


                                              I use node v9.11.1 and the readstream is the response from a http.get callback.






                                              share|improve this answer



























                                                3














                                                I had more luck using like that :



                                                let string = '';
                                                readstream
                                                .on('data', (buf) => string += buf.toString())
                                                .on('end', () => console.log(string));


                                                I use node v9.11.1 and the readstream is the response from a http.get callback.






                                                share|improve this answer

























                                                  3












                                                  3








                                                  3







                                                  I had more luck using like that :



                                                  let string = '';
                                                  readstream
                                                  .on('data', (buf) => string += buf.toString())
                                                  .on('end', () => console.log(string));


                                                  I use node v9.11.1 and the readstream is the response from a http.get callback.






                                                  share|improve this answer













                                                  I had more luck using like that :



                                                  let string = '';
                                                  readstream
                                                  .on('data', (buf) => string += buf.toString())
                                                  .on('end', () => console.log(string));


                                                  I use node v9.11.1 and the readstream is the response from a http.get callback.







                                                  share|improve this answer












                                                  share|improve this answer



                                                  share|improve this answer










                                                  answered May 7 '18 at 14:59









                                                  vdegennevdegenne

                                                  3,64184473




                                                  3,64184473





















                                                      3














                                                      Easy way with the popular (over 5m weekly downloads) and lightweight get-stream library:



                                                      https://www.npmjs.com/package/get-stream



                                                      const fs = require('fs');
                                                      const getStream = require('get-stream');

                                                      (async () =>
                                                      const stream = fs.createReadStream('unicorn.txt');
                                                      console.log(await getStream(stream)); //output is string
                                                      )();





                                                      share|improve this answer



























                                                        3














                                                        Easy way with the popular (over 5m weekly downloads) and lightweight get-stream library:



                                                        https://www.npmjs.com/package/get-stream



                                                        const fs = require('fs');
                                                        const getStream = require('get-stream');

                                                        (async () =>
                                                        const stream = fs.createReadStream('unicorn.txt');
                                                        console.log(await getStream(stream)); //output is string
                                                        )();





                                                        share|improve this answer

























                                                          3












                                                          3








                                                          3







                                                          Easy way with the popular (over 5m weekly downloads) and lightweight get-stream library:



                                                          https://www.npmjs.com/package/get-stream



                                                          const fs = require('fs');
                                                          const getStream = require('get-stream');

                                                          (async () =>
                                                          const stream = fs.createReadStream('unicorn.txt');
                                                          console.log(await getStream(stream)); //output is string
                                                          )();





                                                          share|improve this answer













                                                          Easy way with the popular (over 5m weekly downloads) and lightweight get-stream library:



                                                          https://www.npmjs.com/package/get-stream



                                                          const fs = require('fs');
                                                          const getStream = require('get-stream');

                                                          (async () =>
                                                          const stream = fs.createReadStream('unicorn.txt');
                                                          console.log(await getStream(stream)); //output is string
                                                          )();






                                                          share|improve this answer












                                                          share|improve this answer



                                                          share|improve this answer










                                                          answered Oct 17 '18 at 12:46









                                                          VilleVille

                                                          2229




                                                          2229





















                                                              2














                                                              What about something like a stream reducer ?



                                                              Here is an example using ES6 classes how to use one.



                                                              var stream = require('stream')

                                                              class StreamReducer extends stream.Writable
                                                              constructor(chunkReducer, initialvalue, cb)
                                                              super();
                                                              this.reducer = chunkReducer;
                                                              this.accumulator = initialvalue;
                                                              this.cb = cb;

                                                              _write(chunk, enc, next)
                                                              this.accumulator = this.reducer(this.accumulator, chunk);
                                                              next();

                                                              end()
                                                              this.cb(null, this.accumulator)



                                                              // just a test stream
                                                              class EmitterStream extends stream.Readable
                                                              constructor(chunks)
                                                              super();
                                                              this.chunks = chunks;

                                                              _read()
                                                              this.chunks.forEach(function (chunk)
                                                              this.push(chunk);
                                                              .bind(this));
                                                              this.push(null);



                                                              // just transform the strings into buffer as we would get from fs stream or http request stream
                                                              (new EmitterStream(
                                                              ["hello ", "world !"]
                                                              .map(function(str)
                                                              return Buffer.from(str, 'utf8');
                                                              )
                                                              )).pipe(new StreamReducer(
                                                              function (acc, v)
                                                              acc.push(v);
                                                              return acc;
                                                              ,
                                                              [],
                                                              function(err, chunks)
                                                              console.log(Buffer.concat(chunks).toString('utf8'));
                                                              )
                                                              );





                                                              share|improve this answer



























                                                                2














                                                                What about something like a stream reducer ?



                                                                Here is an example using ES6 classes how to use one.



                                                                var stream = require('stream')

                                                                class StreamReducer extends stream.Writable
                                                                constructor(chunkReducer, initialvalue, cb)
                                                                super();
                                                                this.reducer = chunkReducer;
                                                                this.accumulator = initialvalue;
                                                                this.cb = cb;

                                                                _write(chunk, enc, next)
                                                                this.accumulator = this.reducer(this.accumulator, chunk);
                                                                next();

                                                                end()
                                                                this.cb(null, this.accumulator)



                                                                // just a test stream
                                                                class EmitterStream extends stream.Readable
                                                                constructor(chunks)
                                                                super();
                                                                this.chunks = chunks;

                                                                _read()
                                                                this.chunks.forEach(function (chunk)
                                                                this.push(chunk);
                                                                .bind(this));
                                                                this.push(null);



                                                                // just transform the strings into buffer as we would get from fs stream or http request stream
                                                                (new EmitterStream(
                                                                ["hello ", "world !"]
                                                                .map(function(str)
                                                                return Buffer.from(str, 'utf8');
                                                                )
                                                                )).pipe(new StreamReducer(
                                                                function (acc, v)
                                                                acc.push(v);
                                                                return acc;
                                                                ,
                                                                [],
                                                                function(err, chunks)
                                                                console.log(Buffer.concat(chunks).toString('utf8'));
                                                                )
                                                                );





                                                                share|improve this answer

























                                                                  2












                                                                  2








                                                                  2







                                                                  What about something like a stream reducer ?



                                                                  Here is an example using ES6 classes how to use one.



                                                                  var stream = require('stream')

                                                                  class StreamReducer extends stream.Writable
                                                                  constructor(chunkReducer, initialvalue, cb)
                                                                  super();
                                                                  this.reducer = chunkReducer;
                                                                  this.accumulator = initialvalue;
                                                                  this.cb = cb;

                                                                  _write(chunk, enc, next)
                                                                  this.accumulator = this.reducer(this.accumulator, chunk);
                                                                  next();

                                                                  end()
                                                                  this.cb(null, this.accumulator)



                                                                  // just a test stream
                                                                  class EmitterStream extends stream.Readable
                                                                  constructor(chunks)
                                                                  super();
                                                                  this.chunks = chunks;

                                                                  _read()
                                                                  this.chunks.forEach(function (chunk)
                                                                  this.push(chunk);
                                                                  .bind(this));
                                                                  this.push(null);



                                                                  // just transform the strings into buffer as we would get from fs stream or http request stream
                                                                  (new EmitterStream(
                                                                  ["hello ", "world !"]
                                                                  .map(function(str)
                                                                  return Buffer.from(str, 'utf8');
                                                                  )
                                                                  )).pipe(new StreamReducer(
                                                                  function (acc, v)
                                                                  acc.push(v);
                                                                  return acc;
                                                                  ,
                                                                  [],
                                                                  function(err, chunks)
                                                                  console.log(Buffer.concat(chunks).toString('utf8'));
                                                                  )
                                                                  );





                                                                  share|improve this answer













                                                                  What about something like a stream reducer ?



                                                                  Here is an example using ES6 classes how to use one.



                                                                  var stream = require('stream')

                                                                  class StreamReducer extends stream.Writable
                                                                  constructor(chunkReducer, initialvalue, cb)
                                                                  super();
                                                                  this.reducer = chunkReducer;
                                                                  this.accumulator = initialvalue;
                                                                  this.cb = cb;

                                                                  _write(chunk, enc, next)
                                                                  this.accumulator = this.reducer(this.accumulator, chunk);
                                                                  next();

                                                                  end()
                                                                  this.cb(null, this.accumulator)



                                                                  // just a test stream
                                                                  class EmitterStream extends stream.Readable
                                                                  constructor(chunks)
                                                                  super();
                                                                  this.chunks = chunks;

                                                                  _read()
                                                                  this.chunks.forEach(function (chunk)
                                                                  this.push(chunk);
                                                                  .bind(this));
                                                                  this.push(null);



                                                                  // just transform the strings into buffer as we would get from fs stream or http request stream
                                                                  (new EmitterStream(
                                                                  ["hello ", "world !"]
                                                                  .map(function(str)
                                                                  return Buffer.from(str, 'utf8');
                                                                  )
                                                                  )).pipe(new StreamReducer(
                                                                  function (acc, v)
                                                                  acc.push(v);
                                                                  return acc;
                                                                  ,
                                                                  [],
                                                                  function(err, chunks)
                                                                  console.log(Buffer.concat(chunks).toString('utf8'));
                                                                  )
                                                                  );






                                                                  share|improve this answer












                                                                  share|improve this answer



                                                                  share|improve this answer










                                                                  answered Apr 12 '17 at 15:43









                                                                  FredFred

                                                                  15114




                                                                  15114





















                                                                      2














                                                                      The cleanest solution may be to use the "string-stream" package, which converts a stream to a string with a promise.



                                                                      const streamString = require('stream-string')

                                                                      streamString(myStream).then(string_variable =>
                                                                      // myStream was converted to a string, and that string is stored in string_variable
                                                                      console.log(string_variable)

                                                                      ).catch(err =>
                                                                      // myStream emitted an error event (err), so the promise from stream-string was rejected
                                                                      throw err
                                                                      )





                                                                      share|improve this answer



























                                                                        2














                                                                        The cleanest solution may be to use the "string-stream" package, which converts a stream to a string with a promise.



                                                                        const streamString = require('stream-string')

                                                                        streamString(myStream).then(string_variable =>
                                                                        // myStream was converted to a string, and that string is stored in string_variable
                                                                        console.log(string_variable)

                                                                        ).catch(err =>
                                                                        // myStream emitted an error event (err), so the promise from stream-string was rejected
                                                                        throw err
                                                                        )





                                                                        share|improve this answer

























                                                                          2












                                                                          2








                                                                          2







                                                                          The cleanest solution may be to use the "string-stream" package, which converts a stream to a string with a promise.



                                                                          const streamString = require('stream-string')

                                                                          streamString(myStream).then(string_variable =>
                                                                          // myStream was converted to a string, and that string is stored in string_variable
                                                                          console.log(string_variable)

                                                                          ).catch(err =>
                                                                          // myStream emitted an error event (err), so the promise from stream-string was rejected
                                                                          throw err
                                                                          )





                                                                          share|improve this answer













                                                                          The cleanest solution may be to use the "string-stream" package, which converts a stream to a string with a promise.



                                                                          const streamString = require('stream-string')

                                                                          streamString(myStream).then(string_variable =>
                                                                          // myStream was converted to a string, and that string is stored in string_variable
                                                                          console.log(string_variable)

                                                                          ).catch(err =>
                                                                          // myStream emitted an error event (err), so the promise from stream-string was rejected
                                                                          throw err
                                                                          )






                                                                          share|improve this answer












                                                                          share|improve this answer



                                                                          share|improve this answer










                                                                          answered Mar 18 '18 at 19:26









                                                                          Steve BreeseSteve Breese

                                                                          13724




                                                                          13724





















                                                                              0














                                                                              This worked for me and is based on Node v6.7.0 docs:



                                                                              let output = '';
                                                                              stream.on('readable', function()
                                                                              let read = stream.read();
                                                                              if (read !== null)
                                                                              // New stream data is available
                                                                              output += read.toString();
                                                                              else
                                                                              // Stream is now finished when read is null.
                                                                              // You can callback here e.g.:
                                                                              callback(null, output);

                                                                              );

                                                                              stream.on('error', function(err)
                                                                              callback(err, null);
                                                                              )





                                                                              share|improve this answer



























                                                                                0














                                                                                This worked for me and is based on Node v6.7.0 docs:



                                                                                let output = '';
                                                                                stream.on('readable', function()
                                                                                let read = stream.read();
                                                                                if (read !== null)
                                                                                // New stream data is available
                                                                                output += read.toString();
                                                                                else
                                                                                // Stream is now finished when read is null.
                                                                                // You can callback here e.g.:
                                                                                callback(null, output);

                                                                                );

                                                                                stream.on('error', function(err)
                                                                                callback(err, null);
                                                                                )





                                                                                share|improve this answer

























                                                                                  0












                                                                                  0








                                                                                  0







                                                                                  This worked for me and is based on Node v6.7.0 docs:



                                                                                  let output = '';
                                                                                  stream.on('readable', function()
                                                                                  let read = stream.read();
                                                                                  if (read !== null)
                                                                                  // New stream data is available
                                                                                  output += read.toString();
                                                                                  else
                                                                                  // Stream is now finished when read is null.
                                                                                  // You can callback here e.g.:
                                                                                  callback(null, output);

                                                                                  );

                                                                                  stream.on('error', function(err)
                                                                                  callback(err, null);
                                                                                  )





                                                                                  share|improve this answer













                                                                                  This worked for me and is based on Node v6.7.0 docs:



                                                                                  let output = '';
                                                                                  stream.on('readable', function()
                                                                                  let read = stream.read();
                                                                                  if (read !== null)
                                                                                  // New stream data is available
                                                                                  output += read.toString();
                                                                                  else
                                                                                  // Stream is now finished when read is null.
                                                                                  // You can callback here e.g.:
                                                                                  callback(null, output);

                                                                                  );

                                                                                  stream.on('error', function(err)
                                                                                  callback(err, null);
                                                                                  )






                                                                                  share|improve this answer












                                                                                  share|improve this answer



                                                                                  share|improve this answer










                                                                                  answered Oct 8 '16 at 12:05









                                                                                  anthonygoreanthonygore

                                                                                  2,9262126




                                                                                  2,9262126





















                                                                                      0














                                                                                      Using the quite popular stream-buffers package which you probably already have in your project dependencies, this is pretty straightforward:



                                                                                      // imports
                                                                                      const WritableStreamBuffer = require('stream-buffers');
                                                                                      const promisify = require('util');
                                                                                      const createReadStream = require('fs');
                                                                                      const pipeline = promisify(require('stream').pipeline);

                                                                                      // sample stream
                                                                                      let stream = createReadStream('/etc/hosts');

                                                                                      // pipeline the stream into a buffer, and print the contents when done
                                                                                      let buf = new WritableStreamBuffer();
                                                                                      pipeline(stream, buf).then(() => console.log(buf.getContents().toString()));





                                                                                      share|improve this answer



























                                                                                        0














                                                                                        Using the quite popular stream-buffers package which you probably already have in your project dependencies, this is pretty straightforward:



                                                                                        // imports
                                                                                        const WritableStreamBuffer = require('stream-buffers');
                                                                                        const promisify = require('util');
                                                                                        const createReadStream = require('fs');
                                                                                        const pipeline = promisify(require('stream').pipeline);

                                                                                        // sample stream
                                                                                        let stream = createReadStream('/etc/hosts');

                                                                                        // pipeline the stream into a buffer, and print the contents when done
                                                                                        let buf = new WritableStreamBuffer();
                                                                                        pipeline(stream, buf).then(() => console.log(buf.getContents().toString()));





                                                                                        share|improve this answer

























                                                                                          0












                                                                                          0








                                                                                          0







                                                                                          Using the quite popular stream-buffers package which you probably already have in your project dependencies, this is pretty straightforward:



                                                                                          // imports
                                                                                          const WritableStreamBuffer = require('stream-buffers');
                                                                                          const promisify = require('util');
                                                                                          const createReadStream = require('fs');
                                                                                          const pipeline = promisify(require('stream').pipeline);

                                                                                          // sample stream
                                                                                          let stream = createReadStream('/etc/hosts');

                                                                                          // pipeline the stream into a buffer, and print the contents when done
                                                                                          let buf = new WritableStreamBuffer();
                                                                                          pipeline(stream, buf).then(() => console.log(buf.getContents().toString()));





                                                                                          share|improve this answer













                                                                                          Using the quite popular stream-buffers package which you probably already have in your project dependencies, this is pretty straightforward:



                                                                                          // imports
                                                                                          const WritableStreamBuffer = require('stream-buffers');
                                                                                          const promisify = require('util');
                                                                                          const createReadStream = require('fs');
                                                                                          const pipeline = promisify(require('stream').pipeline);

                                                                                          // sample stream
                                                                                          let stream = createReadStream('/etc/hosts');

                                                                                          // pipeline the stream into a buffer, and print the contents when done
                                                                                          let buf = new WritableStreamBuffer();
                                                                                          pipeline(stream, buf).then(() => console.log(buf.getContents().toString()));






                                                                                          share|improve this answer












                                                                                          share|improve this answer



                                                                                          share|improve this answer










                                                                                          answered Sep 11 '18 at 4:30









                                                                                          andrewdotnandrewdotn

                                                                                          23.4k36699




                                                                                          23.4k36699





















                                                                                              0














                                                                                              setEncoding('utf8');



                                                                                              Well done Sebastian J above.



                                                                                              I had the "buffer problem" with a few lines of test code I had, and added the encoding information and it solved it, see below.



                                                                                              Demonstrate the problem



                                                                                              software



                                                                                              // process.stdin.setEncoding('utf8');
                                                                                              process.stdin.on('data', (data) =>
                                                                                              console.log(typeof(data), data);
                                                                                              );


                                                                                              input



                                                                                              hello world


                                                                                              output



                                                                                              object <Buffer 68 65 6c 6c 6f 20 77 6f 72 6c 64 0d 0a>


                                                                                              Demonstrate the solution



                                                                                              software



                                                                                              process.stdin.setEncoding('utf8'); // <- Activate!
                                                                                              process.stdin.on('data', (data) =>
                                                                                              console.log(typeof(data), data);
                                                                                              );


                                                                                              input



                                                                                              hello world


                                                                                              output



                                                                                              string hello world





                                                                                              share|improve this answer



























                                                                                                0














                                                                                                setEncoding('utf8');



                                                                                                Well done Sebastian J above.



                                                                                                I had the "buffer problem" with a few lines of test code I had, and added the encoding information and it solved it, see below.



                                                                                                Demonstrate the problem



                                                                                                software



                                                                                                // process.stdin.setEncoding('utf8');
                                                                                                process.stdin.on('data', (data) =>
                                                                                                console.log(typeof(data), data);
                                                                                                );


                                                                                                input



                                                                                                hello world


                                                                                                output



                                                                                                object <Buffer 68 65 6c 6c 6f 20 77 6f 72 6c 64 0d 0a>


                                                                                                Demonstrate the solution



                                                                                                software



                                                                                                process.stdin.setEncoding('utf8'); // <- Activate!
                                                                                                process.stdin.on('data', (data) =>
                                                                                                console.log(typeof(data), data);
                                                                                                );


                                                                                                input



                                                                                                hello world


                                                                                                output



                                                                                                string hello world





                                                                                                share|improve this answer

























                                                                                                  0












                                                                                                  0








                                                                                                  0







                                                                                                  setEncoding('utf8');



                                                                                                  Well done Sebastian J above.



                                                                                                  I had the "buffer problem" with a few lines of test code I had, and added the encoding information and it solved it, see below.



                                                                                                  Demonstrate the problem



                                                                                                  software



                                                                                                  // process.stdin.setEncoding('utf8');
                                                                                                  process.stdin.on('data', (data) =>
                                                                                                  console.log(typeof(data), data);
                                                                                                  );


                                                                                                  input



                                                                                                  hello world


                                                                                                  output



                                                                                                  object <Buffer 68 65 6c 6c 6f 20 77 6f 72 6c 64 0d 0a>


                                                                                                  Demonstrate the solution



                                                                                                  software



                                                                                                  process.stdin.setEncoding('utf8'); // <- Activate!
                                                                                                  process.stdin.on('data', (data) =>
                                                                                                  console.log(typeof(data), data);
                                                                                                  );


                                                                                                  input



                                                                                                  hello world


                                                                                                  output



                                                                                                  string hello world





                                                                                                  share|improve this answer













                                                                                                  setEncoding('utf8');



                                                                                                  Well done Sebastian J above.



                                                                                                  I had the "buffer problem" with a few lines of test code I had, and added the encoding information and it solved it, see below.



                                                                                                  Demonstrate the problem



                                                                                                  software



                                                                                                  // process.stdin.setEncoding('utf8');
                                                                                                  process.stdin.on('data', (data) =>
                                                                                                  console.log(typeof(data), data);
                                                                                                  );


                                                                                                  input



                                                                                                  hello world


                                                                                                  output



                                                                                                  object <Buffer 68 65 6c 6c 6f 20 77 6f 72 6c 64 0d 0a>


                                                                                                  Demonstrate the solution



                                                                                                  software



                                                                                                  process.stdin.setEncoding('utf8'); // <- Activate!
                                                                                                  process.stdin.on('data', (data) =>
                                                                                                  console.log(typeof(data), data);
                                                                                                  );


                                                                                                  input



                                                                                                  hello world


                                                                                                  output



                                                                                                  string hello world






                                                                                                  share|improve this answer












                                                                                                  share|improve this answer



                                                                                                  share|improve this answer










                                                                                                  answered Dec 10 '18 at 9:56









                                                                                                  user3070485user3070485

                                                                                                  1,5561716




                                                                                                  1,5561716



























                                                                                                      draft saved

                                                                                                      draft discarded
















































                                                                                                      Thanks for contributing an answer to Stack Overflow!


                                                                                                      • Please be sure to answer the question. Provide details and share your research!

                                                                                                      But avoid


                                                                                                      • Asking for help, clarification, or responding to other answers.

                                                                                                      • Making statements based on opinion; back them up with references or personal experience.

                                                                                                      To learn more, see our tips on writing great answers.




                                                                                                      draft saved


                                                                                                      draft discarded














                                                                                                      StackExchange.ready(
                                                                                                      function ()
                                                                                                      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f10623798%2fread-contents-of-node-js-stream-into-a-string-variable%23new-answer', 'question_page');

                                                                                                      );

                                                                                                      Post as a guest















                                                                                                      Required, but never shown





















































                                                                                                      Required, but never shown














                                                                                                      Required, but never shown












                                                                                                      Required, but never shown







                                                                                                      Required, but never shown

































                                                                                                      Required, but never shown














                                                                                                      Required, but never shown












                                                                                                      Required, but never shown







                                                                                                      Required, but never shown







                                                                                                      Popular posts from this blog

                                                                                                      Kamusi Yaliyomo Aina za kamusi | Muundo wa kamusi | Faida za kamusi | Dhima ya picha katika kamusi | Marejeo | Tazama pia | Viungo vya nje | UrambazajiKuhusu kamusiGo-SwahiliWiki-KamusiKamusi ya Kiswahili na Kiingerezakuihariri na kuongeza habari

                                                                                                      Swift 4 - func physicsWorld not invoked on collision? The Next CEO of Stack OverflowHow to call Objective-C code from Swift#ifdef replacement in the Swift language@selector() in Swift?#pragma mark in Swift?Swift for loop: for index, element in array?dispatch_after - GCD in Swift?Swift Beta performance: sorting arraysSplit a String into an array in Swift?The use of Swift 3 @objc inference in Swift 4 mode is deprecated?How to optimize UITableViewCell, because my UITableView lags

                                                                                                      Access current req object everywhere in Node.js ExpressWhy are global variables considered bad practice? (node.js)Using req & res across functionsHow do I get the path to the current script with Node.js?What is Node.js' Connect, Express and “middleware”?Node.js w/ express error handling in callbackHow to access the GET parameters after “?” in Express?Modify Node.js req object parametersAccess “app” variable inside of ExpressJS/ConnectJS middleware?Node.js Express app - request objectAngular Http Module considered middleware?Session variables in ExpressJSAdd properties to the req object in expressjs with Typescript