Odtnhj

I'm working on a personal project that involves retrieving audio from YouTube, manipulating the audio, and streaming the result to the browser. So far I have the first and last steps down, but the middle is proving a challenge.

Thanks to the youtube-audio-stream package, getting the audio was easy. I wanted to manipulate the raw audio samples, so I followed their README example and piped the stream into a Decoder from the lame package.

I threw together a couple stream transforms... one to merge incoming chunks together until a size threshold was met, and the other one to actually do something with those chunks. At the end of the pipeline, I added a wav writer (which adds a WAV header so the browser isn't confused about the raw data coming in).

This actually results in normal audio output if my audio transform just passes along the chunks without any modification. So I know that the pipeline itself isn't broken. But for some reason, performing the following operation results in garbled noise:

chunk.reverse();

(This isn't the ultimate goal -- that involves FFT -- but I figured reversing audio chunks was a good operation to start with.)

I expected this to transform the stream into reversed fragments of sound, but instead it distorted it beyond recognition. I know that Node.js Buffers are Uint8Arrays, so I'm wondering if each sample is stored as 4 separate 8-bit integers. But I tried doing something like this:

const arr = Float32Array.from(chunk);
this.push(new Buffer(arr.reverse()));

and it's still garbled. I also tried writing a loop that used Buffer.readFloatLE and Buffer.writeFloatLE, but that didn't behave as expected either. What am I missing here? How can I retrieve and set audio sample data in a Node.js Buffer?

Edit: Adding example code (I'm running this locally as a microservice using micro):

index.js

const stream = require('youtube-audio-stream');
const wav = require('wav');
const decoder = require('lame').Decoder;
const Chunker, AudioThing = require('./transforms');

module.exports = (req, res) => 
 const url = 'https://www.youtube.com/watch?v=-L7IdUqaZxo';
 res.setHeader('Content-Type', 'audio/wav');
 return stream(url)
 .pipe(decoder())
 .pipe(new Chunker(2 ** 16))
 .pipe(new AudioThing())
 .pipe(new wav.Writer());

transforms.js

const Transform = require('stream');

class Chunker extends Transform 
 constructor(threshold) 
 super();
 this.size = 0;
 this.chunks = ;
 this.threshold = threshold;
 

 _transform(chunk, encoding, done) 
 this.size += chunk.length;
 this.chunks.push(chunk);
 if (this.size >= this.threshold) 
 this.push(Buffer.concat(this.chunks, this.size));
 this.chunks = ;
 this.size = 0;
 
 done();
 


class AudioThing extends Transform 
 _transform(chunk, encoding, done) 
 this.push(chunk.reverse());
 done();
 


module.exports = Chunker, AudioThing ;

Edit 2: Solved! For future reference, here are the utility functions I wrote to decode/encode the audio data:

function decodeBuffer (buffer) 
 return Array.from(
 length: buffer.length / 2 ,
 (v, i) => buffer.readInt16LE(i * 2) / (2 ** 15)
 );


function encodeArray (array) 
 const buf = Buffer.alloc(array.length * 2);
 for (let i = 0; i < array.length; i++) 
 buf.writeInt16LE(array[i] * (2 ** 15), i * 2);
 
 return buf;

You can't simply reverse the byte array. As you suspected, samples are going to span more than one byte.

It seems plausible that you have the sample format wrong. It's probably not 32-bit float, but is probably signed 16-bit integers. This isn't documented well, but if you dig into the source code for node-lame, you find this:

if (ret == MPG123_NEW_FORMAT) 
 var format = binding.mpg123_getformat(mh);
 debug('new format: %j', format);
 self.emit('format', format);
 return read();

It looks like the underlying MPG123 can return PCM in several formats:

 if (ret == MPG123_OK) {
 Local<Object> o = Nan::New<Object>();
 Nan::Set(o, Nan::New<String>("raw_encoding").ToLocalChecked(), Nan::New<Number>(encoding));
 Nan::Set(o, Nan::New<String>("sampleRate").ToLocalChecked(), Nan::New<Number>(rate));
 Nan::Set(o, Nan::New<String>("channels").ToLocalChecked(), Nan::New<Number>(channels));
 Nan::Set(o, Nan::New<String>("signed").ToLocalChecked(), Nan::New<Boolean>(encoding & MPG123_ENC_SIGNED));
 Nan::Set(o, Nan::New<String>("float").ToLocalChecked(), Nan::New<Boolean>(encoding & MPG123_ENC_FLOAT));
 Nan::Set(o, Nan::New<String>("ulaw").ToLocalChecked(), Nan::New<Boolean>(encoding & MPG123_ENC_ULAW_8));
 Nan::Set(o, Nan::New<String>("alaw").ToLocalChecked(), Nan::New<Boolean>(encoding & MPG123_ENC_ALAW_8));
 if (encoding & MPG123_ENC_8)
 Nan::Set(o, Nan::New<String>("bitDepth").ToLocalChecked(), Nan::New<Integer>(8));
 else if (encoding & MPG123_ENC_16)
 Nan::Set(o, Nan::New<String>("bitDepth").ToLocalChecked(), Nan::New<Integer>(16));
 else if (encoding & MPG123_ENC_24)
 Nan::Set(o, Nan::New<String>("bitDepth").ToLocalChecked(), Nan::New<Integer>(24));
 else if (encoding & MPG123_ENC_32 || encoding & MPG123_ENC_FLOAT_32)
 Nan::Set(o, Nan::New<String>("bitDepth").ToLocalChecked(), Nan::New<Integer>(32));
 else if (encoding & MPG123_ENC_FLOAT_64)
 Nan::Set(o, Nan::New<String>("bitDepth").ToLocalChecked(), Nan::New<Integer>(64));
 rtn = o;

I would try your looping technique again to reverse the samples while keeping the bytes in each sample in-tact, but try this with different sample sizes. Start with 16-bit signed, little-endian.