实时处理网络音频api

Question

我正在使用网络音频 api 和 requestAnimationFrame 来可视化来自麦克风的音频输入。我可以成功地将时域频率数据可视化，但问题是由于网络音频 api 以秒为单位计算时间，所以我的界面每秒都会根据输入的内容而变化。

所以我的问题是，我怎样才能可视化声音并使图表留在屏幕上，这样我就可以在一定的时间内看到我的所有频率数据（假设我说话同时在画布上可视化只有 5 秒)。

我正在使用以下代码（取自示例here）：

MicrophoneSample.prototype.visualize = function() {
  this.canvas.width = this.WIDTH;
  this.canvas.height = this.HEIGHT;
  var drawContext = this.canvas.getContext('2d');

  var times = new Uint8Array(this.analyser.frequencyBinCount);
  this.analyser.getByteTimeDomainData(times);
  for (var i = 0; i < times.length; i++) {
    var value = times[i];
    var percent = value / 256;
    var height = this.HEIGHT * percent;
    var offset = this.HEIGHT - height - 1;
    var barWidth = this.WIDTH/times.length;
    drawContext.fillStyle = 'purple';
    drawContext.fillRect(i * barWidth, offset, 1, 1);

  }
  requestAnimFrame(this.visualize.bind(this));

}

Answer 1

getByteTimeDomainData 不会为您提供频率信息。这些是实时的时域波形值，也称为幅度值。如果您想随着时间的推移将它们可视化，请将其附加到数组中并绘制它。如果您想要真实的频率值，请使用 getByteFrequencyData。

Answer 2

OP，这是一些伪代码。仅供参考，这真的不是网络音频问题，更像是动画问题。

在您的可视化工具原型函数中存储一个变量/字段，以跟踪您想要延迟画布重绘的秒数，并保留一个单独的计数器，该计数器将在每次绘制 requestAnimFrame(...) 时递增。一旦你的计数器达到你的延迟量，然后重新绘制画布。

编辑 现在想想……解决办法应该很简单。如果我错了，请纠正我，但这个粗略的解决方案是假设您从动画循环中调用 MicrophoneSample.visualize() ......因此，其中的代码每秒执行一次。如果您也发布您的 MicrophoneSample 对象代码，或者至少发布您的动画循环，我可能会提供更多帮助。

/* NOTE!
*
*/
// Find a way to put these into your PARENT MicrophoneSample object
var delay = 5;
// Note that I am setting delayCount initially to zero - during the loop
// the delayCount will actually get reset to 1 from thereafter (not 0)...
// this gives us a way to initially draw your visualization on the first frame.
var delayCount = 0;

// Pull var times out so it doesn't get calculated each time.
var times = new Uint8Array(MicrophoneSample.analyser.frequencyBinCount);

// Same goes for the canvas...
// I would set these values inside of the PARENT MicrophoneSample object
MicrophoneSample.canvas.width = this.WIDTH;
MicrophoneSample.canvas.height = this.HEIGHT;

// you only need to establish the drawing context once. Do it in the PARENT
// MicrophoneSample object
var drawContext = this.canvas.getContext('2d');

MicrophoneSample.prototype.visualize = function() {

      /*
      *    NOTE!
      */
      // Here's the juicy meat & potatoes:
      // only if the delayCount reaches the delay amount, should you UPDATE THE
      // TIME DOMAIN DATA ARRAY (times)
      // if your loop runs every second, then delayCount increments each second
      // and after 5 seconds will reach your designated delay amount and update your
      // times array.

      if(delayCount == 0 || delayCount == delay) {
          this.analyser.getByteTimeDomainData(times);

          // Now, it would be redundant (and totally noob-programmer of you) to 
          // redraw the same visualization onto the canvas 5 times in a row, so
          // only draw the visualization after the first pass through the loop and then
          // every 5th pass after that :]
          for (var i = 0; i < times.length; i++) {
              var value = times[i];
              var percent = value / 256;
              var height = this.HEIGHT * percent;
              var offset = this.HEIGHT - height - 1;
              var barWidth = this.WIDTH/times.length;
              drawContext.fillStyle = 'purple';
              drawContext.fillRect(i * barWidth, offset, 1, 1);
          }

          // Note: 1, not 0!
          delayCount = 1;
      } 
      else {
          delayCount++;
      } 


      requestAnimFrame(this.visualize.bind(this));
}

请记住，我还没有实际测试过这些。但它至少应该为您指明正确的方向。