Audio Frequence Visualization Based on WebAudio & Canvas

Part I: Audio Player

There are two approaches of playing audio inside browser, one of them is using the <audio> tag, which is described in the HTML5 specification; and the other approach is using WebAudio API.

WebAudio API offers a multitude of functions which can handle or process audio data. In order to handling audio file/source, we need an AudioSourceNode.

AudioSourceNode could be created with Stream, Buffer and MediaElement. And in the context of processing audio, any node inside the AudioContext can approach the sample data from upstream node, and pass it downward.

In this case, audio source will be loaded from a remote URL. The audio is easy to download and decode via <audio> element. Then we can use MediaElementAudioSourceNode to approache decoded audio data. Here is the sample code:

const audioElement: HTMLAudioElement = /* ref to the audio tag */
const audioContext = new AudioContext()
const sourceNode = new MediaElementSourceNode(audioElement)
sourceNode.connect(audioContext.destination)

Once user trigger the audio element's play behavior, the audio data won't output sound directly, it will be sent to the AudioContext. The code above connects the source and the destination, the audio data will be sent to the original output (eg. sound card) eventually.

The advantage of using <audio/> element is that the built-in functions, such as modifing playback rate & volume, can be utilized.

Part II: Data Source

Now we've got the audio source (the input), we need to extract data from the source input.

AnalyserNode is one of many built-in classes inside WebAudio API. Its instance can process input sample data. For example, AnalyserNode can apply FFT on the input sample, and the node outputs frequence data and time domain data. Here is the example code:

const audioElement: HTMLAudioElement = /* ref to the audio tag */
const audioContext = new AudioContext()
const sourceNode = audioContext.createMediaElementSource(audioElement)
const analyserNode = audioContext.createAnalyser()
analyserNode.fftSize = 1024
 
// sourceNode -> analyserNode -> audiocContext.destination
sourceNode.connect(analyserNode)
analyserNode.connect(audioContext.destination)
 
const LENGTH = analyserNode.frequencyBinCount
const dataArray = new Uint8Array(LENGTH)
 
// each time you call this method
// the analyserNode will put the frequency data into dataArray
analyserNode.getByteFrequencyData(dataArray)

Part III: Frequency Graph Renderder

Since we have frequency data, we can use it to render frequency graph.

I choose to render these data in the form of histogram, and there is some optimization of reduce the count of draw process: First, calculate all of the rect's outlines with one path and draw it after the calculation, then fill it with color. In this way only 2 drawing steps will be processed during each frame. Obviously, this would be faster than draw every single rect during each frame.

The frequency data size may be large due to the large fftSize, which is double of frequencyBinCount, and the graph dosen't need such a high precision, so it would be nice if we chunk the frequency data and display the average value.

Tips: getByteFrequencyData method should be called inside requestAnimationFrame funtion since this data could be different during each frame.

Here is the drawing code:

const ctx = canvas.getContext('2d')
 
const barCount = Math.floor(CANVAS_WIDTH / (BAR_WIDTH + BAR_GAP))
 
const draw = () => {
  ctx.clearRect(0, 0, CANVAS_WIDTH, CANVAS_HEIGHT)
  analyserNode.getByteFrequencyData(dataArray)
  const chunks = chunk(Array.from(dataArray), dataArray.length / barCount)
  ctx.fillStyle = BAR_COLOR
  ctx.beginPath()
  for (let i = 0; i < chunks.length; i++) {
    const chunk = chunks[i]
    const x = i * (BAR_WIDTH + BAR_GAP)
    const avgFrequency = sum(chunk) / chunk.length
    const barHeight = calcBarHeight(avgFrequency, CANVAS_HEIGHT)
    const y = CANVAS_HEIGHT - barHeight
    ctx.rect(x, y / 2, BAR_WIDTH, barHeight)
  }
  ctx.closePath()
  ctx.fill()
  requestAnimationFrame(draw)
}
 
// start drawing
requestAnimationFrame(draw)