今天，我们将研究如何使用 React Flow 和 Web Audio API 创建交互式音频在线运行。我们将从头开始，首先了解 Web Audio API，然后再了解如何处理 React Flow 中的许多常见场景：状态管理、实现自定义节点和添加交互性。

¥Today we’ll be looking at how to create an interactive audio playground using React Flow and the Web Audio API. We’ll start from scratch, first learning about the Web Audio API before looking at how to handle many common scenarios in React Flow: state management, implementing custom nodes, and adding interactivity.

不久前，我与 React Flow discord 服务器  分享了我正在开展的一个项目。它被称为 bleep.cafe ，是一个用于在浏览器内学习数字合成的小型 Web 应用。很多人都有兴趣看看这样的东西是如何组合在一起的：大多数人甚至不知道他们的浏览器内置了整个合成引擎！

¥A while back I shared a project I was working on to the React Flow discord server . It’s called bleep.cafe  and it’s a little web app for learning digital synthesis all inside the browser. A lot of folks were interested to see how something like that was put together: most people don’t even know their browser has a whole synth engine built in!

本教程将逐步指导我们构建类似的东西。我们可能会跳过一些部分，但在大多数情况下，如果你是 React Flow 或 Web Audio API 的新手，你应该能够跟上并在最后完成一些工作。

¥This tutorial will take us step-by-step to build something similar. We may skip over some bits here and there, but for the most part if you’re new to React Flow or the Web Audio API you should be able to follow along and have something working by the end.

如果你已经是 React Flow 向导，你可能希望阅读第一部分介绍 Web Audio API，然后跳到第三部分以了解事物是如何联系在一起的！

¥If you’re already a React Flow wizard you might want to read the first section covering the Web Audio API and then jump to the third to see how things are tied together!

但首先…

¥But first…

演示！

¥A demo!

Web Audio API

在我们陷入 React Flow 和交互式节点编辑器的优点之前，我们需要参加 Web 音频 API  的速成课程。以下是你需要了解的重点：

¥Before we get stuck in to React Flow and interactive node editor goodness, we need to take a crash course on the Web Audio API . Here are the highlights you need to know:

• Web Audio API 提供了各种不同的音频节点，包括源（例如 OscillatorNode 、MediaElementAudioSourceNode ）、效果（例如 GainNode 、DelayNode 、ConvolverNode ）和输出（例如 AudioDestinationNode ）。

  ¥The Web Audio API provides a variety of different audio nodes, including sources (e.g. OscillatorNode , MediaElementAudioSourceNode ), effects (e.g. GainNode , DelayNode , ConvolverNode ), and outputs (e.g. AudioDestinationNode ).

• 音频节点可以连接在一起形成（可能循环的）图。我们倾向于将其称为音频处理图、信号图或信号链。

  ¥Audio nodes can be connected together to form a (potentially cyclic) graph. We tend to call this the audio-processing graph, signal graph, or signal chain.

• 音频处理由原生代码在单独的线程中处理。这意味着即使主 UI 线程繁忙或被阻塞，我们也可以继续生成声音。

  ¥Audio processing is handled in a separate thread by native code. This means we can keep generating sounds even when the main UI thread is busy or blocked.

• AudioContext  充当音频处理图的大脑。我们可以使用它来创建新的音频节点并完全暂停或恢复音频处理。

  ¥An AudioContext  acts as the brain of an audio-processing graph. We can use it to create new audio nodes and suspend or resume audio processing entirely.

你好，声音！

¥Hello, sound!

让我们看看其中的一些实际操作，并构建我们的第一个 Web Audio 应用！我们不会做任何太疯狂的事情：我们将制作一个简单的鼠标 theremin 。我们将使用 React 进行这些示例和其他所有后续工作（毕竟我们被称为 React Flow！）并使用 vite 来处理打包和热重载。

¥Let’s see some of this stuff in action and build our first Web Audio app! We won’t be doing anything too wild: we’ll make a simple mouse theremin . We’ll use React for these examples and everything else moving forward (we’re called React Flow after all!) and vite to handle bundling and hot reloading.

如果你更喜欢另一个像 parcel 或 Create React App 这样的打包器，它们也很酷，它们都做同样的事情。你也可以选择使用 TypeScript 而不是 JavaScript。为简单起见，我们今天不会使用它，但 React Flow 是完全类型化的（并且完全用 TypeScript 编写），所以使用起来很轻松！

¥If you prefer another bundler like parcel or Create React App that’s cool too, they all do largely the same thing. You could also choose to use TypeScript instead of JavaScript. To keep things simple we won’t use it today, but React Flow is fully typed (and written entirely in TypeScript) so it’s a breeze to use!

npm create vite@latest -- --template react

npm create vite@latest -- --template react
# couldn't auto-convert command

npm create vite@latest -- --template react
# couldn't auto-convert command

bunx create-vite@latest --template react

Vite 将为我们搭建一个简单的 React 应用，但可以删除资源并直接进入 App.jsx。删除为我们生成的演示组件，然后开始创建一个新的 AudioContext 并组合我们需要的节点。我们希望 OscillatorNode 生成一些音调，并希望 GainNode 控制音量。

¥Vite will scaffold out a simple React application for us, but can delete the assets and jump right into App.jsx. Remove the demo component generated for us and start by creating a new AudioContext and putting together the nodes we need. We want an OscillatorNode to generate some tones and a GainNode to control the volume.

// Create the brain of our audio-processing graph
const context = new AudioContext();
 
// Create an oscillator node to generate tones
const osc = context.createOscillator();
 
// Create a gain node to control the volume
const amp = context.createGain();
 
// Pass the oscillator's output through the gain node and to our speakers
osc.connect(amp);
amp.connect(context.destination);
 
// Start generating those tones!
osc.start();

对于我们的应用，我们将跟踪鼠标在屏幕上的位置，并使用它来设置振荡器节点的音高和增益节点的音量。

¥For our app, we’ll track the mouse’s position on the screen and use that to set the pitch of the oscillator node and the volume of the gain node.

import React from 'react';
 
const context = new AudioContext();
const osc = context.createOscillator();
const amp = context.createGain();
 
osc.connect(amp);
amp.connect(context.destination);
 
osc.start();
 
const updateValues = (e) => {
  const freq = (e.clientX / window.innerWidth) * 1000;
  const gain = e.clientY / window.innerHeight;
 
  osc.frequency.value = freq;
  amp.gain.value = gain;
};
 
export default function App() {
  return (
    <div
      style={{ width: '100vw', height: '100vh' }}
      onMouseMove={updateValues}
    />
  );
}

如果你按原样尝试此示例，你可能会发现什么都没有发生。AudioContext 通常以暂停状态启动，以避免广告劫持我们的扬声器。我们可以通过添加 <div /> 上的点击处理程序来轻松修复该问题，以便在上下文暂停时恢复上下文。

¥If you try this example as it is, you’ll probably find that nothing happens. An AudioContext often starts in a suspended state in an attempt to avoid ads hijacking our speakers. We can fix that easily by adding a click handler on the <div /> to resume the context if it’s suspended.

const toggleAudio = () => {
  if (context.state === 'suspended') {
    context.resume();
  } else {
    context.suspend();
  }
};
 
export default function App() {
  return (
    <div ...
      onClick={toggleAudio}
    />
  );
};

这就是我们开始使用 Web Audio API 制作声音所需要的一切！以下是我们放在一起的内容，以防你没有在家里跟着做：

¥And that’s everything we need to start making some sounds with the Web Audio API! Here’s what we put together, in case you weren’t following along at home:

现在让我们把这些知识放在一边，看看如何从头开始构建 React Flow 项目。

¥Now let’s put this knowledge to one side and take a look at how to build a React Flow project from scratch.

搭建 React Flow 项目脚手架

¥Scaffolding a React Flow project

稍后，我们将利用我们学到的有关 Web Audio API、振荡器和增益节点的知识，并使用 React Flow 以交互方式构建音频处理图。不过现在，我们需要组装一个空的 React Flow 应用。

¥Later on we’ll take what we’ve learned about the Web Audio API, oscillators, and gain nodes and use React Flow to interactively build audio-processing graphs. For now though, we need to put together an empty React Flow app.

我们已经有一个使用 Vite 设置的 React 应用，所以我们会继续使用它。如果你跳过最后一节，我们运行 npm create vite@latest -- --template react 开始。不过，你可以使用任何你喜欢的打包器和/或开发服务器。这里没有任何特定于 vite 的内容。

¥We already have a React app set up with Vite, so we’ll keep using that. If you skipped over the last section, we ran npm create vite@latest -- --template react to get started. You can use whatever bundler and/or dev server you like, though. Nothing here is vite specific.

我们只需要这个项目的三个额外依赖：@xyflow/react 用于我们的 UI（显然！），zustand 作为我们的简单状态管理库（这是我们在 React Flow 内部使用的），nanoid 作为轻量级 id 生成器。

¥We only need three additional dependencies for this project: @xyflow/react for our UI (obviously!), zustand as our simple state management library (that’s what we use under the hood at React Flow) and nanoid as a lightweight id generator.

npm install @xyflow/react zustand nanoid

pnpm add @xyflow/react zustand nanoid

yarn add @xyflow/react zustand nanoid

bun add @xyflow/react zustand nanoid

我们将从我们的 Web Audio 速成课程中删除所有内容并从头开始。首先修改 main.jsx 以匹配以下内容：

¥We’re going to remove everything from our Web Audio crash course and start from scratch. Start by modifying main.jsx to match the following:

import App from './App';
import React from 'react';
import ReactDOM from 'react-dom/client';
import { ReactFlowProvider } from '@xyflow/react';
 
// 👇 Don't forget to import the styles!
import '@xyflow/react/dist/style.css';
import './index.css';
 
const root = document.querySelector('#root');
 
ReactDOM.createRoot(root).render(
  <React.StrictMode>
    {/* React flow needs to be inside an element with a known height and width to work */}
    <div style={{ width: '100vw', height: '100vh' }}>
      <ReactFlowProvider>
        <App />
      </ReactFlowProvider>
    </div>
  </React.StrictMode>,
);

这里有三件重要的事情需要注意：

¥There are three important things to pay attention to here:

1. 你需要记住导入 React Flow CSS 样式以确保一切正常运行。

   ¥You need to remember to import the React Flow CSS styles to make sure everything works correctly.

2. React Flow 渲染器需要位于具有已知高度和宽度的元素内，因此我们将包含的 <div /> 设置为占据整个屏幕。

   ¥The React Flow renderer needs to be inside an element with a known height and width, so we’ve set the containing <div /> to take up the entire screen.

3. 要使用 React Flow 提供的一些钩子，你的组件需要位于 <ReactFlowProvider /> 内部或 <ReactFlow /> 组件本身内部，因此我们将整个应用封装在提供程序中以确保万无一失。

   ¥To use some of the hooks React Flow provides, your components need to be inside a <ReactFlowProvider /> or inside the <ReactFlow /> component itself, so we’ve wrapped the entire app in the provider to be sure.

接下来，跳转到 App.jsx 并创建一个空流：

¥Next, hop into App.jsx and create an empty flow:

import React from 'react';
import { ReactFlow, Background } from '@xyflow/react';
 
export default function App() {
  return (
    <ReactFlow>
      <Background />
    </ReactFlow>
  );
}

我们将随着时间的推移扩展和添加此组件。目前，我们添加了 React Flow 的一个插件 - <Background /> - 检查一切是否设置正确。继续运行 npm run dev（或者如果你没有选择 vite，则需要执行任何操作来启动开发服务器）并检查你的浏览器。你应该看到一个空流：

¥We’ll expand and add on to this component over time. For now, we’ve added one of React Flow’s plugins - <Background /> - to check if everything is setup correctly. Go ahead and run npm run dev (or whatever you need to do to spin up a dev server if you didn’t choose vite) and check out your browser. You should see an empty flow:

让开发服务器保持运行。我们可以在添加新内容时继续检查进度。

¥Leave the dev server running. We can keep checking back on our progress as we add new bits and bobs.

1. 使用 Zusand 进行状态管理

¥ State management with Zustand

Zustand 存储将保存我们应用的所有 UI 状态。实际上，这意味着它将保存我们的 React Flow 图的节点和边、一些其他状态以及一些用于更新该状态的操作。

¥A Zustand store will hold all the UI state for our application. In practical terms that means it’ll hold the nodes and edges of our React Flow graph, a few other pieces of state, and a handful of actions to update that state.

要获得基本的交互式 React Flow 图，我们需要三个操作：

¥To get a basic interactive React Flow graph going we need three actions:

1. onNodesChange 处理被移动或删除的节点。

   ¥onNodesChange to handle nodes being moved around or deleted.

2. onEdgesChange 处理被移动或删除的边缘。

   ¥onEdgesChange to handle edges being moved around or deleted.

3. addEdge 连接图中的两个节点。

   ¥addEdge to connect two nodes in the graph.

继续创建一个新文件 store.js，并添加以下内容：

¥Go ahead and create a new file, store.js, and add the following:

import { applyNodeChanges, applyEdgeChanges } from '@xyflow/react';
import { nanoid } from 'nanoid';
import { createWithEqualityFn } from 'zustand/traditional';
 
export const useStore = createWithEqualityFn((set, get) => ({
  nodes: [],
  edges: [],
 
  onNodesChange(changes) {
    set({
      nodes: applyNodeChanges(changes, get().nodes),
    });
  },
 
  onEdgesChange(changes) {
    set({
      edges: applyEdgeChanges(changes, get().edges),
    });
  },
 
  addEdge(data) {
    const id = nanoid(6);
    const edge = { id, ...data };
 
    set({ edges: [edge, ...get().edges] });
  },
}));

Zustand 使用起来非常简单。我们创建一个函数，该函数接收 set 和 get 函数，并返回一个包含初始状态以及可用于更新该状态的操作的对象。更新是不可变的，我们可以使用 set 函数来实现。get 函数是我们读取当前状态的方式。而且…这就是 zustand。

¥Zustand is dead simple to use. We create a function that receives both a set and a get function and returns an object with our initial state along with the actions we can use to update that state. Updates happen immutably and we can use the set function for that. The get function is how we read the current state. And… that’s it for zustand.

onNodesChange 和 onEdgesChange 中的 changes 参数表示节点或边被移动或删除等事件。幸运的是，React Flow 提供了一些 helper functions 来为我们应用这些更改。我们只需要使用新的节点数组更新存储。

¥The changes argument in both onNodesChange and onEdgesChange represents events like a node or edge being moved or deleted. Fortunately, React Flow provides some helper functions to apply those changes for us. We just need to update the store with the new array of nodes.

每当两个节点连接时，就会调用 addEdge。data 参数几乎是一个有效的边，只是缺少一个 id。这里我们让 nanoid 生成一个 6 个字符的随机 ID，然后将边添加到我们的图中，没什么特别的。

¥addEdge will be called whenever two nodes get connected. The data argument is almost a valid edge, it’s just missing an id. Here we’re getting nanoid to generate a 6 character random id and then adding the edge to our graph, nothing exciting.

如果我们跳回到我们的 <App /> 组件，我们可以将 React Flow 连接到我们的操作并使其正常工作。

¥If we hop back over to our <App /> component we can hook React Flow up to our actions and get something working.

import React from 'react';
import { ReactFlow, Background } from '@xyflow/react';
import { shallow } from 'zustand/shallow';
 
import { useStore } from './store';
 
const selector = (store) => ({
  nodes: store.nodes,
  edges: store.edges,
  onNodesChange: store.onNodesChange,
  onEdgesChange: store.onEdgesChange,
  addEdge: store.addEdge,
});
 
export default function App() {
  const store = useStore(selector, shallow);
 
  return (
    <ReactFlow
      nodes={store.nodes}
      edges={store.edges}
      onNodesChange={store.onNodesChange}
      onEdgesChange={store.onEdgesChange}
      onConnect={store.addEdge}
    >
      <Background />
    </ReactFlow>
  );
}

那么这个 selector 到底是什么？Zustand 让我们提供一个选择器函数来从存储中挑选出我们需要的确切状态位。与 shallow 相等函数相结合，这意味着当我们不关心状态变化时，我们通常不需要重新渲染。

¥So what’s this selector thing all about? Zustand let’s us supply a selector function to pluck out the exact bits of state we need from the store. Combined with the shallow equality function, this means we typically don’t have re-renders when state we don’t care about changes.

现在，我们的存储很小，我们实际上希望从中获取所有内容来帮助渲染我们的 React Flow 图，但是随着我们对其进行扩展，此选择器将确保我们不会一直重新渲染所有内容。

¥Right now, our store is small and we actually want everything from it to help render our React Flow graph, but as we expand on it this selector will make sure we’re not re-rendering everything all the time.

这是我们拥有交互式图表所需的一切：我们可以移动节点、将它们连接在一起并删除它们。为了演示，暂时将一些虚拟节点添加到你的存储：

¥This is everything we need to have an interactive graph: we can move nodes around, connect them together, and remove them. To demonstrate, temporarily add some dummy nodes to your store:

const useStore = createWithEqualityFn((set, get) => ({
  nodes: [
    { id: 'a', data: { label: 'oscillator' }, position: { x: 0, y: 0 } },
    { id: 'b', data: { label: 'gain' }, position: { x: 50, y: 50 } },
    { id: 'c', data: { label: 'output' }, position: { x: -50, y: 100 } }
  ],
  ...
}));

import React from 'react';
import { ReactFlow, ReactFlowProvider, Background } from '@xyflow/react';
import { shallow } from 'zustand/shallow';
 
import { useStore } from './store';
 
import '@xyflow/react/dist/style.css';
 
const selector = (store) => ({
  nodes: store.nodes,
  edges: store.edges,
  onNodesChange: store.onNodesChange,
  onEdgesChange: store.onEdgesChange,
  addEdge: store.addEdge,
});
 
export default function App() {
  const store = useStore(selector, shallow);
 
  return (
    <ReactFlowProvider>
      <div style={{ width: '100vw', height: '100vh' }}>
        <ReactFlow
          nodes={store.nodes}
          edges={store.edges}
          onNodesChange={store.onNodesChange}
          onEdgesChange={store.onEdgesChange}
          onConnect={store.addEdge}
          fitView
        >
          <Background />
        </ReactFlow>
      </div>
    </ReactFlowProvider>
  );
}

2. 自定义节点

¥ Custom nodes

好的，太好了，我们有一个可以开始使用的交互式 React Flow 实例。我们添加了一些虚拟节点，但它们现在只是默认的未设置样式的节点。在此步骤中，我们将添加三个带有交互式控件的自定义节点：

¥OK great, we have an interactive React Flow instance we can start playing with. We added some dummy nodes but they’re just the default unstyled ones right now. In this step we’ll add three custom nodes with interactive controls:

1. 振荡器节点和用于音高和波形类型的控件。

   ¥An oscillator node and controls for the pitch and waveform type.

2. 增益节点和音量控制

   ¥A gain node and a control for the volume

3. 输出节点和用于打开和关闭音频处理的按钮。

   ¥An output node and a button to toggle audio processing on and off.

让我们创建一个新文件夹 nodes/，并为我们要创建的每个自定义节点创建一个文件。从振荡器开始，我们需要两个控件和一个源句柄来将振荡器的输出连接到其他节点。

¥Let’s create a new folder, nodes/, and create a file for each custom node we want to create. Starting with the oscillator we need two controls and a source handle to connect the output of the oscillator to other nodes.

import React from 'react';
import { Handle } from '@xyflow/react';
 
import { useStore } from '../store';
 
export default function Osc({ id, data }) {
  return (
    <div>
      <div>
        <p>Oscillator Node</p>
 
        <label>
          <span>Frequency</span>
          <input
            className="nodrag"
            type="range"
            min="10"
            max="1000"
            value={data.frequency} />
          <span>{data.frequency}Hz</span>
        </label>
 
        <label>
          <span>Waveform</span>
          <select className="nodrag" value={data.type}>
            <option value="sine">sine</option>
            <option value="triangle">triangle</option>
            <option value="sawtooth">sawtooth</option>
            <option value="square">square</option>
          </select>
      </div>
 
      <Handle type="source" position="bottom" />
    </div>
  );
};

如果我们尝试渲染这个自定义节点，我们会发现输入没有任何作用。这是因为输入值由 data.frequency 和 data.type 固定，但我们没有监听更改的事件处理程序，也没有更新节点数据的机制！

¥If we try rendering this custom node we’ll find that the inputs don’t do anything. That’s because the input values are fixed by data.frequency and data.type but we have no event handlers listening to changes and no mechanism to update a node’s data!

要解决这种情况，我们需要跳回我们的存储并添加 updateNode 操作：

¥To fix the situation we need to jump back to our store and add an updateNode action:

export const useStore = createWithEqualityFn((set, get) => ({
  ...
 
  updateNode(id, data) {
    set({
      nodes: get().nodes.map(node =>
        node.id === id
          ? { ...node, data: { ...node.data, ...data } }
          : node
      )
    });
  },
 
  ...
}));

此操作将处理部分数据更新，例如，如果我们只想更新节点的 frequency，我们可以调用 updateNode(id, { frequency: 220 }。现在我们只需要将操作带入我们的 <Osc /> 组件，并在输入发生变化时调用它。

¥This action will handle partial data updates, such that if we only want to update a node’s frequency, for example, we could just call updateNode(id, { frequency: 220 }. Now we just need to bring the action into our <Osc /> component and call it whenever an input changes.

import React from 'react';
import { Handle } from '@xyflow/react';
import { shallow } from 'zustand/shallow';
 
import { useStore } from '../store';
 
const selector = (id) => (store) => ({
  setFrequency: (e) => store.updateNode(id, { frequency: +e.target.value }),
  setType: (e) => store.updateNode(id, { type: e.target.value }),
});
 
export default function Osc({ id, data }) {
  const { setFrequency, setType } = useStore(selector(id), shallow);
 
  return (
    <div>
      <div>
        <p>Oscillator Node</p>
 
        <label>
          <span>Frequency:</span>
          <input
            className="nodrag"
            type="range"
            min="10"
            max="1000"
            value={data.frequency}
            onChange={setFrequency}
          />
          <span>{data.frequency}Hz</span>
        </label>
 
        <label>
          <span>Waveform:</span>
          <select className="nodrag" value={data.type} onChange={setType}>
            <option value="sine">sine</option>
            <option value="triangle">triangle</option>
            <option value="sawtooth">sawtooth</option>
            <option value="square">square</option>
          </select>
        </label>
      </div>
 
      <Handle type="source" position="bottom" />
    </div>
  );
}

嘿，selector 回来了！请注意，这次我们如何使用它从一般 updateNode 操作中派生两个事件处理程序 setFrequency 和 setType。

¥Hey, that selector is back! Notice how this time we’re using it to derive two event handlers, setFrequency and setType, from the general updateNode action.

拼图的最后一块是告诉 React Flow 如何渲染我们的自定义节点。为此，我们需要创建一个 nodeTypes 对象：键应对应于节点的 type，值将是要渲染的 React 组件。

¥The last piece of the puzzle is to tell React Flow how to render our custom node. For that we need to create a nodeTypes object: the keys should correspond to a node’s type and the value will be the React component to render.

import React from 'react';
import { ReactFlow } from '@xyflow/react';
import { shallow } from 'zustand/shallow';
 
import { useStore } from './store';
import Osc from './nodes/Osc';
 
const selector = (store) => ({
  nodes: store.nodes,
  edges: store.edges,
  onNodesChange: store.onNodesChange,
  onEdgesChange: store.onEdgesChange,
  addEdge: store.addEdge,
});
 
const nodeTypes = {
  osc: Osc,
};
 
export default function App() {
  const store = useStore(selector, shallow);
 
  return (
    <ReactFlow
      nodes={store.nodes}
      nodeTypes={nodeTypes}
      edges={store.edges}
      onNodesChange={store.onNodesChange}
      onEdgesChange={store.onEdgesChange}
      onConnect={store.addEdge}
    >
      <Background />
    </ReactFlow>
  );
}

如果你已运行开发服务器，如果情况尚未发生变化，请不要惊慌！我们的临时节点尚未被赋予正确的类型，因此 React Flow 只会回退到渲染默认节点。如果我们将其中一个节点更改为 osc，并为 frequency 和 type 设置一些初始值，我们应该看到我们的自定义节点被渲染。

¥If you’ve got the dev server running, don’t panic if things haven’t changed yet! None of our temporary nodes have been given the right type yet, so React Flow just falls back to rendering the default node. If we change one of those nodes to be an osc with some initial values for frequency and type we should see our custom node being rendered.

const useStore = createWithEqualityFn((set, get) => ({
  nodes: [
    { type: 'osc',
      id: 'a',
      data: { frequency: 220, type: 'square' },
      position: { x: 0, y: 0 }
    },
    ...
  ],
  ...
}));

实现增益节点几乎是相同的过程，所以我们将把这个留给你。相反，我们将把注意力转向输出节点。此节点没有参数控制，但我们确实想打开和关闭信号处理。现在我们还没有实现任何音频代码，这有点困难，所以在此期间，我们只会在存储中添加一个标志和一个切换它的操作。

¥Implementing a gain node is pretty much the same process, so we’ll leave that one to you. Instead, we’ll turn our attention to the output node. This node will have no parameters control, but we do want to toggle signal processing on and off. That’s a bit difficult right now when we haven’t implemented any audio code yet, so in the meantime we’ll add just a flag to our store and an action to toggle it.

const useStore = createWithEqualityFn((set, get) => ({
  ...
 
  isRunning: false,
 
  toggleAudio() {
    set({ isRunning: !get().isRunning });
  },
 
  ...
}));

自定义节点本身非常简单：

¥The custom node itself is then pretty simple:

import React from 'react';
import { Handle } from '@xyflow/react';
import { shallow } from 'zustand/shallow';
import { useStore } from '../store';
 
const selector = (store) => ({
  isRunning: store.isRunning,
  toggleAudio: store.toggleAudio,
});
 
export default function Out({ id, data }) {
  const { isRunning, toggleAudio } = useStore(selector, shallow);
 
  return (
    <div>
      <Handle type="target" position="top" />
 
      <div>
        <p>Output Node</p>
 
        <button onClick={toggleAudio}>
          {isRunning ? (
            <span role="img" aria-label="mute">
              🔇
            </span>
          ) : (
            <span role="img" aria-label="unmute">
              🔈
            </span>
          )}
        </button>
      </div>
    </div>
  );
}

事情开始变得相当顺利！

¥Things are starting to shape up quite nicely!

那么，下一步就是……

¥The next step, then, is to…

听听它的声音

¥Do sound to it

我们有一个交互式图表，我们能够更新节点数据，现在让我们添加我们对 Web Audio API 的了解。首先创建一个新文件 audio.js，然后创建一个新的音频上下文和一个空的 Map。

¥We have an interactive graph and we’re able to update node data, now let’s add in what we know about the Web Audio API. Start by creating a new file, audio.js, and create a new audio context and an empty Map.

const context = new AudioContext();
const nodes = new Map();

我们管理音频图的方式是通过挂接到我们存储中的不同操作。因此，当调用 addEdge 操作时，我们可能会连接两个音频节点，或者在调用 updateNode 时更新音频节点的属性，等等。

¥The way we’ll manage our audio graph is by hooking into the different actions in our store. So we might connect two audio nodes when the addEdge action is called, or update an audio node’s properties when updateNode is called, and so on.

const context = new AudioContext();
const nodes = new Map();
 
const osc = context.createOscillator();
osc.frequency.value = 220;
osc.type = 'square';
osc.start();
 
const amp = context.createGain();
amp.gain.value = 0.5;
 
const out = context.destination;
 
nodes.set('a', osc);
nodes.set('b', amp);
nodes.set('c', out);

1. 节点更改

¥ Node changes

现在，我们的图表中可能发生两种类型的节点更改，我们需要对此做出响应：更新节点的 data，并从图中删除节点。我们已经有一个针对前者的操作，所以让我们先处理一下。

¥Right now, there are two types of node changes that can happen in our graph and that we need to respond to: updating a node’s data, and removing a node from the graph. We already have an action for the former, so let’s handle that first.

在 audio.js 中，我们将定义一个函数 updateAudioNode，我们将使用节点的 id 和部分 data 对象来调用该函数，并使用它来更新 Map 中的现有节点：

¥In audio.js we’ll define a function, updateAudioNode, that we’ll call with a node’s id and a partial data object and use it to update an existing node in the Map:

export function updateAudioNode(id, data) {
  const node = nodes.get(id);
 
  for (const [key, val] of Object.entries(data)) {
    if (node[key] instanceof AudioParam) {
      node[key].value = val;
    } else {
      node[key] = val;
    }
  }
}

现在我们要更新存储中的 updateNode 操作以在更新过程中调用此函数：

¥Now we’ll want to update our updateNode action in the store to call this function as part of the update:

import { updateAudioNode } from './audio';
 
export const useStore = createWithEqualityFn((set, get) => ({
  ...
 
  updateNode(id, data) {
    updateAudioNode(id, data);
    set({ nodes: ... });
  },
 
  ...
}));
 

我们需要处理的下一个更改是从图中删除一个节点。如果你在图表中选择一个节点并按退格键，React Flow 会将其删除。我们连接的 onNodesChange 动作已为我们隐式处理了这一点，但现在我们想要一些额外的处理，我们需要将新动作连接到 React Flow 的 onNodesDelete 事件。

¥The next change we need to handle is removing a node from the graph. If you select a node in the graph and hit backspace, React Flow will remove it. This is implicitly handled for us by the onNodesChange action we hooked up, but now we want some additional handling we’ll need to wire up a new action to React Flow’s onNodesDelete event.

这实际上非常简单，所以我将为你节省一些阅读时间，并在不注释的情况下展示接下来的三个代码片段。

¥This is actually pretty simple, so I’ll save you some reading and present the next three snippets of code without comment.

export function removeAudioNode(id) {
  const node = nodes.get(id);
 
  node.disconnect();
  node.stop?.();
 
  nodes.delete(id);
}

import { ..., removeAudioNode } from './audio';
 
export const useStore = createWithEqualityFn((set, get) => ({
  ...
 
  removeNodes(nodes) {
    for (const { id } of nodes) {
      removeAudioNode(id)
    }
  },
 
  ...
}));

const selector = store => ({
  ...,
  onNodesDelete: store.removeNodes
});
 
export default function App() {
  const store = useStore(selector, shallow);
 
  return (
    <ReactFlow
      onNodesDelete={store.onNodesDelete}
      ...
    >
      <Background />
    </ReactFlow>
  )
};

唯一需要注意的是，onNodesDelete 使用已删除节点数组调用提供的回调，因为可以一次删除多个节点！

¥The only thing to note is that onNodesDelete calls the provided callback with an array of deleted nodes, because it is possible to delete more than one node at once!

2. 边缘变化

¥ Edge changes

我们非常接近真正发出一些声音了！剩下的就是处理图形边缘的变化。与节点更改一样，我们已经有一个操作来处​​理创建新边，并且我们还隐式处理 onEdgesChange 中已删除的边。

¥We’re getting super close to actually making some sounds! All that’s left is to handle changes to our graph’s edges. Like with node changes, we already have an action to handle creating new edges and we’re also implicitly handling removed edges in onEdgesChange.

要处理新连接，我们只需要在我们的 addEdge 操作中创建的边缘的 source 和 target ID。然后我们可以查找 Map 中的两个节点并将它们连接起来。

¥To handle new connections, we just need the source and target ids from the edge created in our addEdge action. Then we can just look up the two nodes in our Map and connect them up.

export function connect(sourceId, targetId) {
  const source = nodes.get(sourceId);
  const target = nodes.get(targetId);
 
  source.connect(target);
}

import { ..., connect } from './audio';
 
export const useStore = createWithEqualityFn((set, get) => ({
  ...
 
  addEdge(data) {
    ...
 
    connect(data.source, data.target);
  },
 
  ...
}));

我们看到 React Flow 接受了 onNodesDelete 处理程序，但你不知道吗，它还有一个 onEdgesDelete 处理程序！我们实现 disconnect 并将其连接到我们的存储和 React Flow 实例的方法与以前几乎相同，所以我们也将这一个留给你！

¥We saw React Flow accepted an onNodesDelete handler and wouldn’t you know it, there’s an onEdgesDelete handler too! The approach we’d take to implement disconnect and hook it up to our store and React Flow instance is pretty much the same as before, so we’ll leave that one down to you as well!

3. 打开扬声器

¥ Switching the speakers on

你会记得，我们的 AudioContext 可能以暂停状态开始，以防止可能令人讨厌的自动播放问题。我们已经在存储中伪造了 <Out /> 组件所需的数据和操作，现在我们只需要用真实上下文的状态和恢复/暂停方法替换它们。

¥You’ll remember that our AudioContext probably begins in a suspended state to prevent potentially annoying autoplay issues. We already faked the data and actions we need for our <Out /> component in the store, now we just need to replace them with the real context’s state and resume/suspend methods.

export function isRunning() {
  return context.state === 'running';
}
 
export function toggleAudio() {
  return isRunning() ? context.suspend() : context.resume();
}

虽然到目前为止我们还没有从音频函数返回任何东西，但我们需要从 toggleAudio 返回，因为这些方法是异步的，我们不想过早更新存储！

¥Although we haven’t been returning anything from our audio functions up until now, we need to return from toggleAudio because those methods are asynchronous and we don’t want to update the store prematurely!

import { ..., isRunning, toggleAudio } from './audio'
 
export const useStore = createWithEqualityFn((set, get) => ({
  ...
 
  isRunning: isRunning(),
 
  toggleAudio() {
    toggleAudio().then(() => {
      set({ isRunning: isRunning() });
    });
  }
}));

瞧，我们做到了！我们现在已经把足够多的东西放在一起，可以真正发出声音了！让我们看看我们的实际操作。

¥Et voilà, we did it! We’ve now put enough together to actually make sounds! Let’s see what we have in action.

4. 创建新节点

¥ Creating new nodes

到目前为止，我们一直在处理图中的一组硬编码节点。这对于原型设计来说已经很好了，但为了使其真正有用，我们需要一种动态地向图中添加新节点的方法。我们的最终任务是添加此功能：我们将从音频代码开始反向工作，最后创建一个基本的工具栏。

¥Up until now we have been dealing with a hard-coded set of nodes in our graph. This has been fine for prototyping but for it to actually be useful we’ll want a way to add new nodes to the graph dynamically. Our final task will be adding this functionality: we’ll work backwards starting with the audio code and ending by creating a basic toolbar.

实现 createAudioNode 函数将非常简单。我们需要的是新节点的 ID、要创建的节点类型及其初始数据：

¥Implementing a createAudioNode function will be simple enough. All we need is an id for the new node, the type of node to create, and its initial data:

export function createAudioNode(id, type, data) {
  switch (type) {
    case 'osc': {
      const node = context.createOscillator();
      node.frequency.value = data.frequency;
      node.type = data.type;
      node.start();
 
      nodes.set(id, node);
      break;
    }
 
    case 'amp': {
      const node = context.createGain();
      node.gain.value = data.gain;
 
      nodes.set(id, node);
      break;
    }
  }
}

接下来，我们需要在存储中有一个 createNode 函数。节点 ID 将由 nanoid 生成，我们将为每种节点类型硬编码一些初始数据，因此我们唯一需要传入的是要创建的节点类型：

¥Next we’ll need a createNode function in our store. The node id will be generated by nanoid and we’ll hardcode some initial data for each of the node types, so the only thing we need to pass in is the type of node to create:

import { ..., createAudioNode } from './audio';
 
export const useStore = createWithEqualityFn((set, get) => ({
  ...
 
  createNode(type) {
    const id = nanoid();
 
    switch(type) {
      case 'osc': {
        const data = { frequency: 440, type: 'sine' };
        const position = { x: 0, y: 0 };
 
        createAudioNode(id, type, data);
        set({ nodes: [...get().nodes, { id, type, data, position }] });
 
        break;
      }
 
      case 'amp': {
        const data = { gain: 0.5 };
        const position = { x: 0, y: 0 };
 
        createAudioNode(id, type, data);
        set({ nodes: [...get().nodes, { id, type, data, position }] });
 
        break;
      }
    }
  }
}));

我们可以更聪明地计算新节点的位置，但为了简单起见，我们现在只将其硬编码为 { x: 0, y: 0 }。

¥We could be a bit smarter about calculating the position of the new node, but to keep things simple we’ll just hardcode it to { x: 0, y: 0 } for now.

拼图的最后一块是创建一个可以触发新 createNode 操作的工具栏组件。为此，我们将跳回到 App.jsx 并使用 <Panel /> 插件组件。

¥The final piece of the puzzle is to create a toolbar component that can trigger the new createNode action. To do that we’ll jump back to App.jsx and make use of the <Panel /> plugin component.

...
import { ReactFlow,  Panel } from '@xyflow/react';
...
 
const selector = (store) => ({
  ...,
  createNode: store.createNode,
});
 
export default function App() {
  const store = useStore(selector, shallow);
 
  return (
    <ReactFlow>
      <Panel position="top-right">
        ...
      </Panel>
      <Background />
    </ReactFlow>
  );
};

我们这里不需要任何花哨的东西，只需要几个按钮就可以使用适当的类型触发 createNode 操作：

¥We don’t need anything fancy here, just a couple of buttons that trigger the createNode action with the appropriate type:

<Panel position="top-right">
  <button onClick={() => store.createNode('osc')}>osc</button>
  <button onClick={() => store.createNode('amp')}>amp</button>
</Panel>

这就是…一切！我们现在有一个功能齐全的音频图形编辑器，它可以：

¥And that’s… everything! We’ve now got a fully functional audio graph editor that can:

• 创建新的音频节点

  ¥Create new audio nodes

• 使用一些 UI 控件更新节点数据

  ¥Update node data with some UI controls

• 将节点连接在一起

  ¥Connect nodes together

• 删除节点和连接

  ¥Delete nodes and connections

• 启动和停止音频处理

  ¥Start and stop audio processing

以下是从头开始的演示，但这次你可以查看源代码以确保你没有遗漏任何内容。

¥Here’s the demo from the beginning, but this time you can see the source code to make sure you haven’t missed anything.

import React from 'react';
import { ReactFlow, ReactFlowProvider, Background, Panel } from '@xyflow/react';
import { shallow } from 'zustand/shallow';
import { tw } from 'twind';
 
import { useStore } from './store';
import Osc from './nodes/Osc';
import Amp from './nodes/Amp';
import Out from './nodes/Out';
 
import '@xyflow/react/dist/style.css';
 
const nodeTypes = {
  osc: Osc,
  amp: Amp,
  out: Out,
};
 
const selector = (store) => ({
  nodes: store.nodes,
  edges: store.edges,
  onNodesChange: store.onNodesChange,
  onNodesDelete: store.onNodesDelete,
  onEdgesChange: store.onEdgesChange,
  onEdgesDelete: store.onEdgesDelete,
  addEdge: store.addEdge,
  createNode: store.createNode,
});
 
export default function App() {
  const store = useStore(selector, shallow);
 
  return (
    <ReactFlowProvider>
      <div style={{ width: '100vw', height: '100vh' }}>
        <ReactFlow
          nodeTypes={nodeTypes}
          nodes={store.nodes}
          edges={store.edges}
          onNodesChange={store.onNodesChange}
          onNodesDelete={store.onNodesDelete}
          onEdgesChange={store.onEdgesChange}
          onEdgesDelete={store.onEdgesDelete}
          onConnect={store.addEdge}
          fitView
        >
          <Panel className={tw('space-x-4')} position="top-right">
            <button
              className={tw('px-2 py-1 rounded bg-white shadow')}
              onClick={() => store.createNode('osc')}
            >
              Add Osc
            </button>
            <button
              className={tw('px-2 py-1 rounded bg-white shadow')}
              onClick={() => store.createNode('amp')}
            >
              Add Amp
            </button>
          </Panel>
          <Background />
        </ReactFlow>
      </div>
    </ReactFlowProvider>
  );
}

最后的想法

¥Final thoughts

哇，这很长，但我们做到了！通过我们的努力，我们找到了一个有趣的小型交互式音频在线运行，在此过程中了解了一些有关 Web Audio API 的知识，并且对 “running” React Flow 图的一种方法有了更好的了解。

¥Whew that was a long one, but we made it! For our efforts we’ve come out the other side with a fun little interactive audio playground, learned a little bit about the Web Audio API along the way, and have a better idea of one approach to “running” a React Flow graph.

如果你已经做到了这一点，并且正在想“Hayleigh，我永远不会编写 Web Audio 应用。我学到了什么有用的东西吗？”那你真幸运，因为你确实学到了！你可以采用我们的方法连接到 Web Audio API，并将其应用于其他基于图形的计算引擎，如 behave-graph 。事实上，有些人就是这么做的，并创建了 behave-flow ！

¥If you’ve made it this far and are thinking “Hayleigh, I’m never going to write a Web Audio app. Did I learn anything useful?” Then you’re in luck, because you did! You could take our approach to connecting to the Web Audio API and apply it to some other graph-based computation engine like behave-graph . In fact, some has done just that and created behave-flow !

仍有很多方法可以扩展这个项目。如果你想继续研究它，这里有一些想法：

¥There are still plenty of ways to expand on this project. If you’d like to keep working on it, here are some ideas:

• 添加更多节点类型。

  ¥Add more node types.

• 允许节点连接到其他节点上的 AudioParams。

  ¥Allow nodes to connect to AudioParams on other nodes.

• 使用 AnalyserNode 可视化节点或信号的输出。

  ¥Use the AnalyserNode to visualize the output of a node or signal.

• 你能想到的任何其他内容！

  ¥Anything else you can think of!

如果你正在寻找灵感，那么外面有不少项目正在使用基于节点的 UI 来处理音频。我最喜欢的是 Max/MSP 、Reaktor  和 Pure Data 。Max 和 Reaktor 是闭源商业软件，但你仍然可以从它们 🕵️ 中窃取一些想法。

¥And if you’re looking for inspiration, there are quite a few projects out in the wild that are using node-based UIs for audio things. Some of my favorites are Max/MSP , Reaktor , and Pure Data . Max and Reaktor are closed-source commercial software, but you can still steal some ideas from them 🕵️.

你可以使用已完成的 源代码  作为起点，也可以继续在我们今天所做的基础上进行构建。我们很乐意看到你构建的内容，因此请在我们的 Discord 服务器  或 Twitter  上与我们分享。

¥You can use the completed source code  as a starting point, or you can just keep building on top of what we’ve made today. We’d love to see what you build so please share it with us over on our Discord server  or Twitter .

React Flow 是一家由用户资助的独立公司。如果你想支持我们，可以使用 在 Github 上赞助我们  或 订阅我们的 Pro 计划之一 。

¥React Flow is an independent company financed by its users. If you want to support us you can sponsor us on Github  or subscribe to one of our Pro plans .