将 Firecrawl 与 LangChain 集成，构建由网页数据驱动的 AI 应用。
安装与设置#
npm install @langchain/openai firecrawl 创建 .env 文件：
FIRECRAWL_API_KEY=your_firecrawl_keyOPENAI_API_KEY=your_openai_key注意： 如果使用 Node 版本低于 20，请安装
dotenv，并在代码中添加import 'dotenv/config'。
抓取 + 对话#
本示例展示一个简单的工作流：抓取网站，并使用 LangChain 处理抓取到的内容。
import { Firecrawl } from 'firecrawl';import { ChatOpenAI } from '@langchain/openai';import { HumanMessage } from '@langchain/core/messages';const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });const chat = new ChatOpenAI({ model: 'gpt-5-nano', apiKey: process.env.OPENAI_API_KEY});const scrapeResult = await firecrawl.scrape('https://firecrawl.dev', { formats: ['markdown']});console.log('Scraped content length:', scrapeResult.markdown?.length);const response = await chat.invoke([ new HumanMessage(`Summarize: ${scrapeResult.markdown}`)]);console.log('Summary:', response.content);Chains#
本示例演示如何构建一个 LangChain 链，用于处理和分析抓取到的内容。
import { Firecrawl } from 'firecrawl';import { ChatOpenAI } from '@langchain/openai';import { ChatPromptTemplate } from '@langchain/core/prompts';import { StringOutputParser } from '@langchain/core/output_parsers';const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });const model = new ChatOpenAI({ model: 'gpt-5-nano', apiKey: process.env.OPENAI_API_KEY});const scrapeResult = await firecrawl.scrape('https://stripe.com', { formats: ['markdown']});console.log('Scraped content length:', scrapeResult.markdown?.length);// 创建处理链const prompt = ChatPromptTemplate.fromMessages([ ['system', 'You are an expert at analyzing company websites.'], ['user', 'Extract the company name and main products from: {content}']]);const chain = prompt.pipe(model).pipe(new StringOutputParser());// 执行链const result = await chain.invoke({ content: scrapeResult.markdown});console.log('Chain result:', result);工具调用#
此示例演示如何使用 LangChain 的工具调用功能，让模型自动决定何时抓取网站。
import { Firecrawl } from 'firecrawl';import { ChatOpenAI } from '@langchain/openai';import { DynamicStructuredTool } from '@langchain/core/tools';import { z } from 'zod';const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });// 创建抓取工具const scrapeWebsiteTool = new DynamicStructuredTool({ name: 'scrape_website', description: 'Scrape content from any website URL', schema: z.object({ url: z.string().url().describe('The URL to scrape') }), func: async ({ url }) => { console.log('Scraping:', url); const result = await firecrawl.scrape(url, { formats: ['markdown'] }); console.log('Scraped content preview:', result.markdown?.substring(0, 200) + '...'); return result.markdown || 'No content scraped'; }});const model = new ChatOpenAI({ model: 'gpt-5-nano', apiKey: process.env.OPENAI_API_KEY}).bindTools([scrapeWebsiteTool]);const response = await model.invoke('What is Firecrawl? Visit firecrawl.dev and tell me about it.');console.log('Response:', response.content);console.log('Tool calls:', response.tool_calls);结构化数据提取#
此示例演示如何使用 LangChain 的结构化输出功能来提取结构化数据。
import { Firecrawl } from 'firecrawl';import { ChatOpenAI } from '@langchain/openai';import { z } from 'zod';const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });const scrapeResult = await firecrawl.scrape('https://stripe.com', { formats: ['markdown']});console.log('Scraped content length:', scrapeResult.markdown?.length);const CompanyInfoSchema = z.object({ name: z.string(), industry: z.string(), description: z.string(), products: z.array(z.string())});const model = new ChatOpenAI({ model: 'gpt-5-nano', apiKey: process.env.OPENAI_API_KEY}).withStructuredOutput(CompanyInfoSchema);const companyInfo = await model.invoke([ { role: 'system', content: 'Extract company information from website content.' }, { role: 'user', content: `Extract data: ${scrapeResult.markdown}` }]);console.log('Extracted company info:', companyInfo);更多示例，请参阅 LangChain 文档。