web-reader

Implement web page content extraction capabilities using the z-ai-web-dev-sdk. Use this skill when the user needs to scrape web pages, extract article content, retrieve page metadata, or build applications that process web content. Supports automatic content extraction with title, HTML, and publication time retrieval.

26stars15forksUpdated 7/16/2026

#metadata #sdk #web-scraping #cli #content-extraction #javascript

Security Assessment

Medium Risk(50/100)

Detected risks:

Remote Code Execution([SKILL.md] exec()

Security Score50/100

About web-reader

Web-reader guides the implementation of web page content extraction using the z-ai-web-dev-sdk package, enabling applications to fetch and process web page content programmatically. It supports automatic content extraction returning the title, HTML, plain text, and publication time, and is intended for scraping web pages, extracting article content, retrieving page metadata, or building applications that process web content. The SDK handles content extraction automatically, providing clean, structured data from any web URL.

The skill offers two paths. For simple tasks, the z-ai CLI runs the page_reader function without writing code, for example z-ai function --name "page_reader" --args '{"url": "https://example.com"}', with -o to save output to a JSON file; CLI parameters are --name/-n (use page_reader), --args/-a (a JSON object with a required url string), and --output/-o. The CLI response includes title, html, text, publish_time, url, and metadata. For programmatic use, the SDK is imported as ZAI from z-ai-web-dev-sdk, initialized with ZAI.create(), and invoked via zai.functions.invoke('page_reader', { url }), returning data with fields such as title, url, publishedTime, html, and usage.tokens. The skill provides code examples for simple page reading, extracting article text only (stripping HTML tags), reading multiple pages in a loop with per-URL success and error handling, and an advanced WebContentAnalyzer class that caches results and computes page metadata like content length and word count. It also shows a shell loop for processing multiple URLs from the CLI. The skill recommends the CLI for quick extraction and testing and the SDK for batch processing, web-application integration, complex pipelines, and production applications with error handling.

The skill targets developers building web-content extraction into applications. A stated requirement is that z-ai-web-dev-sdk must be used in backend code only and never in client-side code. The package is assumed to be already installed, and reference test scripts, including scripts/web-reader.ts, are provided for quick testing. The skill is MIT licensed.

FAQ

What does the page_reader function return?

It returns structured data including the page title, main content HTML, plain text, publication time (if available), the original URL, and additional metadata such as author and description. In the SDK, the result data also includes fields like publishedTime and usage.tokens.

Can I use it without writing code?

Yes. For simple tasks the z-ai CLI runs the page_reader function directly, for example z-ai function --name "page_reader" --args '{"url": "https://example.com"}'. Add --output/-o to save the extracted content to a JSON file. Required parameters are --name/-n (page_reader) and --args/-a (a JSON object with a required url).

When should I use the CLI versus the SDK?

Use the CLI for quick content extraction, testing URL accessibility, simple scraping, and one-off retrieval. Use the SDK for batch URL processing with custom logic, integration with web applications, complex content-processing pipelines, and production applications with error handling.

Are there any usage restrictions?

Yes. The z-ai-web-dev-sdk must be used in backend code only and never in client-side code. The package is assumed to be already installed, and it is imported as ZAI from z-ai-web-dev-sdk.

How do I read many pages at once?

The skill shows a readMultiplePages SDK function that loops over URLs and records per-URL success or error, and a shell loop that calls the CLI for each URL, saving each result to a JSON file named from the URL's hash.