Power your agent
Yayıncının kendi komutu. PlugMCP uç noktası değildir.
Host a copy with our keys and logs.
mcpb https://github.com/PxyUp/fitter/releases/download/v1.8.3/fitter-mcp-darwin-arm64.mcpbClaude Code'un yerel komutlar için kurulum bağlantısı yok. Aşağıdaki komutu kopyalayın ve bir terminalde çalıştırın.
FITTER_PLUGINS=<value> claude mcp add io.github.pxyup-fitter -- mcpb https://github.com/PxyUp/fitter/releases/download/v1.8.3/fitter-mcp-darwin-arm64.mcpbBir terminalde çalıştırın.
{ "mcpServers": { "io.github.pxyup-fitter": { "command": "mcpb", "args": [ "https://github.com/PxyUp/fitter/releases/download/v1.8.3/fitter-mcp-darwin-arm64.mcpb" ], "env": { "FITTER_PLUGINS": "<value>" } } }}claude_desktop_config.json
Uzak URL mi, yoksa makinenizde bir komut mu? İki kurulum arasındaki fark
PlugMCP’de çalıştır
A new server you create. It is not this listing's official endpoint. Use the publisher install when you want their URL and their credentials.
BaşlatFitter turns any website or API into structured JSON — declaratively. One JSON/YAML config describes where the data lives (HTTP request, headless browser, file, static value) and what to extract (JSON paths, CSS selectors, XPath). No code, no brittle scraping scripts.
🚀 Try it in your browser — the real engine compiled to WebAssembly: live examples, a visual config builder, no install.
Because configs are plain data, LLMs can author them. The built-in MCP server lets Claude Code, Claude Desktop, or any MCP client write and run scraping pipelines on your machine, on demand:
"Get the top 5 HackerNews stories with titles and scores" → the model authors a fitter config, validates it, runs it locally, and gets clean JSON back.
One engine, five ways to use it:
Why fitter for AI agents?
Fitter MCP is a Model Context Protocol server (stdio transport) which lets any MCP client — Claude Code, Claude Desktop, IDE assistants, custom agents — run Fitter configs and get structured JSON back.
Download fitter-mcp-<os>-<arch>.mcpb from the release page and open it — Claude Desktop installs the server automatically.
# 1. get the binary: download fitter_mcp_<version>-<os>-<arch> from the release page
# https://github.com/PxyUp/fitter/releases — or build it from source:
go build -o fitter_mcp ./cmd/mcp
# 2. register it once, available in every project
claude mcp add fitter -s user -- "$(pwd)/fitter_mcp"
Then just ask:
Get the top 5 HackerNews stories with titles and scores using fitter
The model calls fitter_config_reference, authors a config, optionally checks it with fitter_validate_config and executes it via fitter_run — all data fetching happens locally on your machine. For a ready-made pipeline try examples/config_morning_briefing.json:
Run examples/config_morning_briefing.json with fitter and give me the briefing
{
"mcpServers": {
"fitter": {
"command": "/path/to/fitter_mcp"
}
}
}
The .mcpb bundle and native binary ship without browsers: HTTP, static and file connectors work out of the box, but browser configs (the playwright connector) need Playwright's browsers. A few ways to get them:
.mcpb): set "install": true in the playwright connector — fitter downloads the driver + browser matching its built-in playwright-go version on first use (one-time, cached), so no separate install step is needed.playwright-go version fitter is built against (check go.mod, currently v0.6100.0):
go run github.com/mxschmitt/playwright-go/cmd/[email protected] install
# Linux: append --with-deps to also install the required OS libraries
go.mod exactly — playwright-go refuses to run against a mismatched driver. Then run configs without "install": true.ghcr.io/pxyup/fitter-mcp:playwright image, which bundles Chromium, Firefox and WebKit preinstalled (no "install": true needed).The reference is also exposed as MCP resource fitter://config-reference for clients that support resources.
The config format is exactly the same as for Fitter_CLI: a top-level object with item (required), limits and references. Notifiers work too (the result is additionally pushed to http/telegram/redis/file/console); trigger_config and http_server are service-mode only and are ignored in MCP calls.
By default fitter_mcp talks stdio. Pass --http to serve the streamable HTTP transport instead — for a shared team server, a container, or any remote deployment:
# serve MCP at http://<host>:8080/mcp (health probe at /healthz)
FITTER_MCP_AUTH_TOKEN=my-secret fitter_mcp --http :8080
# register the remote endpoint in Claude Code
claude mcp add --transport http fitter http://localhost:8080/mcp --header "Authorization: Bearer my-secret"
--http <addr> (env FITTER_MCP_HTTP_ADDR) — listen address; stdio mode when emptyFITTER_MCP_AUTH_TOKEN — when set, every /mcp request must send Authorization: Bearer <token>; without it the endpoint is unauthenticated, so bind to localhost or put it behind a proxy--stateless (env FITTER_MCP_STATELESS=true) — no per-session state, so replicas can sit behind a load balancer without sticky sessions--metrics-addr <addr> (env FITTER_MCP_METRICS_ADDR) — serve Prometheus metrics on a separate address (e.g. 127.0.0.1:9091); works in stdio mode too; empty disables. Exposes MCP request counts by method, tool call durations by tool and outcome, and outbound HTTP request durations by host, method and status. The endpoint is unauthenticated — bind it to localhost or a private interfaceThe server shuts down gracefully on SIGINT/SIGTERM.
A slim multi-arch image (linux/amd64 + linux/arm64) ships with every release:
# hosted HTTP mode
docker run --rm -p 8080:8080 \
-e FITTER_MCP_HTTP_ADDR=:8080 \
-e FITTER_MCP_AUTH_TOKEN=my-secret \
ghcr.io/pxyup/fitter-mcp:latest
# or stdio mode, spawned by the MCP client
claude mcp add fitter -s user -- docker run --rm -i ghcr.io/pxyup/fitter-mcp:latest
The slim image contains only the fitter binary and CA certificates: server/static/file connectors work, browser connectors (chromium/docker/playwright) do not.
For browser-based configs use the playwright variant, which bundles Playwright with Chromium, Firefox and WebKit (matched to the playwright-go version fitter is built against, so no "install": true is needed in configs):
docker run --rm -i ghcr.io/pxyup/fitter-mcp:playwright # stdio mode
# per-release tag: ghcr.io/pxyup/fitter-mcp:vX.Y.Z-playwright
It is built from Dockerfile.mcp-playwright; build with --build-arg PLAYWRIGHT_BROWSERS=chromium for a smaller Chromium-only image.
Both images ship fitter_cli, so the one-time OAuth2 login can run inside the container. Store the token on a volume mounted at /tokens (pre-created writable in the image) and share it with the MCP server:
# one-time login, device flow: no ports needed — open the printed url on any device
docker run --rm -it -v fitter-tokens:/tokens --entrypoint fitter_cli \
ghcr.io/pxyup/fitter-mcp:latest \
auth --provider github --client-id <ID> --client-secret <SECRET> --token-file /tokens/github.json
# or browser flow (device flow not enabled for the app): publish the callback port and
# bind on 0.0.0.0 so the published port reaches the listener; the browser still visits 127.0.0.1
docker run --rm -it -p 8988:8988 -e FITTER_AUTH_LISTEN=0.0.0.0 \
-v fitter-tokens:/tokens --entrypoint fitter_cli ghcr.io/pxyup/fitter-mcp:latest \
auth --provider github --client-id <ID> --client-secret <SECRET> --token-file /tokens/github.json
# then run the MCP server with the same volume; configs reference "token_file": "/tokens/github.json"
# stdio mode (spawned by the MCP client, no port):
docker run --rm -i -v fitter-tokens:/tokens ghcr.io/pxyup/fitter-mcp:latest
# hosted HTTP mode (MCP endpoint on 8080, like the run examples above):
docker run --rm -p 8080:8080 -v fitter-tokens:/tokens \
-e FITTER_MCP_HTTP_ADDR=:8080 \
-e FITTER_MCP_AUTH_TOKEN=my-secret \
ghcr.io/pxyup/fitter-mcp:latest
Note: 8988 is only for the one-time browser-flow login; the MCP server itself needs no port in stdio mode and only 8080 in hosted HTTP mode.
Browser sessions need the playwright image (the slim one has no browsers). The one-time headed login needs a display, so run it on the host, then bind-mount the session dir into the container (the image pre-creates a writable /sessions):
# on the host: log in once, save the session
fitter_cli browser-login --url https://example.com/login --storage-state ~/.fitter/sessions/example.json
# run the MCP server with the sessions dir mounted; configs reference "storage_state_file": "/sessions/example.json"
docker run --rm -i -v ~/.fitter/sessions:/sessions ghcr.io/pxyup/fitter-mcp:playwright
Use a bind mount (not a named volume): the container writes refreshed cookies back after every run, so the host copy stays current and can be re-extended with browser-login at any time.
The volume must stay writable for the server: rotated refresh tokens are written back on every refresh.
--plugins flag of Fitter/Fitter_CLI--http--stateless--metrics-addrComplete, tested configs showing the main patterns. All of them run unchanged via Fitter_MCP (fitter_run_file), Fitter_CLI or the library — more in examples/.
GitHub trending has no official API — scrape the HTML for repo slugs (html_attribute reads the href), then fan each one out into the GitHub REST API with {PL}:
{
"item": {
"connector_config": {
"response_type": "HTML",
"url": "https://github.com/trending",
"server_config": { "method": "GET", "headers": { "User-Agent": "Mozilla/5.0 (fitter demo)" } }
},
"model": {
"array_config": {
"root_path": "article.Box-row h2 a",
"length_limit": 5,
"item_config": {
"field": {
"type": "string",
"html_attribute": "href",
"generated": { "model": {
"type": "object",
"connector_config": {
"response_type": "json",
"url": "https://api.github.com/repos{PL}",
"server_config": { "method": "GET", "headers": { "User-Agent": "fitter-demo" } },
"null_on_error": true
},
"model": { "object_config": { "fields": {
"repo": { "base_field": { "type": "string", "path": "full_name" } },
"stars": { "base_field": { "type": "int", "path": "stargazers_count" } },
"language": { "base_field": { "type": "string", "path": "language" } }
} } }
} }
}
}
}
}
},
"limits": { "host_request_limiter": { "api.github.com": 2 } }
}
[{"repo": "block/buzz", "stars": 6214, "language": "Rust"}, {"repo": "koala73/worldmonitor", "stars": 71179, "language": "TypeScript"}]
When array items are objects, the join key lives inside them — pull it out with {{{FromExp=...}}} (expr-lang over fRes, the current item). Book search → author details, search query supplied via input:
"url": "https://openlibrary.org/authors/{{{FromExp=fromJSON(fRes).author_key[0]}}}.json"
./fitter_cli --path=examples/config_book_authors.json --input=dune
[{"title": "Dune", "year": 1965, "author": {"name": "Frank Herbert", "born": "8 October 1920", "died": "11 February 1986"}}]
The file_storage generated field turns fields into writes — top-5 crypto coins appended to a CSV, one row per item. Bare {{{json.path}}} placeholders read the current item; {HUMAN_INDEX} stamps the 1-based rank (items are processed in parallel, so appends land in completion order — sort by the rank column):
"file_storage": {
"content": "{HUMAN_INDEX},{{{name}}},{{{current_price}}},{{{price_change_percentage_24h}}}\n",
"file_name": "coins.csv",
"path": "/tmp/fitter-report",
"append": true
}
$ sort -n /tmp/fitter-report/coins.csv
1,Bitcoin,64778,-2.3
2,Ethereum,1881.01,-3.4
3,Tether,0.999265,0
response_type: "pdf" turns any fetched PDF into a JSON document — {"text": "...", "pages": ["..."], "total_pages": N} — so regular JSON paths (text, pages.0) and expressions work on it. The Bitcoin whitepaper, page count plus a trimmed intro:
{
"item": {
"connector_config": {
"response_type": "pdf",
"url": "https://bitcoin.org/bitcoin.pdf",
"server_config": { "method": "GET" }
},
"model": {
"object_config": {
"fields": {
"total_pages": { "base_field": { "type": "int", "path": "total_pages" } },
"intro": {
"base_field": {
"type": "string",
"path": "pages.0",
"generated": {
"calculated": {
"type": "string",
"expression": "trim(fRes[:100]) + \"...\""
}
}
}
}
}
}
}
}
}
{"intro": "Bitcoin: A Peer-to-Peer Electronic Cash SystemSatoshi [email protected]...", "total_pages": 9}
{"text": "...", "pages": ["..."], "total_pages": N} so regular JSON paths like text or pages.0 workgo get github.com/PxyUp/fitter
package main
import (
"fmt"
"github.com/PxyUp/fitter/lib"
"github.com/PxyUp/fitter/pkg/config"
"log"
"net/http"
)
func main() {
res, err := lib.Parse(&config.Item{
ConnectorConfig: &config.ConnectorConfig{
ResponseType: config.Json,
Url: "https://random-data-api.com/api/appliance/random_appliance",
ServerConfig: &config.ServerConnectorConfig{
Method: http.MethodGet,
},
},
Model: &config.Model{
ObjectConfig: &config.ObjectConfig{
Fields: map[string]*config.Field{
"my_id": {
BaseField: &config.BaseField{
Type: config.Int,
Path: "id",
},
},
"generated_id": {
BaseField: &config.BaseField{
Generated: &config.GeneratedFieldConfig{
UUID: &config.UUIDGeneratedFieldConfig{},
},
},
},
"generated_array": {
ArrayConfig: &config.ArrayConfig{
RootPath: "@this|@keys",
ItemConfig: &config.ObjectConfig{
Field: &config.BaseField{
Type: config.String,
},
},
},
},
},
},
},
}, nil, nil, nil, nil)
if err != nil {
log.Fatal(err)
}
fmt.Println(res.ToJson())
}
Output:
{
"generated_array": ["id","uid","brand","equipment"],
"my_id": 6000,
"generated_id": "26b08b73-2f2e-444d-bcf2-dac77ac3130e"
}
Use lib.ParseCtx(ctx, ...) to pass a context.Context: cancelling it aborts in-flight fetches (HTTP requests, headless browsers, docker containers) and applies deadlines end-to-end. lib.Parse is equivalent to lib.ParseCtx(context.Background(), ...).
or locally:
go run cmd/fitter/main.go --path=./examples/config_api.json
or locally:
go run cmd/cli/main.go --path=./examples/cli/config_cli.json
--input=\""124"\" --input=124 --input='{"test": 5}'./fitter_cli_${VERSION} --path=./examples/cli/config_cli.json --copy=true
One-time interactive login which stores a (refresh) token for the oauth2 connector config:
# device flow (default when the provider supports it): no callback, works headless
./fitter_cli_${VERSION} auth --provider github --client-id <ID> --client-secret <SECRET> --token-file ~/.fitter/tokens/github.json
# custom provider without preset
./fitter_cli_${VERSION} auth --auth-url https://.../authorize --token-url https://.../token --client-id <ID> --token-file ./token.json
Arguments:
github|google|microsoft|gitlab|spotifyoauth2.token_fileauto (device if available, else browser), device (visit a url + enter a code) or browser (localhost callback with PKCE, default port 8988 — register http://127.0.0.1:8988/callback as the app callback url)FITTER_AUTH_PORT); with the default the callback url to register at the provider is http://127.0.0.1:8988/callback127.0.0.1; set 0.0.0.0 inside a container so the published port reaches the listener (env FITTER_AUTH_LISTEN)FITTER_AUTH_REDIRECT_URL)Running inside Docker: see OAuth2 accounts in Docker.
After login the command prints the ready-to-use oauth2 config block. The connector refreshes the access token automatically and writes rotated refresh tokens back to the token file, so the login is needed only once.
For sites without an API/OAuth: log in once by hand in a real (headed) browser window — any auth scheme works, including passwords, 2FA, SSO and captchas — and save the session for headless scraping via storage_state_file:
./fitter_cli_${VERSION} browser-login --url https://example.com/login --storage-state ~/.fitter/sessions/example.json
# a browser window opens; log in, then press Enter in the terminal to save the session
Arguments:
playwright.storage_state_file (required)Re-running the command loads the existing state first, so you can extend/refresh a session without logging in from scratch. The scraping connector also writes refreshed cookies back after every run, keeping the session alive as long as it is used regularly. Needs a display: inside Docker run this command on the host and mount the file — see browser sessions in Docker.
Examples:
Fitter Agent is an AI-powered CLI that uses Claude to convert natural language requests into Fitter configurations and execute them automatically.
or locally:
export ANTHROPIC_API_KEY=<your-anthropic-api-key>
go run cmd/agent/main.go
ANTHROPIC_API_KEY environment variable so the key does not end up in your shell history┌─────────────────────────────────────────────────────────────────┐
│ 1. User enters natural language request │
│ "Get top 5 HackerNews stories with titles and scores" │
│ ↓ │
│ 2. Claude returns a config in a schema-constrained response │
│ ↓ │
│ 3. Agent validates it; on failure the error is handed back │
│ to Claude to repair (up to 3 attempts) │
│ ↓ │
│ 4. Agent displays config and asks for confirmation │
│ ↓ │
│ 5. On confirmation, executes via lib.Parse() │
│ ↓ │
│ 6. Returns structured JSON result │
└─────────────────────────────────────────────────────────────────┘
The agent keeps the conversation, so after a config is generated you can just say what to change instead of restating the whole request:
> Get top 3 HackerNews stories with titles and scores
refine> Only return 5 items and also include the article URL
Use new to forget the current config and start a fresh session.
$ export ANTHROPIC_API_KEY=sk-ant-...
$ ./fitter_agent
╔══════════════════════════════════════════════════════════════╗
║ Fitter Agent - AI-Powered Data Extraction ║
╚══════════════════════════════════════════════════════════════╝
Describe what you want to extract. Follow-up messages refine the
previous config. Type 'help' for commands.
> Get top 3 HackerNews stories with titles and scores
┌─ Generated Fitter Config ───────────────────────────────────────
{
"item": {
"connector_config": {
"response_type": "json",
"url": "https://hacker-news.firebaseio.com/v0/topstories.json",
"server_config": { "method": "GET" }
},
"model": {
"array_config": {
"root_path": "@this",
"length_limit": 3,
"item_config": {
"fields": {
"id": { "base_field": { "type": "int" } },
"story": {
"base_field": {
"type": "int",
"generated": {
"model": {
"type": "object",
"connector_config": {
"response_type": "json",
"url": "https://hacker-news.firebaseio.com/v0/item/{PL}.json",
"server_config": { "method": "GET" }
},
"model": {
"object_config": {
"fields": {
"title": { "base_field": { "type": "string", "path": "title" } },
"score": { "base_field": { "type": "int", "path": "score" } }
}
}
}
}
}
}
}
}
}
}
}
}
}
└──────────────────────────────────────────────────────────────────
Execute this config? [y/n]: y
┌─ Result ────────────────────────────────────────────────────────
[
{
"id": 46740029,
"story": { "title": "Show HN: Open-source project", "score": 161 }
},
{
"id": 46737630,
"story": { "title": "Interesting article", "score": 237 }
},
{
"id": 46735644,
"story": { "title": "New technology release", "score": 192 }
}
]
└──────────────────────────────────────────────────────────────────
> exit
Goodbye!
The agent can generate configs for:
It is the way how you fetch the data
type ConnectorConfig struct {
ResponseType ParserType `json:"response_type" yaml:"response_type"`
Url string `json:"url" yaml:"url"`
Attempts uint32 `json:"attempts" yaml:"attempts"`
NullOnError bool `yaml:"null_on_error" json:"null_on_error"`
StaticConfig *StaticConnectorConfig `json:"static_config" yaml:"static_config"`
IntSequenceConfig *IntSequenceConnectorConfig `json:"int_sequence_config" yaml:"int_sequence_config"`
ServerConfig *ServerConnectorConfig `json:"server_config" yaml:"server_config"`
BrowserConfig *BrowserConnectorConfig `yaml:"browser_config" json:"browser_config"`
PluginConnectorConfig *PluginConnectorConfig `json:"plugin_connector_config" yaml:"plugin_connector_config"`
ReferenceConfig *ReferenceConnectorConfig `yaml:"reference_config" json:"reference_config"`
FileConfig *FileConnectorConfig `json:"file_config" yaml:"file_config"`
}
https://api.open-meteo.com/v1/forecast?latitude={{{latitude}}}&longitude={{{longitude}}}&hourly=temperature_2m&forecast_days=1Config can be one of:
Example:
{
"response_type": "xpath",
"attempts": 3,
"url": "https://openweathermap.org/find?q={PL}",
"browser_config": {
"playwright": {
"timeout": 30,
"wait": 30,
"install": false,
"browser": "Chromium"
}
}
}
Connector can be defined via plugin system. For use that you need apply next flags to Fitter/Cli(location of the plugins):
... --plugins=./examples/plugin
--plugins - looking for all files with ".so" extension in provided folder(subdirs excluded)
type PluginConnectorConfig struct {
Name string `json:"name" yaml:"name"`
Config json.RawMessage `json:"config" yaml:"config"`
}
{
"name": "connector",
"config": {
"name": "Elon"
}
}
Build plugin
go build -buildmode=plugin -gcflags="all=-N -l" -o examples/plugin/connector.so examples/plugin/connector/connector.go
Make sure you export Plugin variable which implements pl.ConnectorPlugin interface
Example for CLI:
Plugin example:
package main
import (
"context"
"encoding/json"
"fmt"
"github.com/PxyUp/fitter/pkg/config"
"github.com/PxyUp/fitter/pkg/logger"
"github.com/PxyUp/fitter/pkg/builder"
pl "github.com/PxyUp/fitter/pkg/plugins/plugin"
)
var (
_ pl.ConnectorPlugin = &plugin{}
Plugin plugin
)
type plugin struct {
log logger.Logger
Name string `json:"name" yaml:"name"`
}
func (pl *plugin) Get(ctx context.Context, parsedValue builder.Interfacable, index *uint32, input builder.Interfacable) ([]byte, error) {
return []byte(fmt.Sprintf(`{"name": "%s"}`, pl.Name)), nil
}
func (pl *plugin) SetConfig(cfg *config.PluginConnectorConfig, logger logger.Logger) {
pl.log = logger
if cfg.Config != nil {
err := json.Unmarshal(cfg.Config, pl)
if err != nil {
pl.log.Errorw("cant unmarshal plugin configuration", "error", err.Error())
return
}
}
}
Connector which allow get prefetched data from references
type ReferenceConnectorConfig struct {
Name string `yaml:"name" json:"name"`
}
Example
Improved version of static connector which generate int sequence as result
type IntSequenceConnectorConfig struct {
Start int `json:"start" yaml:"start"`
End int `json:"end" yaml:"end"`
Step int `json:"step" yaml:"step"`
}
Example
{
"start": 0,
"end": 2
// Generate [0, 1]
}
Connector type which fetch data from provided file
type FileConnectorConfig struct {
Path string `yaml:"path" json:"path"`
UseFormatting bool `yaml:"use_formatting" json:"use_formatting"`
}
Connector type which fetch data from provided string
type StaticConnectorConfig struct {
Value string `json:"value" yaml:"value"`
Raw json.RawMessage `json:"raw" yaml:"raw"`
}
Example:
{
"value": "[1,2,3,4,5]"
}
Connector type which fetch data using golang http.Client(server side request like curl)
type ServerConnectorConfig struct {
Method string `json:"method" yaml:"method"`
Headers map[string]string `yaml:"headers" json:"headers"`
Timeout uint32 `yaml:"timeout" json:"timeout"`
JsonRawBody json.RawMessage `json:"json_raw_body" yaml:"json_raw_body"`
Body string `yaml:"body" json:"body"`
ErrorOnStatus bool `json:"error_on_status" yaml:"error_on_status"`
Proxy *ProxyConfig `yaml:"proxy" json:"proxy"`
OAuth2 *OAuth2Config `yaml:"oauth2" json:"oauth2"`
}
false. When true, an HTTP response status >= 400 is treated as a fetch error (flowing through attempts / null_on_error) instead of parsing the error body — so you can distinguish a failed fetch from a genuinely empty result. Leaving it false keeps the original behavior of parsing whatever body came back.Authorization header configRequests send an identifiable
User-Agent(fitter (+https://github.com/PxyUp/fitter)) by default; set your ownUser-AgentinHeadersto override it.
Example:
{
"method": "GET",
"proxy": {
"server": "http://localhost:8080",
"username": "pyx"
}
}
Automatically obtains an access token before the request and injects it as the Authorization header (overriding one set via headers). Tokens are cached in memory and refreshed before expiry; on a 401 response the cached token is dropped and the request is retried once with a fresh one.
type OAuth2Config struct {
TokenUrl string `json:"token_url" yaml:"token_url"`
GrantType OAuth2GrantType `json:"grant_type" yaml:"grant_type"`
ClientId string `json:"client_id" yaml:"client_id"`
ClientSecret string `json:"client_secret" yaml:"client_secret"`
Scopes []string `json:"scopes" yaml:"scopes"`
RefreshToken string `json:"refresh_token" yaml:"refresh_token"`
EndpointParams map[string]string `json:"endpoint_params" yaml:"endpoint_params"`
AuthStyle string `json:"auth_style" yaml:"auth_style"`
TokenFile string `json:"token_file" yaml:"token_file"`
}
{{{FromEnv=CLIENT_SECRET}}}audience for Auth0), "client_credentials" grant only~/) for persisting tokens between runs; the stored token is preferred over RefreshToken and rotated refresh tokens are written back — required for providers with single-use refresh tokens (GitHub Apps and similar). Create it with fitter_cli authExample:
{
"method": "GET",
"oauth2": {
"token_url": "https://oauth2.googleapis.com/token",
"grant_type": "refresh_token",
"client_id": "{{{FromEnv=GOOGLE_CLIENT_ID}}}",
"client_secret": "{{{FromEnv=GOOGLE_CLIENT_SECRET}}}",
"refresh_token": "{{{FromEnv=GOOGLE_REFRESH_TOKEN}}}"
}
}
type ProxyConfig struct {
// Proxy to be used for all requests. HTTP and SOCKS proxies are supported, for example
// `http://myproxy.com:3128` or `socks5://myproxy.com:3128`. Short form `myproxy.com:3128`
// is considered an HTTP proxy.
Server string `json:"server" yaml:"server"`
// Optional username to use if HTTP proxy requires authentication.
Username string `json:"username" yaml:"username"`
// Optional password to use if HTTP proxy requires authentication.
Password string `json:"password" yaml:"password"`
}
{
"server": "http://localhost:8080",
"username": "pyx"
}
Connector type which emulate fetching of data via browser
type BrowserConnectorConfig struct {
Chromium *ChromiumConfig `json:"chromium" yaml:"chromium"`
Docker *DockerConfig `json:"docker" yaml:"docker"`
Playwright *PlaywrightConfig `json:"playwright" yaml:"playwright"`
}
Config can be one of:
Example:
{
"docker": {
"wait": 10000,
"image": "docker.io/zenika/alpine-chrome:with-node",
"entry_point": "chromium-browser",
"purge": true
}
}
Use locally installed Chromium for fetch the data
type ChromiumConfig struct {
Path string `yaml:"path" json:"path"`
Timeout uint32 `yaml:"timeout" json:"timeout"`
Wait uint32 `yaml:"wait" json:"wait"`
Flags []string `yaml:"flags" json:"flags"`
}
Example:
{
"path": "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome",
"wait": 10000
}
Use Docker for spin up container for fetch data
type DockerConfig struct {
Image string `yaml:"image" json:"image"`
EntryPoint string `json:"entry_point" yaml:"entry_point"`
Timeout uint32 `yaml:"timeout" json:"timeout"`
Wait uint32 `yaml:"wait" json:"wait"`
Flags []string `yaml:"flags" json:"flags"`
Purge bool `json:"purge" yaml:"purge"`
NoPull bool `yaml:"no_pull" json:"no_pull"`
PullTimeout uint32 `yaml:"pull_timeout" json:"pull_timeout"`
}
Docker default image: docker.io/zenika/alpine-chrome
Example:
{
"wait": 10000,
"image": "docker.io/zenika/alpine-chrome:with-node",
"entry_point": "chromium-browser",
"purge": true
}
Run browsers via playwright framework
type PlaywrightConfig struct {
Browser PlaywrightBrowser `json:"browser" yaml:"browser"`
Install bool `yaml:"install" json:"install"`
Timeout uint32 `yaml:"timeout" json:"timeout"`
Wait uint32 `yaml:"wait" json:"wait"`
TypeOfWait *playwright.WaitUntilState `json:"type_of_wait" yaml:"type_of_wait"`
PreRunScript string `json:"pre_run_script" yaml:"pre_run_script"`
PostRunScript string `json:"post_run_script" yaml:"post_run_script"`
Stealth bool `json:"stealth" yaml:"stealth"`
StorageStateFile string `json:"storage_state_file" yaml:"storage_state_file"`
IndexedDB bool `json:"indexed_db" yaml:"indexed_db"`
Proxy *ProxyConfig `yaml:"proxy" json:"proxy"`
}
playwright-go version on first use; not needed with the ghcr.io/pxyup/fitter-mcp:playwright image, which ships them preinstalled)~/) to a playwright storage state json (cookies + localStorage): loaded into the browser context before navigation, written back after every run so refreshed sessions stay alive. Lets headless runs reuse a real login — create the file once with fitter_cli browser-login. Use the same browser for login and scraping: sites may bind sessions to the browser fingerprint. Also support formattingExample
{
"timeout": 30,
"wait": 30,
"install": false,
"browser": "Chromium"
}
With model we define result of the scrapping
type Model struct {
ObjectConfig *ObjectConfig `yaml:"object_config" json:"object_config"`
ArrayConfig *ArrayConfig `json:"array_config" yaml:"array_config"`
BaseField *BaseField `json:"base_field" yaml:"base_field"`
IsArray bool `json:"is_array" yaml:"is_array"`
}
Config can be one of:
Example:
{
"object_config": {}
}
Configuration of the object and fields
type ObjectConfig struct {
Fields map[string]*Field `json:"fields" yaml:"fields"`
Field *BaseField `json:"field" yaml:"field"`
ArrayConfig *ArrayConfig `json:"array_config" yaml:"array_config"`
Condition string `json:"condition" yaml:"condition"`
}
Config can be one of:
Example:
{
"fields": {
"title": {
"base_field": {
"type": "string",
"path": "type"
}
}
}
}
Configuration of the array and fields
type ArrayConfig struct {
RootPath string `json:"root_path" yaml:"root_path"`
Reverse bool `yaml:"reverse" json:"reverse"`
ItemConfig *ObjectConfig `json:"item_config" yaml:"item_config"`
LengthLimit uint32 `json:"length_limit" yaml:"length_limit"`
Condition string `json:"condition" yaml:"condition"`
ItemCondition string `json:"item_condition" yaml:"item_condition"`
StaticConfig *StaticArrayConfig `json:"static_array" yaml:"static_array"`
}
nulls to preserve the declared size (this is intentional) — omit length_limit to get exactly the source length insteadConfig can be one of:
Example:
{
"root_path": "#content dt.quote > a",
"item_config": {
"field": {
"type": "string"
}
}
}
Common of the field
type Field struct {
BaseField *BaseField `json:"base_field" yaml:"base_field"`
ObjectConfig *ObjectConfig `json:"object_config" yaml:"object_config"`
ArrayConfig *ArrayConfig `json:"array_config" yaml:"array_config"`
FirstOf []*Field `json:"first_of" yaml:"first_of"`
}
Config can be one of:
Example:
{
"base_field": {
"type": "string",
"path": "div.current-temp span.heading"
}
}
In case we want get some static information or generate new one
type BaseField struct {
Type FieldType `yaml:"type" json:"type"`
Path string `yaml:"path" json:"path"`
HTMLAttribute string `json:"html_attribute" yaml:"html_attribute"`
Condition string `json:"condition" yaml:"condition"`
Generated *GeneratedFieldConfig `yaml:"generated" json:"generated"`
FirstOf []*BaseField `json:"first_of" yaml:"first_of"`
}
Important: by default "string" type trimmed and all special chars is replaced, if you need plain string use "raw_string"
Config can be one of or empty:
Examples
{
"generated": {
"uuid": {}
}
}
{
"type": "string",
"path": "text()"
}
Every field can carry a condition - an expr-lang expression (predefined values). When it evaluates to anything except true the field is omitted from the output (the key/item disappears), not set to null. An invalid expression also omits the field and logs an error.
Where the condition is evaluated:
condition - after extraction: fRes is the extracted value, fSrc the node the field was resolved from (its siblings included) - so fSrc.on_sale == true can gate a field on data you did not extract. A false condition skips generated work entirely (no sub-request, no file download)condition / ArrayConfig.condition - before resolution: fRes/fSrc are the source node (parsed value for json, text content for html)item_condition - against every built item: fRes is the item, fSrc the source element it was built from, fIndex its index; false items are dropped - declarative array filtering. Use fSrc to filter on source attributes without adding them to the outputFilter array items - fSrc.in_stock reads the source element (not extracted into the output), fRes.price the built item:
{
"array_config": {
"root_path": "products",
"item_condition": "fSrc.in_stock && fRes.price > 0",
"item_config": {
"fields": {
"title": { "base_field": { "type": "string", "path": "title" } },
"price": { "base_field": { "type": "float", "path": "price" } }
}
}
}
}
Omit a key unless the value passes a check:
{
"discount": {
"base_field": {
"type": "float",
"path": "discount_pct",
"condition": "fRes > 0"
}
}
}
Special cases:
null (positions are fixed by definition, indexes never shift)nullRunnable example: examples/config_conditions.json
Provide functionality of generating field on the flight
type GeneratedFieldConfig struct {
UUID *UUIDGeneratedFieldConfig `yaml:"uuid" json:"uuid"`
Static *StaticGeneratedFieldConfig `yaml:"static" json:"static"`
Formatted *FormattedFieldConfig `json:"formatted" yaml:"formatted"`
Plugin *PluginFieldConfig `yaml:"plugin" json:"plugin"`
Calculated *CalculatedConfig `yaml:"calculated" json:"calculated"`
File *FileFieldConfig `yaml:"file" json:"file"`
Model *ModelField `yaml:"model" json:"model"`
FileStorageField *FileStorageField `json:"file_storage" yaml:"file_storage"`
}
Config can be one of:
Examples:
{
"uuid": {}
}
{
"model": {
"type": "array",
"model": {
"array_config": {
"root_path": "#content dt.quote > a",
"item_config": {
"field": {
"type": "string"
}
}
}
},
"connector_config": {
"response_type": "HTML",
"url": "http://www.quotationspage.com/random.php",
"attempts": 3,
"browser_config": {
"chromium": {
"path": "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome",
"wait": 10000
}
}
}
}
}
Generate random UUID V4 on the flight, can be used for generate uniq id
type UUIDGeneratedFieldConfig struct {
Regexp string `yaml:"regexp" json:"regexp"`
}
Generate static field
type StaticGeneratedFieldConfig struct {
Type FieldType `yaml:"type" json:"type"`
Value string `json:"value" yaml:"value"`
Raw json.RawMessage `json:"raw" yaml:"raw"`
}
Example
{
"type": "int",
"value": "65"
}
{
"type": "array",
"value": "[65,45]"
}
{
"type": "array",
"raw": [65,45]
}
Generate formatted field which will pass value from parent base field
type FormattedFieldConfig struct {
Template string `yaml:"template" json:"template"`
}
{
"template": "https://news.ycombinator.com/item?id={PL}"
}
Field can be used for store field result as local file
type FileStorageField struct {
Content string `json:"content" yaml:"content"`
Raw json.RawMessage `yaml:"raw" yaml:"raw"`
FileName string `json:"file_name" yaml:"file_name"`
Path string `json:"path" yaml:"path"`
Append bool `json:"append" yaml:"append"`
}
{
"content": "{{{id}}}, {{{message}}}\n",
"append": true,
"file_name": "{{{id}}}.csv",
"path": "/Users/pxyup/fitter/examples/cli/test/csv"
}
Field can be used for download file from server locally
type FileFieldConfig struct {
Config *ServerConnectorConfig `yaml:"config" json:"config"`
Url string `yaml:"url" json:"url"`
FileName string `json:"file_name" yaml:"file_name"`
Path string `json:"path" yaml:"path"`
}
Result of the field will be local file path as string
{
"url": "https://images.shcdn.de/resized/w680/p/dekostoff-gobelinstoff-panel-oriental-cat-46-x-46_P19-KP_2.jpg",
"path": "/Users/pxyup/fitter/bin",
"config": {
"method": "GET"
}
}
With propagated URL (inject of the parent value as a string)
{
"url": "https://picsum.photos{PL}",
"path": "/Users/pxyup/fitter/bin",
"config": {
"method": "GET"
}
}
Config example:
Field can generate different types depends from expression
type CalculatedConfig struct {
Type FieldType `yaml:"type" json:"type"`
Expression string `yaml:"expression" json:"expression"`
}
FNull - alias for builder.Nullvalue
FNil - alias for nil
isNull(value T) - function for check is value is FNull
fRes - it is raw(with proper type) result from the parsing base field
fIndex - it is index in parent array(only if parent was array field)
fResJson - it is JSON string representation of the raw result
fResRaw - result in bytes format
fSrc - only in condition/item_condition expressions: the source node the value was resolved from (parsed value for json - siblings included, text content for html). Not available in calculated/formatted/notifier expressions
FNewLine - new line separator
{
"type": "bool",
"expression": "fRes > 500"
}
Field can be some external plugin for fitter
type PluginFieldConfig struct {
Name string `json:"name" yaml:"name"`
Config json.RawMessage `json:"config" yaml:"config"`
}
type ModelField struct {
// Type of parsing
ConnectorConfig *ConnectorConfig `yaml:"connector_config" json:"connector_config"`
// Model of the response
Model *Model `yaml:"model" json:"model"`
Type FieldType `yaml:"type" json:"type"`
Path string `yaml:"path" json:"path"`
Expression string `yaml:"expression" json:"expression"`
}
Examples:
{
"type": "array",
"model": {
"array_config": {
"root_path": "#content dt.quote > a",
"item_config": {
"field": {
"type": "string"
}
}
}
}
}
{
"type": "string",
"path": "temp.temp",
"model": {
"object_config": {
"fields": {
"temp": {
"base_field": {
"type": "string",
"path": "//div[@id='forecast_list_ul']//td/b/a/@href",
"generated": {
"model": {
"type": "string",
"model": {
"object_config": {
"fields": {
"temp": {
"base_field": {
"type": "string",
"path": "div.current-temp span.heading"
}
}
}
}
},
"connector_config": {
"response_type": "HTML",
"attempts": 4,
"url": "https://openweathermap.org{PL}",
"browser_config": {
"playwright": {
"timeout": 30,
"wait": 30,
"install": false,
"browser": "FireFox",
"type_of_wait": "networkidle"
}
}
}
}
}
}
}
}
}
},
"connector_config": {
"response_type": "xpath",
"attempts": 3,
"url": "https://openweathermap.org/find?q={PL}",
"browser_config": {
"playwright": {
"timeout": 30,
"wait": 30,
"install": false,
"browser": "Chromium"
}
}
}
}
Provide static(fixed length) array generation
type StaticArrayConfig struct {
Items map[uint32]*Field `yaml:"items" json:"items"`
Length uint32 `yaml:"length" json:"length"`
}
Examples:
{
"0": {
"base_field": {
"type": "string",
"path": "div.current-temp span.heading"
}
}
}
{
"length": 4,
"0": {
"base_field": {
"type": "string",
"path": "div.current-temp span.heading"
}
}
}
{
"length": 4,
"2": {
"base_field": {
"type": "string",
"path": "div.current-temp span.heading"
}
}
}
Examples:
{{{FromExp="{{{FromEnv=TEST_VAL}}}" + "hello"}}}
Current time is: {PL} with token from TokenRef={{{RefName=TokenRef}}} and TokenObjectRef={{{RefName=TokenObjectRef token}}}
Current time is: {PL} with token from TokenRef={{{RefName=TokenRef}}} and TokenObjectRef={{{RefName=TokenObjectRef token}}}
TokenRef={{{RefName=TokenRef}}} and TokenObjectRef={{{RefName=TokenObjectRef token}}} Object={{{value}}} {PL} Env={{{FromEnv=TEST_VAL}}} {INDEX} {HUMAN_INDEX}
Special map which prefetched(before any processing) and can be user for connector or for placeholder
Can be used for:
type Reference struct {
*ModelField
Expire *uint32 `yaml:"expire" json:"expire"`
}
For Fitter
type RefMap map[string]*Reference
type Config struct {
// Other Config Fields
Limits *Limits `yaml:"limits" json:"limits"`
References RefMap `json:"references" yaml:"references"`
}
For Fitter Cli
type RefMap map[string]*Reference
type CliItem struct {
// Other Config Fields
Limits *Limits `yaml:"limits" json:"limits"`
References RefMap `json:"references" yaml:"references"`
}
Example
{
"references": {
"TokenRef": {
"expire": 10,
"connector_config": {
"response_type": "json",
"static_config": {
"value": "\"plain token\""
}
},
"model": {
"base_field": {
"type": "string"
}
}
},
"TokenObjectRef": {
"connector_config": {
"response_type": "json",
"static_config": {
"value": "{\"token\":\"token from object\"}"
}
},
"model": {
"object_config": {
"fields": {
"token": {
"base_field": {
"type": "string",
"path": "token"
}
}
}
}
}
}
}
}
Optional per-item config item.notifier_config which pushes the parse result somewhere after processing. The result is still returned as usual (CLI/MCP output, service logs); the notifier additionally delivers it. Works in Fitter (service mode), Fitter_CLI and Fitter_MCP.
type NotifierConfig struct {
Expression string `yaml:"expression" json:"expression"`
Force bool `json:"force" yaml:"force"`
SendArrayByItem bool `yaml:"send_array_by_item" json:"send_array_by_item"`
Template string `yaml:"template" json:"template"`
// exactly ONE destination:
Console *ConsoleConfig `yaml:"console" json:"console"`
TelegramBot *TelegramBotConfig `yaml:"telegram_bot" json:"telegram_bot"`
Http *HttpConfig `yaml:"http" json:"http"`
Redis *RedisNotifierConfig `json:"redis" yaml:"redis"`
File *FileStorageField `json:"file" yaml:"file"`
}
fRes (parsed value), fResRaw (raw bytes), fResJson (JSON string), e.g. len(fResRaw) > 0console, telegram_bot, http, redis, fileDestination configs:
type HttpConfig struct {
Url string `yaml:"url" json:"url"`
Method string `json:"method" yaml:"method"`
Headers map[string]string `yaml:"headers" json:"headers"`
Timeout uint32 `yaml:"timeout" json:"timeout"`
}
type TelegramBotConfig struct {
Token string `json:"token" yaml:"token"`
UsersId []int64 `json:"users_id" yaml:"users_id"`
Pretty bool `json:"pretty" yaml:"pretty"`
OnlyMsg bool `json:"only_msg" yaml:"only_msg"`
}
type RedisNotifierConfig struct {
Addr string `json:"addr" yaml:"addr"`
Password string `json:"password" yaml:"password"`
DB int `json:"db" yaml:"db"`
Channel string `json:"channel" yaml:"channel"`
}
type ConsoleConfig struct {
OnlyResult bool `json:"only_result" yaml:"only_result"`
}
The file destination uses the same FileStorageField as the file field type.
Example (examples/config_telegram.json):
{
"item": {
"connector_config": { "...": "..." },
"model": { "...": "..." },
"notifier_config": {
"expression": "len(fResRaw) > 0",
"telegram_bot": {
"token": "{{{FromEnv=TG_TOKEN}}}",
"users_id": [123456],
"pretty": true
}
}
}
}
Provide limitation for prevent DDOS, big usage of memory
type Limits struct {
HostRequestLimiter HostRequestLimiter `yaml:"host_request_limiter" json:"host_request_limiter"`
ChromiumInstance uint32 `yaml:"chromium_instance" json:"chromium_instance"`
DockerContainers uint32 `yaml:"docker_containers" json:"docker_containers"`
PlaywrightInstance uint32 `yaml:"playwright_instance" json:"playwright_instance"`
}
{
"limits": {
"host_request_limiter": {
"hacker-news.firebaseio.com": 5
},
"chromium_instance": 3,
"docker_containers": 3,
"playwright_instance": 3
}
}
Turn any website or API into structured JSON with LLM-authored declarative scraping configs.
{ "mcpServers": { "io.github.pxyup-fitter": { "command": "mcpb", "args": [ "https://github.com/PxyUp/fitter/releases/download/v1.8.3/fitter-mcp-darwin-arm64.mcpb" ], "env": { "FITTER_PLUGINS": "<value>" } } }}Araçlar
Henüz araç listesi yok. Uzak sunucular ilk sağlık kontrolünden sonra bir liste alır.
SSS
Bu sayfadan Cursor, Claude Desktop, Claude Code veya VS Code için stdio yapılandırmasını kopyalayın. PlugMCP proxy'sini değil, yayıncının kurulumunu kullanır.
Yayıncı şu ortam değişkenlerini listeliyor: FITTER_PLUGINS.
Fitter yerel olarak stdio üzerinden çalışır.
Son kontrol: 2026-11-09. Turn any website or API into structured JSON with LLM-authored declarative scraping configs.