Loading
Migrate a whole ArcGIS Hub site into a PortalJS Arc portal end-to-end. Harvests the Hub /data.json (DCAT-US) inventory, exports every FeatureService layer through the ArcGIS REST query API with result
The skill file stays in the publisher's repository.
Paste the copied command in a terminal. It installs this SKILL.md into that agent.
npx skills add datopian/portaljs --skill arcgis-to-portaljs -a cursor -y
Migrate a whole ArcGIS Hub site into a PortalJS Arc portal end-to-end. Harvests the Hub /data.json (DCAT-US) inventory, exports every FeatureService layer through the ArcGIS REST query API with resultOffset paging, converts each to the serverless dual tier (PMTiles render + GeoParquet query) with tabular items to Parquet, pushes everything to Cloudflare R2 via Git LFS, appends dual-tier datasets.json entries, and writes a source-vs-derived parity report. Use to move a City or sector ArcGIS Hub open-data portal onto PortalJS with no server-side compute.
Migrate a whole ArcGIS Hub site into a PortalJS Arc portal end-to-end. Harvests the Hub /data.json (DCAT-US) inventory, exports every FeatureService layer through the ArcGIS REST query API with resultOffset paging, converts each to the serverless dual tier (PMTiles render + GeoParquet query) with tabular items to Parquet, pushes everything to Cloudflare R2 via Git LFS, appends dual-tier datasets.json entries, and writes a source-vs-derived parity report. Use to move a City or sector ArcGIS Hub open-data portal onto PortalJS with no server-side compute.
name: arcgis-to-portaljs description: Migrate a whole ArcGIS Hub site into a PortalJS Arc portal end-to-end. Harvests the Hub /data.json (DCAT-US) inventory, exports every FeatureService layer through the ArcGIS REST query API with resultOffset paging, converts each to the serverless dual tier (PMTiles render + GeoParquet query) with tabular items to Parquet, pushes everything to Cloudflare R2 via Git LFS, appends dual-tier datasets.json entries, and writes a source-vs-derived parity report. Use to move a City or sector ArcGIS Hub open-data portal onto PortalJS with no server-side compute. allowed-tools: Read, Write, Edit, Bash(ogr2ogr:), Bash(ogrinfo:), Bash(tippecanoe:), Bash(duckdb:), Bash(git:), Bash(curl:), Bash(jq:), Bash(npx:), Bash(node:), Bash(mkdir:), Bash(cp:), Bash(wc:), Bash(command:*), WebFetch version: 0.1.0 author: Datopian [email protected] license: MIT compatibility: Claude Code with PortalJS portals (Next.js 14, React 18/19, Node 18+). Requires native GDAL, tippecanoe, duckdb, and jq on the operator's machine (macOS or Linux/WSL). Runs from any project via the plugin, a personal install, or a portaljs clone. tags:
Migrate an entire ArcGIS Hub open-data site into a PortalJS Arc portal in one pass.
Every Hub site is machine-readable — a DCAT-US catalog at /data.json, with every dataset
backed by an ArcGIS REST FeatureService — so migration is a harvest → export → convert →
publish → verify pipeline that runs almost fully automated on the operator's machine, no
server-side compute. The tooling is the reusable arcgis-to-portaljs migrator: input is one
Hub URL, output is a ready-to-deploy PortalJS catalog plus a parity report.
The skill is an orchestrator: it reuses the DCAT-US harvest from portaljs-migrate, the
ogr2ogr/tippecanoe/duckdb dual-tier conversion from portaljs-add-geo, and the bulk
Git-LFS → R2 push from portaljs-migrate. Its novel parts are the FeatureService REST export
loop (paged features, not just a link) and the source-vs-derived parity report.
components/MapPreview.tsx and
components/GeoQuery.tsx (PR #1647 or later). Run portaljs-new-portal first if none.ogr2ogr, ogrinfo), tippecanoe, duckdb (with spatial),
and jq. macOS: brew install gdal tippecanoe duckdb jq; Debian/Ubuntu:
apt-get install gdal-bin duckdb jq plus tippecanoe (apt or build from source); Windows via
WSL. The skill hard-stops with the install hint if any is missing.portaljs-deploy resolves), or an OSS
self-hosted Giftless.The canonical, full step-by-step workflow is
.claude/commands/arcgis-to-portaljs.md
— the single source of truth. Read and follow it when executing. Summary:
--limit,
--only, --dry-run, --namespace-mode). Interview if missing; never dead-end.ogr2ogr, tippecanoe, duckdb + spatial, jq). Any missing →
print the per-OS install and stop./data.json (reuse the portaljs-migrate DCAT-US map) and classify each
item: vector (FeatureService), table, or non-data (web map / 3D / imagery → skipped).
Under --namespace-mode owner, resolve namespaces through a publisher-normalization
table with title-prefix fallback for broken {{source}} publishers (multi-publisher
Hubs ship dirty publisher labels). Dedup near-duplicate hosted-view layers — but only
after a mandatory live record-count check on BOTH twins: equal ⇒ dedup (keep the source
layer, log the pair); different ⇒ keep both as distinct datasets. Consolidate per-year
dataset series into one year-partitioned Parquet with legacy per-year view entries. Enrich
from the AGOL item: sanitized metadata (license/description/dates), cleaned display title
(cleanTitle — raw title still drives the slug), category (item categories → meaningful
theme → keyword mapping), and a thumbnail snapshot into public/thumbnails/.query API with resultOffset paging
(f=geojson, outSR=4326); fall back to keyset paging on transfer limits; accept a
customer File Geodatabase dump for very large layers.portaljs-add-geo recipe (PMTiles +
GeoParquet); tabular items to Parquet. Preserve the native-CRS original.datasets.json entries (upsert on (namespace, slug)).arcgis-parity-report.md — record count, extent, attribute schema, and geometry
validity, source vs derived, per dataset, plus the migrated/skipped/failed accounting.data/<namespace>/<slug>.pmtiles, .parquet, and the original per vector
dataset (all LFS-tracked → R2); Parquet + original per table; arcgis-parity-report.md.datasets.json (one dual-tier entry per vector dataset, one resource entry
per table); .gitattributes (LFS tracking)./@<namespace>/<slug> renders <MapPreview> + <GeoQuery> for each vector
dataset with no page edits; the catalog lists everything migrated.| Symptom | Cause | Fix |
|---|---|---|
MISSING_INPUT | No Hub URL provided | Pass the site root (e.g. https://hub-lewisville.opendata.arcgis.com) and retry. |
MISSING_TOOLS | ogr2ogr/tippecanoe/duckdb/jq (or duckdb spatial) absent | Print the per-OS install line and stop; re-run after installing. |
NOT_A_PORTAL | Target dir has no datasets.json / geo components | Run portaljs-new-portal first, then re-run. |
HARVEST_FAILED | /data.json unreachable or not DCAT-US | Confirm the site is an ArcGIS Hub and the feed loads in a browser. |
EXPORT_FAILED | One FeatureService layer errored or hit a hard transfer cap | Logged and skipped; try keyset paging or a customer FGDB dump for that layer. |
LFS_PUSH_FAILED | Missing/expired Arc token or unset lfs.url | Re-mint the JWT (see portaljs-deploy); confirm git config lfs.url. |
/arcgis-to-portaljs https://hub-lewisville.opendata.arcgis.com slug=lewisville
/arcgis-to-portaljs https://streamwaterdata.co.uk --dry-run
/arcgis-to-portaljs https://streamwaterdata.co.uk --only sewer-catchments,water-boundaries --namespace-mode owner
.claude/commands/arcgis-to-portaljs.mdreferences/reference.mdreferences/sync-and-cutover.mdportaljs-migrate, portaljs-add-geo, portaljs-add-dataset, portaljs-deploy