Guide

Architecture

8 min read 1523 words Updated 14 August 2026

System Overview

WordPress Export JSON is a monorepo containing two complementary applications for exporting WordPress content:

1. wpexportjson - REST API based exporter with brute force capabilities
2. wpxmlrpc - XML-RPC based exporter for authenticated access

Export flow

graph TB
    A[CLI Interface] --> B[Configuration Manager]
    B --> C[WordPress API Client]
    C --> D[Content Discovery]
    D --> E[Brute Force Scanner]
    D --> F[Media Downloader]
    E --> G[Export Engine]
    F --> G
    G --> H[JSON Exporter]
    G --> I[Markdown Exporter]
    G --> K[Shopify Exporter]
    G --> L[Magento Exporter]
    G --> M[Wix/Squarespace/Webflow]
    G --> N[Ghost/Strapi/Contentful]
    G --> O[Weebly/PrestaShop]
    H --> J[Output Files]
    I --> J
    K --> J
    L --> J
    M --> J
    N --> J
    O --> J

High-Level Architecture

graph TB
    subgraph "WordPress Export JSON Monorepo"
        subgraph "Applications"
            A1[wpexportjson CLI]
            A2[wpxmlrpc CLI]
        end
        subgraph "Internal Packages"
            API[api - REST Client]
            XMLRPC[xmlrpc - XML-RPC Client]
            CONFIG[config - Configuration]
            EXPORT[export - Export Engine]
            MEDIA[media - Media Downloader]
            BRUTE[bruteforce - Content Scanner]
        end
        subgraph "Models"
            MODELS[models - Data Structures]
        end
    end
    subgraph "WordPress Site"
        WP_REST[WordPress REST API]
        WP_XMLRPC[WordPress XML-RPC]
        WP_MEDIA[Media Files]
    end
    subgraph "Output"
        JSON[JSON Export]
        MD[Markdown Export]
        MEDIA_FILES[Downloaded Media]
    end
    A1 --> API
    A1 --> BRUTE
    A2 --> XMLRPC
    API --> CONFIG
    XMLRPC --> CONFIG
    A1 --> EXPORT
    A2 --> EXPORT
    EXPORT --> MEDIA
    API --> MODELS
    XMLRPC --> MODELS
    EXPORT --> MODELS
    API --> WP_REST
    XMLRPC --> WP_XMLRPC
    MEDIA --> WP_MEDIA
    EXPORT --> JSON
    EXPORT --> MD
    MEDIA --> MEDIA_FILES

Component Architecture

1. Command Line Applications

wpexportjson

• Purpose: Primary application for WordPress content export
• Protocol: WordPress REST API
• Features:
  • Public API access (no authentication required)
  • Brute force content discovery
  • High performance concurrent processing
  • Media download with progress tracking

wpxmlrpc

• Purpose: Alternative exporter for authenticated access
• Protocol: WordPress XML-RPC
• Features:
  • Authenticated access to private content
  • Legacy WordPress support
  • User credential based authentication

2. Core Internal Packages

api Package

1type Client struct {
2    config     *config.Config
3    httpClient *resty.Client
4    baseURL    string
5}

• REST API client implementation
• Handles pagination automatically
• Supports concurrent requests
• Error handling and retries
• Discovers custom post types from /wp/v2/types (posttypes.go) and fetches their entries, excluding WordPress internals, plugin bookkeeping types and a theme's saved layouts/templates

xmlrpc Package

1type Client struct {
2    config   *config.Config
3    username string
4    password string
5    endpoint string
6    blogID   int
7}

• XML-RPC protocol implementation
• Authentication handling
• XML marshaling/unmarshaling
• Legacy WordPress compatibility

config Package

 1type Config struct {
 2    URL           string
 3    Output        string
 4    Format        string
 5    BruteForce    bool
 6    MaxID         int
 7    DownloadMedia bool
 8    Concurrent    int
 9    // ... other fields
10}

• Centralized configuration management
• Environment variable support
• File-based configuration
• Validation and defaults

export Package

1type Exporter struct {
2    config     *config.Config
3    downloader *media.Downloader
4}

• Multi-format export engine
• JSON and Markdown output
• Media path resolution
• Content transformation

media Package

1type Downloader struct {
2    config     *config.Config
3    httpClient *http.Client
4    mediaDir   string
5    progress   *progressbar.ProgressBar
6}

• Concurrent media downloading
• Progress tracking
• File deduplication
• Path sanitization
• Content URL rewriting (urlrewrite.go)

urlrewrite.go is deliberately separate from downloader.go: downloading and content rewriting are distinct responsibilities that happen at different stages of an export. It builds a urlIndex keyed on the normalised upload path — scheme, host, query and case stripped — rather than on the literal source_url, because WordPress stores post_content with whatever URL form was current when the post was written while the REST API reports the site's present-day form. A single regex pass over the rendered content then resolves each URL-ish token against that index, so src, href and srcset are rewritten uniformly. Unresolvable references with a -{width}x{height} suffix fall back to the nearest surviving width.

The URLRewriter is built once per export (Exporter.updateMediaPaths) and reused for every field, since indexing is O(media) and would otherwise repeat per post. The same instance localises body content, excerpt, og_image and the mediaMap behind featured_image — one mechanism rather than a second one per field. Because it only substitutes on an index hit, a URL that is not a downloaded attachment (a CDN image, an external og:image) passes through untouched, and addresses of the source site (canonical_url, link, hreflangs) are simply never fed to it.

bruteforce Package

1type Scanner struct {
2    config    *config.Config
3    apiClient *api.Client
4}

• ID enumeration scanning
• Concurrent content discovery
• Progress reporting
• Configurable limits

3. Data Models

Core WordPress Types

 1type WordPressPost struct {
 2    ID              int
 3    Date            WordPressTime
 4    Title           RenderedContent
 5    Content         RenderedContent
 6    // ... other fields
 7}
 8
 9type WordPressMedia struct {
10    ID              int
11    SourceURL       string
12    MediaDetails    MediaDetails
13    // ... other fields
14}

Custom Types

1type WordPressTime struct {
2    time.Time
3}
4
5func (wt *WordPressTime) UnmarshalJSON(data []byte) error {
6    // Handles multiple WordPress date formats
7}

Data Flow Architecture

REST API Export Flow

sequenceDiagram
    participant CLI as wpexportjson CLI
    participant API as REST Client
    participant WP as WordPress REST API
    participant BF as Brute Force Scanner
    participant EXP as Export Engine
    participant MEDIA as Media Downloader
    CLI->>API: Initialize client
    CLI->>API: Get site info
    API->>WP: GET /wp-json/wp/v2/
    WP-->>API: Site information
    CLI->>API: Get posts
    API->>WP: GET /wp-json/wp/v2/posts
    WP-->>API: Posts data
    CLI->>API: Get pages, media, etc.
    API->>WP: Multiple paginated requests
    WP-->>API: Content data
    opt Brute Force Enabled
        CLI->>BF: Scan for missing content
        BF->>API: Test individual IDs
        API->>WP: GET /wp-json/wp/v2/posts/{id}
        WP-->>API: Additional content
    end
    CLI->>EXP: Export data
    EXP->>MEDIA: Download media files
    MEDIA->>WP: Download media URLs
    EXP->>EXP: Generate output files

XML-RPC Export Flow

sequenceDiagram
    participant CLI as wpxmlrpc CLI
    participant XMLRPC as XML-RPC Client
    participant WP as WordPress XML-RPC
    participant EXP as Export Engine
    CLI->>XMLRPC: Initialize with credentials
    CLI->>XMLRPC: Test connection
    XMLRPC->>WP: wp.getOptions
    WP-->>XMLRPC: Connection confirmed
    CLI->>XMLRPC: Get posts
    XMLRPC->>WP: wp.getPosts
    WP-->>XMLRPC: Posts data
    CLI->>XMLRPC: Get pages, media, etc.
    XMLRPC->>WP: Multiple XML-RPC calls
    WP-->>XMLRPC: Content data
    CLI->>EXP: Export data
    EXP->>EXP: Generate output files

File System Architecture

Project Structure

wpexportjson/
├── cmd/                    # Application entry points
│   ├── wpexportjson/      # REST API client
│   └── wpxmlrpc/          # XML-RPC client
├── internal/              # Internal packages
│   ├── api/               # REST API client
│   ├── xmlrpc/            # XML-RPC client
│   ├── config/            # Configuration management
│   ├── export/            # Export engine
│   ├── media/             # Media downloader
│   └── bruteforce/        # Brute force scanner
├── pkg/                   # Public packages
│   └── models/            # Data models
├── docs/                  # Documentation
├── build/                 # Build artifacts
├── dist/                  # Release binaries
└── export/                # Default export directory

Export Output Structure

JSON Format

export/
├── export.json           # Complete export data
└── media/               # Downloaded media files
    ├── 1_image.jpg
    ├── 2_video.mp4
    └── ...

Markdown Format

export/
├── README.md            # Site information
├── posts/               # Individual post files
│   ├── 2024-01-01-post-title.md
│   └── ...
├── pages/               # Individual page files
│   ├── 2024-01-01-page-title.md
│   ├── cpt_services/    # One directory per custom post type
│   │   └── wms-implementation.md
│   └── ...
├── media/               # Downloaded media files
│   ├── 1_image.jpg
│   └── ...
└── metadata.json        # Categories, tags, users, marketing (incl. theme
                         # palette), custom_types

Concurrency Architecture

Worker Pool Pattern

1// Media downloader uses worker pools
2jobs := make(chan models.WordPressMedia, len(mediaItems))
3results := make(chan bool, len(mediaItems))
4
5// Start workers
6for i := 0; i < d.config.Concurrent; i++ {
7    go d.worker(jobs, results)
8}

Brute Force Scanning

 1// Concurrent ID scanning
 2for i := 0; i < s.config.Concurrent; i++ {
 3    wg.Add(1)
 4    go func() {
 5        defer wg.Done()
 6        for id := range jobs {
 7            // Process ID
 8        }
 9    }()
10}

Error Handling Architecture

Layered Error Handling

1. HTTP Level: Connection errors, timeouts
2. API Level: HTTP status codes, rate limiting
3. Data Level: JSON parsing, validation
4. Application Level: Business logic errors

Retry Strategy

1// Exponential backoff with jitter
2for attempt := 0; attempt <= maxRetries; attempt++ {
3    if success := operation(); success {
4        return nil
5    }
6    time.Sleep(time.Duration(attempt+1) * time.Second)
7}

Security Architecture

Authentication

• REST API: No authentication (public endpoints)
• XML-RPC: Username/password or application passwords
• Media Downloads: Follows WordPress authentication

Data Protection

• No credential storage in configuration files
• Environment variable support
• HTTPS enforcement for XML-RPC

Performance Considerations

Optimization Strategies

1. Concurrent Processing: Configurable worker pools
2. Pagination: Automatic handling of large datasets
3. Caching: File existence checks for media
4. Progress Tracking: Real-time feedback
5. Memory Management: Streaming for large files

Scalability Limits

• REST API: Rate limiting by WordPress
• XML-RPC: Generally slower than REST
• Memory: Proportional to content size
• Disk: Media files can be large

Extension Points

Adding New Export Formats

1. Implement format in export package
2. Add format validation in config
3. Update CLI flags and documentation

Adding New Content Types

1. Define models in pkg/models
2. Add API methods in api or xmlrpc
3. Update export logic

Custom Authentication

1. Extend xmlrpc.Client for new auth methods
2. Add configuration options
3. Update CLI interface

Testing Architecture

Unit Tests

• Individual package testing
• Mock HTTP clients
• Configuration validation

Integration Tests

• End-to-end export testing
• WordPress test site setup
• Output validation

Performance Tests

• Large dataset handling
• Concurrent operation testing
• Memory usage profiling