Skip to content

Adam Greenwell

Index Management

The search index is the foundation of Lumenare Search. This guide covers building, updating, and maintaining your search index.

Current release: 2.2.01

Open Settings → Lumenare Search → Index Management under Maintain the index. Search & Answers, under Configure search, contains the separate content-analysis controls. On smaller screens, choose a section from the Settings section menu.

What is the Search Index?

The search index is a custom database table that stores:

  • Keywords extracted from your content
  • Per-field occurrence counts for titles, excerpts, content, taxonomies, authors, custom fields, and optional document attachments
  • Post relationships
  • Reverse terms for suffix matching
  • Fast lookup for search queries

Why it's important:

  • Makes search fast (direct database lookups)
  • Enables relevance ranking
  • Supplies keyword candidates shared by search and answers

Meaning reranking uses two additional stores: prepared passages split eligible public content into sections, and embedding vectors record analyses from your selected AI service and model. A healthy keyword index does not establish that those stores are ready, and stored vector totals can include earlier configurations.

Building Your Index

Initial Index Build

When: After first installation or when index is missing

Steps:

  1. Go to Settings → Lumenare Search → Index Management
  2. Check "Indexed Posts" count (should be 0 or incomplete)
  3. Click "Reindex All Content"
  4. Wait for completion (runs in batches)
  5. Verify "Indexed Posts" count matches the published content included by your indexing settings

Time: Depends on content volume, document parsing and hosting resources. Follow the progress display rather than a fixed time estimate.

Note: The plugin automatically indexes new content as you publish it, but existing content needs initial indexing.

Full Reindex

When to use:

  • After changing stop words
  • After changing post types to search
  • After enabling a field that had a zero weight during indexing
  • After changing which custom fields are indexed
  • After enabling document indexing or changing document MIME types
  • After bulk content updates
  • When index seems incomplete or outdated

Steps:

  1. Go to Index Management section
  2. Click "Reindex All Content"
  3. Wait for completion
  4. Verify indexed count

Note: Full reindex rebuilds the keyword index. It does not generate missing embedding vectors for meaning reranking. Changing an already enabled field from one positive weight to another affects query scoring immediately; use partial reindex for smaller content updates.

Partial Reindex

When to use:

  • After publishing new posts
  • After updating specific content
  • After changing settings that don't require full reindex
  • For incremental updates

Options:

  • Index New Posts: Only index posts not yet indexed
  • Index Post Type: Index specific post types
  • Index Date Range: Index posts within date range
  • Automatic New-Content Indexing: Schedule WP-Cron to keep large catalogs caught up without a full rebuild

Steps:

  1. Go to Index Management section
  2. Choose partial reindex option
  3. Select parameters (post type, date range, etc.)
  4. Click reindex button
  5. Wait for completion

Automatic New-Content Indexing

For large catalogs, enable automatic new-content reindexing in Index Management and choose a WP-Cron interval. Lumenare records the last run status, continuation state, and next scheduled run so you can see whether the catch-up job is healthy.

Version 2.1.02 also repairs stale overdue WP-Cron events by rescheduling them instead of leaving automatic indexing stuck in the past.

Use this for normal ongoing publishing. Use a full reindex after required schema repairs or changes to post types, stop words, enabled content fields or document extraction. Positive-to-positive weight tuning does not require a keyword rebuild.

Prepare Content for Meaning Search

Content analysis is separate from keyword reindexing. You can prepare it before enabling meaning reranking:

  1. Activate the WordPress AI plugin and an embedding-capable connector. Set credentials in Settings → Connectors and, when required, approve Lumenare Search in Tools → Connector Approvals.
  2. Open Search & Answers → Advanced → AI service and model for content analysis.
  3. Choose a service, click Refresh model list, choose a supported embedding model and click Save Settings. Answer-writing models serve a separate purpose.
  4. Click Analyze site content. Keep the tab open as the browser prepares public, searchable posts, then analyzes missing or changed passages in small batches. Provider charges may apply.
  5. Wait for analysis to complete and review the analyzed and pending passage counts for the saved configuration.

Pause stops the browser run after its current batch. Resume analysis continues saved progress, including after reopening the page. Scheduled analysis may still run when meaning search is enabled. A request failure stops the browser run; check the service, use Refresh status and resume instead of repeatedly starting over.

Opening this page, saving settings and refreshing status do not start provider work. Refresh model list explicitly contacts the service for metadata but sends no site content. Analysis reuses compatible stored vectors and submits only missing or changed passages. It does not train a model or enable meaning reranking, answer writing, public Ask Site or answer cards; configure those choices separately in Search & Answers.

After content changes, check for pending analysis and run it again. Changing the service, model, analysis size, passage size or overlap, or enabled content sources requires saving first and starting work for the new configuration. Searches ignore incompatible vectors and passages that no longer match current public content; they do not repair them during a visitor request.

Index Status

Checking Index Status

Indexed Posts Count:

  • Shows number of posts in index
  • Compare to eligible published content in the selected post types
  • Excluded post types and content should not count toward expected coverage

Index Coverage:

  • Percentage of content indexed
  • Helps identify missing content
  • Check coverage against the content you chose to index

Meaning-Search Preparation:

  • Check Search & Answers → Advanced for prepared-post and analyzed-passage progress
  • Review pending work for the selected service and model
  • Treat stored passage/vector totals as storage counts, rather than proof that every item is compatible and usable

Index Health

Signs of healthy index:

  • Indexed count matches eligible published content
  • Search returns expected results
  • No database errors
  • Fast query performance

Signs of issues:

  • Indexed count much lower than published
  • Search missing expected results
  • Database errors in logs
  • Slow query performance

Maintenance

Regular Maintenance

Weekly:

  • Check indexed post count
  • Verify new content is indexed
  • Review index coverage

Monthly:

  • Full reindex if needed
  • Check for orphaned index entries
  • Review index size
  • Optimize database if needed

After Configuration Changes

Requires Reindex:

  • Stop words changes
  • Post types changes
  • Enabling a field previously excluded by a zero weight
  • Indexed custom field changes
  • Document indexing or document MIME type changes

No Reindex Needed:

  • Positive-to-positive field or taxonomy weight changes
  • Match mode changes
  • Display settings
  • Analytics settings
  • Security settings
  • Synonyms, query promotions, freshness, click, audio, and trending boosts

Requires a Content-Analysis Check:

  • Edited or newly published content
  • Changed embedding service, model or analysis size
  • Changed passage size, overlap or enabled content sources

Check pending work in Search & Answers → Advanced and run Analyze site content for the saved configuration. These checks are separate from keyword reindexing.

Troubleshooting

Index Not Building

Symptoms:

  • Index count stays at 0
  • Reindex button doesn't work
  • Database errors

Solutions:

  1. Check database permissions
  2. Verify table exists (wp_lumenare_search_index)
  3. Check WordPress debug log
  4. Increase PHP memory/execution time
  5. Deactivate/reactivate plugin
  6. Run the built-in schema repair/reindex prompt if Lumenare reports failed index writes

Index Incomplete

Symptoms:

  • Index count lower than published posts
  • Some posts not searchable
  • Missing results

Solutions:

  1. Check post types are selected
  2. Verify posts are published (not draft)
  3. Run full reindex
  4. Check for errors in logs
  5. Verify database table integrity
  6. Review automatic reindex status if WP-Cron is responsible for catching up new content
  7. If an old automatic reindex event is overdue, update to 2.1.02 or later so the schedule can self-repair

Slow Indexing

Symptoms:

  • Indexing takes very long
  • Timeouts during indexing
  • Server load high

Solutions:

  1. Increase PHP execution time
  2. Increase PHP memory limit
  3. Let indexing complete (don't interrupt)
  4. Use partial reindex for updates
  5. Schedule indexing during off-peak hours
  6. Enable automatic new-content indexing for smaller, repeated catch-up runs

Content Analysis Cannot Start

Read the message beside Analyze site content. Save the selected service, model and analysis settings first, then use Refresh status. Verify credentials, connector approval and an embedding-capable model. A busy batch or service error backoff may need time to clear.

If the message reports a database-read failure, check the site error log. 2.2.00 fixes the Illegal mix of collations comparison that could block analysis on older databases; update an older build before considering a database conversion. See Troubleshooting for the separate setup and database checks.

Index Too Large

Symptoms:

  • Large database table
  • Slow queries
  • High storage usage

Solutions:

  1. Add more stop words
  2. Limit post types indexed
  3. Review index content
  4. Optimize database
  5. Consider archiving old content

Best Practices

Initial Setup

  1. Build index immediately after installation
  2. Verify index count matches content
  3. Test search to ensure it works
  4. Check coverage is complete

Ongoing Maintenance

  1. Monitor index status regularly
  2. Use partial reindex when possible
  3. Full reindex only when needed
  4. Keep index updated with content changes

Performance

  1. Schedule full reindex during off-peak hours
  2. Use batch indexing (automatic)
  3. Don't interrupt indexing process
  4. Monitor server resources during indexing

After Changes

  1. Reindex after stop words changes
  2. Reindex after enabling fields excluded during earlier indexing
  3. Reindex after post type changes
  4. Test search after reindexing
  5. Check content analysis after content or embedding-configuration changes

Database Details

Table Structure

Table: wp_lumenare_search_index (prefix may vary)

Contains:

  • Post IDs
  • Keywords
  • Field-level counts used by the weighting system
  • Taxonomy, author, custom field, and attachment signals when enabled
  • Index metadata

Table Management

Don't manually edit the index table. Use plugin interface.

Backup: Include the keyword index and optional wp_lumenare_search_chunks and wp_lumenare_search_vectors tables in database backups. Your site's table prefix may differ.

Restore: Check the plugin's schema notices and rebuild the keyword index when needed. Prepared passages and vectors have their own preparation step in Search & Answers.

← Back to Documentation Home | Weighting System →