Introduction
Magento 2 is a powerful eCommerce platform, and it integrates seamlessly with Elasticsearch for search and product catalog indexing. However, when upgrading Elasticsearch or Magento versions, issues may arise with products not showing in category pages (PLP), even though they are visible in the front-end and their configurations seem correct. This post will walk you through a common issue where products are missing from category pages due to an Elasticsearch misconfiguration and how to resolve it.
Scenario: Products Not Showing in Category Pages
After migrating Magento to use Elasticsearch 8.x, we noticed that products were visible in the front-end (search and individual product pages), but not showing in category listings (PLP). Despite reindexing, clearing the cache, and verifying configurations, the issue persisted.
Troubleshooting Steps
1. Verify Product and Category Configuration
The first step in troubleshooting any Magento 2 catalog issue is to verify that the product is assigned to the correct category and that the category itself is set up to display products.
Check Product Category Assignment: Ensure that the product is correctly assigned to the right categories in Magento.
Check Catalog Category Product Index: Use the following query to check the catalog_category_product table:
SELECT * FROM catalog_category_product WHERE product_id = xxx; If this returns no rows, the product is not indexed correctly for the category.
Rebuild Catalog Index: Run the indexing command to ensure your product data is indexed correctly:
bin/magento indexer:reindex catalog_category_productStill no results? Let’s move on to Elasticsearch.
2. Ensure Elasticsearch is Properly Configured
Magento 2 relies on Elasticsearch for search and indexing, but incorrect Elasticsearch settings can interfere with product visibility.
Test Elasticsearch Connection: Ensure that Magento is connected to Elasticsearch and the connection test is successful in the Magento Admin Panel (Stores > Configuration > Catalog > Catalog Search).
Check Elasticsearch Index Status: Verify that the Elasticsearch index for products is working fine. You can do this using:
curl -X GET "localhost:9200/_cat/indices?v"If the index status shows yellow or red, this indicates an issue with Elasticsearch’s index or shard allocation.
3. Diagnose Elasticsearch Errors
In our case, Elasticsearch logs revealed the error:
main.CRITICAL: Elastic\Elasticsearch\Exception\ClientResponseException: 400 Bad Request: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Fielddata access on the _id field is disallowed, you can re-enable it by updating the dynamic cluster setting: indices.id_field_data.enabled"}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"magento2_product_1_v21","node":"VJlHqogTTHOeFrsYG6EWKA","reason":{"type":"illegal_argument_exception","reason":"Fielddata access on the _id field is disallowed, you can re-enable it by updating the dynamic cluster setting: indices.id_field_data.enabled"}}],"caused_by":{"type":"illegal_argument_exception","reason":"Fielddata access on the _id field is disallowed, you can re-enable it by updating the dynamic cluster setting: indices.id_field_data.enabled","caused_by":{"type":"illegal_argument_exception","reason":"Fielddata access on the _id field is disallowed, you can re-enable it by updating the dynamic cluster setting: indices.id_field_data.enabled"}}},"status":400} in /var/www/magento/public_html/vendor/elasticsearch/elasticsearch/src/Response/Elasticsearch.php:65
This error indicates that fielddata access on the _id field is disabled, which is required for the correct operation of product indexing for Magento 2. Elasticsearch 8.x disabled this feature by default for performance reasons.
4. Solution: Enabling Fielddata for the _id Field
The solution is to enable fielddata for the _id field in your Elasticsearch configuration. You can do this by running the following commands in your Elasticsearch:
Enable Fielddata Access for the _id Field:
curl -X PUT "localhost:9200/_cluster/settings" -H 'Content-Type: application/json' -d' { "persistent": { "indices.id_field_data.enabled": true } }'Verify the Setting:
curl -X GET "localhost:9200/_cluster/settings"Reindex Magento Catalog: Run the reindex command again after updating Elasticsearch:bashCopyEditbin/magento indexer:reindex
Clear Cache: Clear Magento’s cache to apply the changes:
bin/magento cache:clean bin/magento cache:flushFinal Verification
After applying these changes, you should now see the products appearing in the category pages as expected. The issue with missing products was caused by Elasticsearch’s inability to process the _id field, which Magento relies on for indexing product-category relationships. Once we enabled fielddata access, the index rebuilt correctly, and the products were successfully linked to their categories.
Conclusion
While Elasticsearch is a powerful search engine, its default settings can sometimes interfere with Magento 2’s catalog indexing, especially after an upgrade to newer Elasticsearch versions. In this case, enabling field data for the _id field resolved the issue of missing products in category pages. If you encounter similar issues, be sure to check Elasticsearch logs, verify the index status, and adjust Elasticsearch settings accordingly.
If you continue to experience issues with product visibility or Elasticsearch configurations, don’t hesitate to reach out to us for further assistance.
