UI: Store Configuration & Testing
Relevant source files
The following files were used as context for generating this wiki page:The Store Configuration & Testing feature provides a user interface within the Filament admin panel for creating, managing, and testing web scraping configurations for different online retailers. This system is central to PriceBuddy's ability to fetch product information from various websites. Each "Store" entity defines the rules and settings required to extract product details like title, price, and image from a product page URL.
This functionality is primarily managed through the StoreResource class, which defines the UI for listing, creating, editing, and testing stores. The system is designed to be flexible, supporting different scraping services and strategies to accommodate a wide range of website structures.
Sources: app/Filament/Resources/StoreResource.php, README.md:10-11
Architecture Overview
The Store management UI is built using the Filament PHP framework. It consists of a Resource class that orchestrates several Page classes and Widget classes to provide a complete CRUD and testing interface for the Store model.
The main components are:
StoreResource: The central class defining the data schema (form), table columns, and relationships for stores.ListStores: The page for listing all configured stores.CreateStore/EditStore: Pages for creating and editing a store's configuration.TestStore: A specialized page for testing a store's scraping rules against a live product URL.TestResultsWidget: A widget displayed on theTestStorepage to show the outcome of a scrape test.
Sources: app/Filament/Resources/StoreResource.php:143-150
Component Relationship Diagram
This diagram illustrates how the main PHP classes for the Store UI interact.
Store Configuration
The store configuration form is defined in StoreResource.php and is used by both the CreateStore and EditStore pages. It is composed of several sections to group related settings.
Sources: app/Filament/Resources/StoreResource.php:32-117
Form Fields
The form for creating and editing a store contains the following fields:
| Section | Field | Component Type | Description |
|---|---|---|---|
| Basics | name | TextInput | The name of the store. This is a required field. |
| Domains | domains | Repeater | A list of domain names this store configuration applies to. Required. |
| Scrape Strategy | scrape_strategy | Group | A group of sections defining how to extract the title, price, and image. |
| Scraper service | settings.scraper_service | Radio | The scraping service to use: http (default, fast) or api (for JavaScript-rendered pages). |
| Scraper service | settings.scraper_service_settings | Textarea | Additional settings for the api scraper service, such as device or sleep. Hidden if the service is http. |
| Locale | settings.locale_settings | Group | Overrides for default locale (en) and currency (USD) for this specific store. |
| Notes | notes | RichEditor | A rich text field for any additional notes about the store's scraping configuration. |
Sources: app/Filament/Resources/StoreResource.php:32-117, tests/Feature/Filament/StoreTest.php:44-55
Scraping Strategy
The scraping strategy is the core of the store configuration. It defines how to find the product title, price, and image on a page. For each of these three data points, a strategy must be defined.
Each strategy consists of the following fields:
| Field | Description |
|---|---|
type | The method to use for extraction. Options are selector, regex, and jsonpath. |
value | The actual selector, regex pattern, or JSONPath expression. |
prepend | An optional static value to prepend to the extracted result. |
append | An optional static value to append to the extracted result. |
The default values for the strategy are based on Open Graph meta tags.
// app/Filament/Resources/StoreResource.php:22-26
public const array DEFAULT_SELECTORS = [
'title' => 'meta[property=og:title]|content',
'price' => 'meta[property=og:price:amount]|content',
'image' => 'meta[property=og:image]|content',
];
Sources: app/Filament/Resources/StoreResource.php:22-26, 152-171
The type field determines how the value is interpreted:
| Type | value Help Text |
|---|---|
selector | CSS selector to find an element. |
regex | A regular expression to run against the page. |
jsonpath | A JSONPath expression to run against JSON data. |
Sources: app/Filament/Resources/StoreResource.php:164-165
Store Testing
The application provides a dedicated page to test a store's scraping configuration against a specific product URL. This allows for rapid iteration and debugging of scraping rules without needing to create a full product entry.
*Sources: app/Filament/Resources/StoreResource.php:148, app/Filament/Resources/StoreResource/Pages/TestStore.php
Test Page (TestStore.php)
The TestStore page is a modified EditRecord page that presents a simplified form containing a single field: test_url.
When the user enters a URL and submits the form by clicking the "Test url scrape" button, the following process is initiated:
- The
save()method is triggered. - An instance of
ScrapeUrlis created with the provided URL. - The
scrape()method is called on theScrapeUrlservice, passing the currentStorerecord and forcing a non-cached request (use_cache => false). - The result of the scrape operation is stored in the user's session under the key
test_scrape. - The
test_urlis saved to the store's settings for future convenience. - The user is redirected back to the same test page, where the
TestResultsWidgetwill now display the results from the session.
Sources: app/Filament/Resources/StoreResource/Pages/TestStore.php:44-58
Test Flow Diagram
The following diagram shows the user flow and backend process for testing a store configuration.
Test Results Widget
The TestResultsWidget is responsible for displaying the outcome of the scrape test. It is included as a footer widget on the TestStore page.
The widget's getScrape() method retrieves the scrape data from the session using session()->get('test_scrape'). This data is then passed to its Blade view, which renders the extracted title, price, image, and any errors that occurred during the scraping process.
Sources: app/Filament/Resources/StoreResource/Pages/TestStore.php:104-107, app/Filament/Resources/StoreResource/Widgets/TestResultsWidget.php:17-21