4 |
5 | Permission is hereby granted, free of charge, to any person obtaining a copy
6 | of this software and associated documentation files (the "Software"), to deal
7 | in the Software without restriction, including without limitation the rights
8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9 | copies of the Software, and to permit persons to whom the Software is
10 | furnished to do so, subject to the following conditions:
11 |
12 | The above copyright notice and this permission notice shall be included in
13 | all copies or substantial portions of the Software.
14 |
15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
21 | THE SOFTWARE.
22 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 | [](https://packagist.org/packages/pdphilip/elasticlens)
6 | [](https://github.com/pdphilip/elasticlens/actions?query=workflow%3Arun-tests+branch%3Amain)
7 | [](https://github.com/pdphilip/elasticlens/actions?query=workflow%3A"Fix+PHP+code+style+issues"+branch%3Amain)
8 | [](https://packagist.org/packages/pdphilip/elasticlens)
9 |
10 |
11 |
12 | Search your Laravel models with the convenience of Eloquent and the power of Elasticsearch
13 |
14 |
15 | ElasticLens for Laravel uses Elasticsearch to create and sync a searchable index of your Laravel models.
16 |
17 |
19 |
23 |
40 | How?
41 |
42 |
43 | > The `Index-Model` acts as a separate Elasticsearch model managed by ElasticLens, yet you retain full control over it, just like any other Laravel model. In addition to working directly with the `Index-Model`, ElasticLens offers tools for
44 | > mapping fields (with embedding relationships) during the build process, and managing index migrations.
45 |
46 | > For Example, a base `User` Model will sync with an Elasticsearch `IndexedUser` Model that provides all the features from [Laravel-Elasticsearch](https://github.com/pdphilip/laravel-elasticsearch) to search your `Base-Model`.
47 |
48 |
49 |
50 |
51 |
52 | # Requirements
53 |
54 | - Laravel 10.x/11.x/12.x
55 | - Elasticsearch 8.x
56 |
57 |
58 | # Installation
59 |
60 |
61 | NB: Before you start, set the Laravel-Elasticsearch DB Config (click to expand)
62 |
63 | > See [Laravel-Elasticsearch](https://github.com/pdphilip/laravel-elasticsearch) for more details
64 | >
65 | > Update `.env`
66 |
67 | ```dotenv
68 | ES_AUTH_TYPE=http
69 | ES_HOSTS="http://localhost:9200"
70 | ES_USERNAME=
71 | ES_PASSWORD=
72 | ES_CLOUD_ID=
73 | ES_API_ID=
74 | ES_API_KEY=
75 | ES_SSL_CA=
76 | ES_INDEX_PREFIX=my_app_
77 | # prefix will be added to all indexes created by the package with an underscore
78 | # ex: my_app_user_logs for UserLog model
79 | ES_SSL_CERT=
80 | ES_SSL_CERT_PASSWORD=
81 | ES_SSL_KEY=
82 | ES_SSL_KEY_PASSWORD=
83 | # Options
84 | ES_OPT_ID_SORTABLE=false
85 | ES_OPT_VERIFY_SSL=true
86 | ES_OPT_RETRIES=
87 | ES_OPT_META_HEADERS=true
88 | ES_ERROR_INDEX=
89 | ES_OPT_BYPASS_MAP_VALIDATION=false
90 | ES_OPT_DEFAULT_LIMIT=1000
91 | ```
92 |
93 | > Update `config/database.php`
94 |
95 | ```php
96 | 'elasticsearch' => [
97 | 'driver' => 'elasticsearch',
98 | 'auth_type' => env('ES_AUTH_TYPE', 'http'), //http or cloud
99 | 'hosts' => explode(',', env('ES_HOSTS', 'http://localhost:9200')),
100 | 'username' => env('ES_USERNAME', ''),
101 | 'password' => env('ES_PASSWORD', ''),
102 | 'cloud_id' => env('ES_CLOUD_ID', ''),
103 | 'api_id' => env('ES_API_ID', ''),
104 | 'api_key' => env('ES_API_KEY', ''),
105 | 'ssl_cert' => env('ES_SSL_CA', ''),
106 | 'ssl' => [
107 | 'cert' => env('ES_SSL_CERT', ''),
108 | 'cert_password' => env('ES_SSL_CERT_PASSWORD', ''),
109 | 'key' => env('ES_SSL_KEY', ''),
110 | 'key_password' => env('ES_SSL_KEY_PASSWORD', ''),
111 | ],
112 | 'index_prefix' => env('ES_INDEX_PREFIX', false),
113 | 'options' => [
114 | 'bypass_map_validation' => env('ES_OPT_BYPASS_MAP_VALIDATION', false),
115 | 'logging' => env('ES_OPT_LOGGING', false),
116 | 'ssl_verification' => env('ES_OPT_VERIFY_SSL', true),
117 | 'retires' => env('ES_OPT_RETRIES', null),
118 | 'meta_header' => env('ES_OPT_META_HEADERS', true),
119 | 'default_limit' => env('ES_OPT_DEFAULT_LIMIT', 1000),
120 | 'allow_id_sort' => env('ES_OPT_ID_SORTABLE', false),
121 | ],
122 | ],
123 | ```
124 |
125 |
126 | Install the package via composer:
127 |
128 | ```bash
129 | composer require pdphilip/elasticlens
130 | ```
131 | Publish the config file:
132 | ```bash
133 | php artisan lens:install
134 | ```
135 | Run the migrations to create the index build and migration logs indexes:
136 | ```bash
137 | php artisan migrate
138 | ```
139 |
140 |
141 |
142 | # Read the [Documentation](https://elasticsearch.pdphilip.com/elasticlens/getting-started/)
143 |
144 |
145 | ## Features
146 |
147 | - [Zero config setup](#step-1-zero-config-setup): Start indexing with minimal configuration. [Docs](https://elasticsearch.pdphilip.com/elasticlens/index-model/)
148 | - [Eloquent Querying](#step-2-search-your-models): Search your models with Eloquent and the full power of Elasticsearch. [Docs](https://elasticsearch.pdphilip.com/elasticlens/full-text-search)
149 | - [Custom Field Mapping](#step-3-create-a-field-map): Control how your index is built, including [mapping model relationships as embedded fields](#step-4-update-fieldmap-to-include-relationships-as-embedded-fields). [Docs](https://elasticsearch.pdphilip.com/elasticlens/field-mapping/)
150 | - [Manage Elasticsearch Migrations](#step-5-define-your-index-models-migrationmap): Define a required blueprint for your index migrations. [Docs](https://elasticsearch.pdphilip.com/elasticlens/index-model-migrations/)
151 | - [Control Observed models](#step-6-fine-tune-the-observers): Tailor which models are observed for changes. [Docs](https://elasticsearch.pdphilip.com/elasticlens/model-observers/)
152 | - [Comprehensive CLI Tools](#step-7-monitor-and-administer-all-your-indexes-with-artisan-commands): Manage index health, migrate/rebuild indexes, and more with Artisan commands. [Docs](https://elasticsearch.pdphilip.com/elasticlens/artisan-cli-tools/)
153 | - [Built-in IndexableBuildState model](#step-8-optionally-access-the-built-in-indexablebuild-model-to-track-index-build-states): Track the build states of your indexes. [Docs](https://elasticsearch.pdphilip.com/elasticlens/build-migration-states/)
154 | - [Built-in Migration Logs](#step-9-optionally-access-the-built-in-indexablemigrationlog-model-for-index-migration-status): Track the build states of your indexes. [Docs](https://elasticsearch.pdphilip.com/elasticlens/build-migration-states/)
155 |
156 |
157 | ### Example Usage
158 |
159 | The Walkthrough below will demonstrate all the features by way of an example. In this example, we'll index a `User` model.
160 |
161 | # Step 1: Zero config setup
162 |
163 | ## [Docs → Indexing your Base-Model](https://elasticsearch.pdphilip.com/elasticlens/index-model/)
164 |
165 | #### 1. Add the `Indexable` Trait to Your Base-Model:
166 |
167 | ```php
168 | use PDPhilip\ElasticLens\Indexable;
169 |
170 | class User extends Eloquent implements Authenticatable, CanResetPassword
171 | {
172 | use Indexable;
173 | ```
174 |
175 | #### 2. Create an Index-Model for Your Base-Model:
176 |
177 | ElasticLens expects the `Index-Model` to be named as `Indexed` + `BaseModelName` and located in the `App\Models\Indexes` directory.
178 |
179 | 2(a) Create the `User` index with artisan:
180 | ```bash
181 | php artisan lens:make User
182 | ```
183 |
184 |
185 | 2(b) or create the `User` index directly:
186 |
187 | ```php
188 | /**
189 | * Create: App\Models\Indexes\IndexedUser.php
190 | */
191 | namespace App\Models\Indexes;
192 |
193 | use PDPhilip\ElasticLens\IndexModel;
194 |
195 | class IndexedUser extends IndexModel{}
196 |
197 | ```
198 |
199 | - That's it! Your User model will now automatically sync with the IndexedUser model whenever changes occur. You can search your User model effortlessly, like:
200 |
201 | ```php
202 | User::viaIndex()->searchTerm('running')->orSearchTerm('swimming')->get();
203 | ```
204 |
205 | # Step 2: Search your models
206 |
207 | ## [Docs → Full-text base-model search](https://elasticsearch.pdphilip.com/elasticlens/full-text-search)
208 |
209 | Perform quick and easy full-text searches:
210 |
211 | ```php
212 | User::search('loves espressos');
213 | ```
214 |
215 | > Search for the phrase `loves espressos` across all fields and return the base `User` models
216 |
217 | Cute. But that's not why we're here...
218 |
219 | To truly harness the power of [Laravel-Elasticsearch](https://github.com/pdphilip/laravel-elasticsearch) for eloquent-like querying, you can use more advanced queries:
220 |
221 | ```php
222 | BaseModel::viaIndex()->{build_your_es_eloquent_query}->first();
223 | BaseModel::viaIndex()->{build_your_es_eloquent_query}->get();
224 | BaseModel::viaIndex()->{build_your_es_eloquent_query}->paginate();
225 | BaseModel::viaIndex()->{build_your_es_eloquent_query}->avg('orders');
226 | BaseModel::viaIndex()->{build_your_es_eloquent_query}->distinct();
227 | BaseModel::viaIndex()->{build_your_es_eloquent_query}->{etc}
228 | ```
229 |
230 | #### Examples:
231 |
232 | ##### 1. Basic Term Search:
233 |
234 | ```php
235 | User::viaIndex()->searchTerm('nara')
236 | ->where('state','active')
237 | ->limit(3)->get();
238 | ```
239 |
240 | > This searches all users who are `active` for the term 'nara' across all fields and returns the top 3 results.
241 | > - [https://elasticsearch.pdphilip.com/full-text-search#term-search-term](https://elasticsearch.pdphilip.com/eloquent/search-queries/#search-term)
242 |
243 | #### 2. Phrase Search:
244 |
245 | ```php
246 | User::viaIndex()->searchPhrase('Ice bathing')
247 | ->orderByDesc('created_at')
248 | ->limit(5)->get();
249 | ```
250 |
251 | > Searches all fields for the phrase 'Ice bathing' and returns the three newest results. Phrases match exact words in order.
252 | > - [https://elasticsearch.pdphilip.com/eloquent/search-queries/#search-phrase](https://elasticsearch.pdphilip.com/eloquent/search-queries/#search-phrase)
253 |
254 | #### 3. Boosting Terms fields:
255 |
256 | ```php
257 | User::viaIndex()->searchTerm('David',['first_name^3', 'last_name^2', 'bio'])->get();
258 | ```
259 |
260 | > Searches for the term 'David', boosts the first_name field by 3, last_name by 2, and checks the bio field. Results are ordered by score.
261 | > - [https://elasticsearch.pdphilip.com/full-text-search#boosting-terms](https://elasticsearch.pdphilip.com/eloquent/search-queries/#parameter-fields)
262 | > - [https://elasticsearch.pdphilip.com/full-text-search#minimum-score](https://elasticsearch.pdphilip.com/eloquent/search-queries/#parameter-options)
263 |
264 | #### 4. Geolocation Filtering:
265 |
266 | ```php
267 | User::viaIndex()->where('status', 'active')
268 | ->filterGeoPoint('home.location', '5km', [0, 0])
269 | ->orderByGeo('home.location',[0, 0])
270 | ->get();
271 | ```
272 |
273 | > Finds all active users within a 5km radius from the coordinates [0, 0], ordering them from closest to farthest. Not kidding.
274 | > - [https://elasticsearch.pdphilip.com/es-specific#geo-point](https://elasticsearch.pdphilip.com/eloquent/es-queries/#where-geo-distance)
275 | > - [https://elasticsearch.pdphilip.com/ordering-and-pagination#order-by-geo-distance](https://elasticsearch.pdphilip.com/eloquent/ordering-and-pagination/#orderby-geo-distance)
276 |
277 | #### 5. Regex Search:
278 |
279 | ```php
280 | User::viaIndex()->whereRegex('favourite_color', 'bl(ue)?(ack)?')->get();
281 | ```
282 |
283 | > Finds all users whose favourite colour is blue or black.
284 | > - [https://elasticsearch.pdphilip.com/full-text-search#regular-expressions](https://elasticsearch.pdphilip.com/eloquent/es-queries/#where-regex)
285 |
286 | #### 6. Pagination:
287 |
288 | ```php
289 | User::viaIndex()->whereRegex('favorite_color', 'bl(ue)?(ack)?')->paginate(10);
290 | ```
291 |
292 | > Paginate search results.
293 | > - [https://elasticsearch.pdphilip.com/ordering-and-pagination](https://elasticsearch.pdphilip.com/eloquent/ordering-and-pagination/)
294 |
295 | #### 7. Nested Object Search:
296 |
297 | ```php
298 | User::viaIndex()->whereNestedObject('user_logs', function (Builder $query) {
299 | $query->where('user_logs.country', 'Norway')
300 | ->where('user_logs.created_at', '>=',Carbon::now()->modify('-1 week'));
301 | })->get();
302 | ```
303 |
304 | > Searches nested user_logs for users who logged in from Norway within the last week. Whoa.
305 | > - [https://elasticsearch.pdphilip.com/nested-queries](https://elasticsearch.pdphilip.com/eloquent/nested-queries/)
306 |
307 | #### 8. Fuzzy Search:
308 |
309 | ```php
310 | User::viaIndex()->searchFuzzy('quikc')
311 | ->orSearchFuzzy('brwn')
312 | ->orSearchFuzzy('foks')
313 | ->get();
314 | ```
315 |
316 | > No spell, no problem. Search Fuzzy.
317 | > - [https://elasticsearch.pdphilip.com/full-text-search](https://elasticsearch.pdphilip.com/eloquent/search-queries/#search-term-fuzzy)
318 |
319 | #### 9. Highlighting Search Results:
320 |
321 | ```php
322 | User::viaIndex()->searchTerm('espresso')
323 | ->withHighlights()->get();
324 | ```
325 |
326 | > Searches for 'espresso' across all fields and highlights where it was found.
327 | > - [https://elasticsearch.pdphilip.com/full-text-search#highlighting](https://elasticsearch.pdphilip.com/eloquent/search-queries/#highlighting)
328 |
329 | #### 10. Phrase prefix search:
330 |
331 | ```php
332 | User::viaIndex()->searchPhrasePrefix('loves espr')
333 | ->withHighlights()->get();
334 | ```
335 |
336 | > Searches for the phrase prefix 'loves espr' across all fields and highlights where it was found.
337 | > - [https://elasticsearch.pdphilip.com/full-text-search#highlighting](https://elasticsearch.pdphilip.com/eloquent/search-queries/#search-phrase-prefix)
338 |
339 | ### Note on `Index-Model` vs `Base-Model` Results
340 |
341 | - Since the `viaIndex()` taps into the `IndexModel`, the results will be instances of `IndexedUser`, not the base `User` model.
342 | - This can be useful for display purposes, such as highlighting embedded fields.
343 | - **However, in most cases you'll need to return and work with the `Base-Model`**
344 |
345 | ### To search and return results as `Base-Models`:
346 |
347 | #### 1. use `asBase()`
348 |
349 | - Simply chain `->asBase()` at the end of your query:
350 |
351 | ```php
352 | User::viaIndex()->searchTerm('david')->orderByDesc('created_at')->limit(3)->get()->asBase();
353 | User::viaIndex()->whereRegex('favorite_color', 'bl(ue)?(ack)?')->get()->asBase();
354 | User::viaIndex()->whereRegex('favorite_color', 'bl(ue)?(ack)?')->first()->asBase();
355 | ```
356 |
357 | #### 2. use `getBase()` instead of `get()->asBase()`
358 |
359 | ```php
360 | User::viaIndex()->searchTerm('david')->orderByDesc('created_at')->limit(3)->getBase();
361 | User::viaIndex()->whereRegex('favorite_color', 'bl(ue)?(ack)?')->getBase();
362 | ```
363 |
364 | ### To search and paginate results as `Base-Models` use: `paginateBase()`
365 |
366 | - Complete the query string with `->paginateBase()`
367 |
368 | ```php
369 | // Returns a pagination instance of Users ✔️:
370 | User::viaIndex()->whereRegex('favorite_color', 'bl(ue)?(ack)?')->paginateBase(10);
371 |
372 | // Returns a pagination instance of IndexedUsers:
373 | User::viaIndex()->whereRegex('favorite_color', 'bl(ue)?(ack)?')->paginate(10);
374 |
375 | // Will not paginate ❌ (but will at least return a collection of 10 Users):
376 | User::viaIndex()->whereRegex('favorite_color', 'bl(ue)?(ack)?')->paginate(10)->asBase();
377 | ```
378 |
379 | # Step 3: Create a field Map
380 |
381 | ## [Docs → Index-model field mapping](https://elasticsearch.pdphilip.com/elasticlens/field-mapping/)
382 |
383 | You can define the `fieldMap()` method in your `Index-Model` to control how the index is built during synchronisation.
384 |
385 | ```php
386 | use PDPhilip\ElasticLens\Builder\IndexBuilder;
387 | use PDPhilip\ElasticLens\Builder\IndexField;
388 |
389 | class IndexedUser extends IndexModel
390 | {
391 | protected $baseModel = User::class;
392 |
393 | public function fieldMap(): IndexBuilder
394 | {
395 | return IndexBuilder::map(User::class, function (IndexField $field) {
396 | $field->text('first_name');
397 | $field->text('last_name');
398 | $field->text('email');
399 | $field->bool('is_active');
400 | $field->type('state', UserState::class); //Maps enum
401 | $field->text('created_at');
402 | $field->text('updated_at');
403 | });
404 | }
405 | ```
406 |
407 | ### Notes:
408 |
409 | - The `IndexedUser` records will contain only the fields defined in the `fieldMap()`. The value of `$user->id` will correspond to `$indexedUser->id`.
410 | - Fields can also be derived from attributes in the `Base-Model`. For example, `$field->bool('is_active')` could be derived from a custom attribute in the `Base-Model`:
411 | ```php
412 | public function getIsActiveAttribute(): bool
413 | {
414 | return $this->updated_at >= Carbon::now()->modify('-30 days');
415 | }
416 | ```
417 | - When mapping enums, ensure that you also cast them in the `Index-Model`.
418 | - If a value is not found during the build process, it will be stored as `null`.
419 |
420 |
421 | # Step 4: Update `fieldMap()` to include relationships as embedded fields
422 |
423 | ## [Docs → Relationships as embedded fields](https://elasticsearch.pdphilip.com/elasticlens/field-mapping/#relationships-as-embedded-fields)
424 |
425 | You can further customise indexing by embedding relationships as nested objects within your Index-Model. The builder allows you to define fields and embed relationships, enabling more complex data structures in your
426 | Elasticsearch index.
427 |
428 | ### Examples:
429 |
430 | 1. If a `User` has many `Profiles`
431 |
432 | ```php
433 | use PDPhilip\ElasticLens\Builder\IndexBuilder;
434 | use PDPhilip\ElasticLens\Builder\IndexField;
435 |
436 | class IndexedUser extends IndexModel
437 | {
438 | protected $baseModel = User::class;
439 |
440 | public function fieldMap(): IndexBuilder
441 | {
442 | return IndexBuilder::map(User::class, function (IndexField $field) {
443 | $field->text('first_name');
444 | $field->text('last_name');
445 | $field->text('email');
446 | $field->bool('is_active');
447 | $field->type('type', UserType::class);
448 | $field->type('state', UserState::class);
449 | $field->text('created_at');
450 | $field->text('updated_at');
451 | $field->embedsMany('profiles', Profile::class)->embedMap(function (IndexField $field) {
452 | $field->text('profile_name');
453 | $field->text('about');
454 | $field->array('profile_tags');
455 | });
456 | });
457 | }
458 | ```
459 |
460 | 2. If a `Profile` has one `ProfileStatus`
461 |
462 | ```php
463 | use PDPhilip\ElasticLens\Builder\IndexBuilder;
464 | use PDPhilip\ElasticLens\Builder\IndexField;
465 |
466 | class IndexedUser extends IndexModel
467 | {
468 | protected $baseModel = User::class;
469 |
470 | public function fieldMap(): IndexBuilder
471 | {
472 | return IndexBuilder::map(User::class, function (IndexField $field) {
473 | $field->text('first_name');
474 | $field->text('last_name');
475 | $field->text('email');
476 | $field->bool('is_active');
477 | $field->type('type', UserType::class);
478 | $field->type('state', UserState::class);
479 | $field->text('created_at');
480 | $field->text('updated_at');
481 | $field->embedsMany('profiles', Profile::class)->embedMap(function (IndexField $field) {
482 | $field->text('profile_name');
483 | $field->text('about');
484 | $field->array('profile_tags');
485 | $field->embedsOne('status', ProfileStatus::class)->embedMap(function (IndexField $field) {
486 | $field->text('id');
487 | $field->text('status');
488 | });
489 | });
490 | });
491 | }
492 | ```
493 |
494 | 3. If a `User` belongs to an `Account`
495 |
496 | ```php
497 | use PDPhilip\ElasticLens\Builder\IndexBuilder;
498 | use PDPhilip\ElasticLens\Builder\IndexField;
499 |
500 | class IndexedUser extends IndexModel
501 | {
502 | protected $baseModel = User::class;
503 |
504 | public function fieldMap(): IndexBuilder
505 | {
506 | return IndexBuilder::map(User::class, function (IndexField $field) {
507 | $field->text('first_name');
508 | $field->text('last_name');
509 | $field->text('email');
510 | $field->bool('is_active');
511 | $field->type('type', UserType::class);
512 | $field->type('state', UserState::class);
513 | $field->text('created_at');
514 | $field->text('updated_at');
515 | $field->embedsMany('profiles', Profile::class)->embedMap(function (IndexField $field) {
516 | $field->text('profile_name');
517 | $field->text('about');
518 | $field->array('profile_tags');
519 | $field->embedsOne('status', ProfileStatus::class)->embedMap(function (IndexField $field) {
520 | $field->text('id');
521 | $field->text('status');
522 | });
523 | });
524 | $field->embedsBelongTo('account', Account::class)->embedMap(function (IndexField $field) {
525 | $field->text('name');
526 | $field->text('url');
527 | });
528 | });
529 | }
530 | ```
531 |
532 | 4. If a `User` belongs to a `Country` and you don't need to observe the `Country` model:
533 |
534 | ```php
535 | use PDPhilip\ElasticLens\Builder\IndexBuilder;
536 | use PDPhilip\ElasticLens\Builder\IndexField;
537 |
538 | class IndexedUser extends IndexModel
539 | {
540 | protected $baseModel = User::class;
541 |
542 | public function fieldMap(): IndexBuilder
543 | {
544 | return IndexBuilder::map(User::class, function (IndexField $field) {
545 | $field->text('first_name');
546 | $field->text('last_name');
547 | $field->text('email');
548 | $field->bool('is_active');
549 | $field->type('type', UserType::class);
550 | $field->type('state', UserState::class);
551 | $field->text('created_at');
552 | $field->text('updated_at');
553 | $field->embedsMany('profiles', Profile::class)->embedMap(function (IndexField $field) {
554 | $field->text('profile_name');
555 | $field->text('about');
556 | $field->array('profile_tags');
557 | $field->embedsOne('status', ProfileStatus::class)->embedMap(function (IndexField $field) {
558 | $field->text('id');
559 | $field->text('status');
560 | });
561 | });
562 | $field->embedsBelongTo('account', Account::class)->embedMap(function (IndexField $field) {
563 | $field->text('name');
564 | $field->text('url');
565 | });
566 | $field->embedsBelongTo('country', Country::class)->embedMap(function (IndexField $field) {
567 | $field->text('country_code');
568 | $field->text('name');
569 | $field->text('currency');
570 | })->dontObserve(); // Don't observe changes in the country model
571 | });
572 | }
573 | ```
574 |
575 | 5. If a `User` has Many `UserLogs` and you only want to embed the last 10:
576 |
577 | ```php
578 | use PDPhilip\ElasticLens\Builder\IndexBuilder;
579 | use PDPhilip\ElasticLens\Builder\IndexField;
580 |
581 | class IndexedUser extends IndexModel
582 | {
583 | protected $baseModel = User::class;
584 |
585 | public function fieldMap(): IndexBuilder
586 | {
587 | return IndexBuilder::map(User::class, function (IndexField $field) {
588 | $field->text('first_name');
589 | $field->text('last_name');
590 | $field->text('email');
591 | $field->bool('is_active');
592 | $field->type('type', UserType::class);
593 | $field->type('state', UserState::class);
594 | $field->text('created_at');
595 | $field->text('updated_at');
596 | $field->embedsMany('profiles', Profile::class)->embedMap(function (IndexField $field) {
597 | $field->text('profile_name');
598 | $field->text('about');
599 | $field->array('profile_tags');
600 | $field->embedsOne('status', ProfileStatus::class)->embedMap(function (IndexField $field) {
601 | $field->text('id');
602 | $field->text('status');
603 | });
604 | });
605 | $field->embedsBelongTo('account', Account::class)->embedMap(function (IndexField $field) {
606 | $field->text('name');
607 | $field->text('url');
608 | });
609 | $field->embedsBelongTo('country', Country::class)->embedMap(function (IndexField $field) {
610 | $field->text('country_code');
611 | $field->text('name');
612 | $field->text('currency');
613 | })->dontObserve(); // Don't observe changes in the country model
614 | $field->embedsMany('logs', UserLog::class, null, null, function ($query) {
615 | $query->orderBy('created_at', 'desc')->limit(10); // Limit the logs to the 10 most recent
616 | })->embedMap(function (IndexField $field) {
617 | $field->text('title');
618 | $field->text('ip');
619 | $field->array('log_data');
620 | });
621 | });
622 | }
623 | ```
624 |
625 | ### `IndexField $field` Methods:
626 |
627 | - `text($field)`
628 | - `integer($field)`
629 | - `array($field)`
630 | - `bool($field)`
631 | - `type($field, $type)` - Set own type (like Enums)
632 | - `embedsMany($field, $relatedModelClass, $whereRelatedField, $equalsLocalField, $query)`
633 | - `embedsBelongTo($field, $relatedModelClass, $whereRelatedField, $equalsLocalField, $query)`
634 | - `embedsOne($field, $relatedModelClass, $whereRelatedField, $equalsLocalField, $query)`
635 |
636 | **Note**: For embeds the `$whereRelatedField`, `$equalsLocalField`, `$query` parameters are optional.
637 |
638 | - `$whereRelatedField` is the `foreignKey`, and `$equalsLocalField` is the `localKey`; if they are not provided, they will be inferred from the relationship.
639 | - `$query` is a closure that allows you to customise the query for the related model.
640 |
641 | ### Embedded Relationship Builder Methods:
642 |
643 | - `embedMap(function (IndexField $field) {})` - Define the mapping for the embedded relationship
644 | - `dontObserve()` - Don't observe changes in the `$relatedModelClass`
645 |
646 |
647 | # Step 5: Define your `Index-Model`'s `migrationMap()`
648 |
649 | ## [Docs → Index-model migrations](https://elasticsearch.pdphilip.com/elasticlens/index-model-migrations/)
650 |
651 | Elasticsearch automatically indexes new fields it encounters, but it might not always index them in the way you need. To ensure the index is structured correctly, you can define a `migrationMap()` in your Index-Model.
652 |
653 | Since the `Index-Model` utilises the [Laravel-Elasticsearch](https://github.com/pdphilip/laravel-elasticsearch) package, you can use `IndexBlueprint` to customise your `migrationMap()`
654 |
655 | ```php
656 | use PDPhilip\Elasticsearch\Schema\Blueprint;
657 |
658 | class IndexedUser extends IndexModel
659 | {
660 | //......
661 | public function migrationMap(): callable
662 | {
663 | return function (Blueprint $index) {
664 | $index->text('name');
665 | $index->keyword('first_name');
666 | $index->text('first_name');
667 | $index->keyword('last_name');
668 | $index->text('last_name');
669 | $index->keyword('email');
670 | $index->text('email');
671 | $index->text('avatar')->indexField(false);
672 | $index->keyword('type');
673 | $index->text('type');
674 | $index->keyword('state');
675 | $index->text('state');
676 | //...etc
677 | };
678 | }
679 | ```
680 |
681 | ### Notes:
682 |
683 | - **Documentation**: For more details on migrations, refer to the: https://elasticsearch.pdphilip.com/migrations
684 | - **Running the Migration**: To execute the migration and rebuild all your indexed, use the following command:
685 |
686 | ```bash
687 | php artisan lens:migrate User
688 | ```
689 |
690 | This command will delete the existing index, run the migration, and rebuild all records.
691 |
692 | # Step 6: Fine-tune the Observers
693 |
694 | ## [Docs → Base Model Observers](https://elasticsearch.pdphilip.com/elasticlens/model-observers/)
695 |
696 | By default, the `Base Model` is observed for changes (saves) and deletions. When the `Base Model` is deleted, the corresponding `Index Model` will also be deleted, even in cases of soft deletion.
697 |
698 | ### Handling Embedded Models
699 |
700 | The related models are also observed when you define a `fieldMap()` with embedded fields. For example:
701 |
702 | - A save or delete action on `ProfileStatus` will trigger a chain reaction, fetching the related `Profile` and then `User`, which in turn initiates a rebuild of the `IndexedUser`.
703 |
704 | However, to ensure these observers are loaded, you need to reference the User model explicitly:
705 |
706 | ```php
707 | //This alone will not trigger a rebuild
708 | $profileStatus->status = 'Unavailable';
709 | $profileStatus->save();
710 |
711 | //This will since the observers are loaded in the User model
712 | new User::class
713 | $profileStatus->status = 'Unavailable';
714 | $profileStatus->save();
715 | ```
716 |
717 | ### Customising Observers
718 |
719 |
720 | If you want ElasticLens to observe `ProfileStatus` without requiring a reference to `User`, follow these steps:
721 |
722 | 1. Add the `HasWatcher` Trait to `ProfileStatus`:
723 |
724 | ```php
725 | use PDPhilip\ElasticLens\HasWatcher;
726 |
727 | class ProfileStatus extends Eloquent
728 | {
729 | use HasWatcher;
730 | ```
731 |
732 | 2. Define the Watcher in the `elasticlens.php` Config File:
733 |
734 | ```php
735 | 'watchers' => [
736 | \App\Models\ProfileStatus::class => [
737 | \App\Models\Indexes\IndexedUser::class,
738 | ],
739 | ],
740 | ```
741 |
742 | ### Disabling Base-Model Observation
743 |
744 | If you want to disable the automatic observation of the `Base-Model`, include the following in your `Index-Model`:
745 |
746 | ```php
747 | class IndexedUser extends IndexModel
748 | {
749 | protected $baseModel = User::class;
750 |
751 | protected $observeBase = false;
752 |
753 | ```
754 |
755 | ---
756 |
757 | # Step 7: Monitor and administer all your indexes with Artisan commands
758 |
759 | ## [Docs → Artisan CLI Tools](https://elasticsearch.pdphilip.com/elasticlens/artisan-cli-tools/)
760 |
761 | Use the following Artisan commands to manage and monitor your Elasticsearch indexes:
762 |
763 | 1. Check Overall Status:
764 |
765 | ```bash
766 | php artisan lens:status
767 | ```
768 |
769 | Displays the overall status of all your indexes and the ElasticLens configuration.
770 |
771 |
772 |
776 |
785 |
789 |
801 |
805 |
816 |
820 |
831 |
835 |
846 | Fields
847 |
848 | ### Model Fields:
849 |
850 | - string `$model`: The Base-Model being indexed.
851 | - string `$model_id`: The ID of the Base-Model.
852 | - string `$index_model`: The corresponding Index-Model.
853 | - string `$last_source`: The last source of the build state.
854 | - IndexableStateType `$state`: The current state of the index build.
855 | - array `$state_data`: Additional data related to the build state.
856 | - array `$logs`: Logs of the indexing process.
857 | - Carbon `$created_at`: Timestamp of when the build state was created.
858 | - Carbon `$updated_at`: Timestamp of the last update to the build state.
859 |
860 | ### Attributes:
861 |
862 | - @property-read string `$state_name`: The name of the current state.
863 | - @property-read string `$state_color`: The colour associated with the current state.
864 |
865 |
866 |
867 |
868 | Built-in methods include:
869 |
870 | ```php
871 | IndexableBuild::returnState($model, $modelId, $indexModel);
872 | IndexableBuild::countModelErrors($indexModel);
873 | IndexableBuild::countModelRecords($indexModel);
874 | ```
875 |
876 | **Note**: While you can query the `IndexableBuild` model directly, avoid writing or deleting records within it manually, as this can interfere with the health checks and overall integrity of the indexing process. The model should be
877 | used for reading purposes only to ensure accurate monitoring and reporting.
878 |
879 | ---
880 |
881 | # Step 9: Optionally Access the Built-in `IndexableMigrationLog` Model for Index Migration Status
882 |
883 | ## [Docs → Access IndexableMigrationLog model](https://elasticsearch.pdphilip.com/elasticlens/build-migration-states/#access-indexablemigrationlog-model)
884 |
885 | ElasticLens includes a built-in `IndexableMigrationLog` model for monitoring and tracking the state of index migrations. This model logs each migration related to an `Index-Model`.
886 |
887 |
888 | Fields
889 |
890 | - string `$index_model`: The migrated Index-Model.
891 | - IndexableMigrationLogState `$state`: State of the migration
892 | - array `$map`: Migration map passed to Elasticsearch.
893 | - int `$version_major`: Major version of the indexing process.
894 | - int `$version_minor`: Minor version of the indexing process.
895 | - Carbon `$created_at`: Timestamp of when the migration was created.
896 |
897 | ### Attributes:
898 |
899 | - @property-read string `$version`: Parsed version, ex v2.03
900 | - @property-read string `$state_name`: Current state name.
901 | - @property-read string `$state_color`: Colour representing the current state.
902 |
903 |
904 |
905 |
906 | Built-in methods include:
907 |
908 | ```php
909 | IndexableMigrationLog::getLatestVersion($indexModel);
910 | IndexableMigrationLog::getLatestMigration($indexModel);
911 | ```
912 |
913 | **Note**: While you can query the `IndexableMigrationLog` model directly, avoid writing or deleting records within it manually, as this can interfere with versioning of the migrations. The model should be used for reading purposes only, to
914 | ensure accuracy.
915 |
916 | ---
917 |
918 | ## Credits
919 |
920 | - [David Philip](https://github.com/pdphilip)
921 |
922 | ## License
923 |
924 | The MIT License (MIT). Please see [License File](LICENSE.md) for more information.
925 |
--------------------------------------------------------------------------------
/composer.json:
--------------------------------------------------------------------------------
1 | {
2 | "name": "pdphilip/elasticlens",
3 | "description": "Search your Laravel models with the convenience of Eloquent and the power of Elasticsearch",
4 | "keywords": [
5 | "PDPhilip",
6 | "laravel",
7 | "Elasticsearch",
8 | "Eloquent",
9 | "elasticlens"
10 | ],
11 | "homepage": "https://github.com/pdphilip/elasticlens",
12 | "license": "MIT",
13 | "authors": [
14 | {
15 | "name": "David Philip",
16 | "email": "pd.philip@gmail.com",
17 | "role": "Developer"
18 | }
19 | ],
20 | "require": {
21 | "php": "^8.2",
22 | "illuminate/contracts": "^10.0||^11.0||^12.0",
23 | "pdphilip/elasticsearch": "^5.0.2",
24 | "pdphilip/omniterm": "^1.0.5",
25 | "spatie/laravel-package-tools": "^1.16"
26 | },
27 | "require-dev": {
28 | "laravel/pint": "^1.14",
29 | "nunomaduro/collision": "^8.1.1||^7.10.0",
30 | "larastan/larastan": "^2.9",
31 | "orchestra/testbench": "^9.0.0||^8.22.0",
32 | "pestphp/pest": "^2.34",
33 | "pestphp/pest-plugin-arch": "^2.7",
34 | "pestphp/pest-plugin-laravel": "^2.3",
35 | "phpstan/extension-installer": "^1.3",
36 | "phpstan/phpstan-deprecation-rules": "^1.1",
37 | "phpstan/phpstan-phpunit": "^1.3"
38 | },
39 | "autoload": {
40 | "psr-4": {
41 | "PDPhilip\\ElasticLens\\": "src/"
42 | }
43 | },
44 | "autoload-dev": {
45 | "psr-4": {
46 | "PDPhilip\\ElasticLens\\Tests\\": "tests/",
47 | "Workbench\\App\\": "workbench/app/"
48 | }
49 | },
50 | "scripts": {
51 | "post-autoload-dump": "@composer run prepare",
52 | "clear": "@php vendor/bin/testbench package:purge-elasticlens --ansi",
53 | "prepare": "@php vendor/bin/testbench package:discover --ansi",
54 | "build": [
55 | "@composer run prepare",
56 | "@php vendor/bin/testbench workbench:build --ansi"
57 | ],
58 | "start": [
59 | "Composer\\Config::disableProcessTimeout",
60 | "@composer run build",
61 | "@php vendor/bin/testbench serve"
62 | ],
63 | "types": "phpstan analyse --ansi --memory-limit=2G",
64 | "lint": "pint -v",
65 | "pest": "pest --colors=always",
66 | "test-coverage": "pest --coverage",
67 | "test:lint": "pint --test -v",
68 | "test:types": "@types",
69 | "test:unit": "pest --colors=always",
70 | "test": [
71 | "@test:lint",
72 | "@test:types",
73 | "@test:unit"
74 | ]
75 | },
76 | "config": {
77 | "sort-packages": true,
78 | "allow-plugins": {
79 | "pestphp/pest-plugin": true,
80 | "phpstan/extension-installer": true,
81 | "php-http/discovery": true
82 | }
83 | },
84 | "extra": {
85 | "laravel": {
86 | "providers": [
87 | "PDPhilip\\ElasticLens\\ElasticLensServiceProvider"
88 | ]
89 | }
90 | },
91 | "minimum-stability": "dev",
92 | "prefer-stable": true
93 | }
94 |
--------------------------------------------------------------------------------
/config/elasticlens.php:
--------------------------------------------------------------------------------
1 | 'elasticsearch',
5 |
6 | 'queue' => null, // Set queue to use for dispatching index builds, ex: default, high, low, etc.
7 |
8 | 'watchers' => [
9 | // \App\Models\Profile::class => [
10 | // \App\Models\Indexes\IndexedUser::class,
11 | // ],
12 | ],
13 |
14 | 'index_build_state' => [
15 | 'enabled' => true, // Recommended to keep this enabled
16 | 'log_trim' => 2, // If null, the logs field will be empty
17 | ],
18 |
19 | 'index_migration_logs' => [
20 | 'enabled' => true, // Recommended to keep this enabled
21 | ],
22 | 'namespaces' => [
23 | 'App\Models' => 'App\Models\Indexes',
24 | ],
25 |
26 | 'index_paths' => [
27 | 'app/Models/Indexes/' => 'App\Models\Indexes',
28 | ],
29 | ];
30 |
--------------------------------------------------------------------------------
/database/migrations/create_indexable_build_index.php:
--------------------------------------------------------------------------------
1 | deleteIfExists('indexable_builds');
14 |
15 | Schema::on($connectionName)->create('indexable_builds', function (Blueprint $index) {
16 | $index->keyword('model');
17 | $index->keyword('model_id');
18 | $index->keyword('index_model');
19 | $index->keyword('state');
20 | $index->keyword('last_source');
21 | $index->text('last_source');
22 |
23 | $index->flattened('state_data')->indexField(false);
24 | $index->flattened('logs')->indexField(false);
25 | });
26 | }
27 |
28 | public function down()
29 | {
30 | $connectionName = config('elasticlens.connection') ?? 'elasticsearch';
31 |
32 | Schema::on($connectionName)->deleteIfExists('indexable_builds');
33 | }
34 | };
35 |
--------------------------------------------------------------------------------
/database/migrations/create_indexable_migration_logs_index.php:
--------------------------------------------------------------------------------
1 | deleteIfExists('indexable_migration_logs');
14 |
15 | Schema::on($connectionName)->create('indexable_migration_logs', function (Blueprint $index) {
16 | $index->keyword('index_model');
17 | $index->keyword('state');
18 | $index->integer('version_major');
19 | $index->integer('version_minor');
20 | $index->flattened('map')->indexField(false);
21 | });
22 | }
23 |
24 | public function down()
25 | {
26 | $connectionName = config('elasticlens.connection') ?? 'elasticsearch';
27 |
28 | Schema::on($connectionName)->deleteIfExists('indexable_migration_logs');
29 | }
30 | };
31 |
--------------------------------------------------------------------------------
/resources/stubs/IndexedBase.php.stub:
--------------------------------------------------------------------------------
1 |
10 |
11 | @include('elasticlens::cli.components.loader-spin',['message' => $title,'i' => $i,'state' => $state])
12 | @include('elasticlens::cli.components.hr')
13 | @include('elasticlens::cli.components.data-row-value',['key' => 'Created','value' => $created,'class' => 'text-sky-500'])
14 | @include('elasticlens::cli.components.data-row-value',['key' => 'Updated','value' => $updated,'class' => 'text-emerald-500'])
15 | @include('elasticlens::cli.components.data-row-value',['key' => 'Skipped','value' => $skipped,'class' => 'text-amber-500'])
16 | @include('elasticlens::cli.components.data-row-value',['key' => 'Failed','value' => $failed,'class' => 'text-rose-500'])
17 |
--------------------------------------------------------------------------------
/resources/views/cli/components/code-trait.blade.php:
--------------------------------------------------------------------------------
1 |
2 |
3 | namespace App\Models;
4 |
5 | use PDPhilip\ElasticLens\Indexable;
6 |
7 | class {{$model}} extends Model
8 | {
9 | use Indexable;
10 |
11 |
12 |
--------------------------------------------------------------------------------
/resources/views/cli/components/data-row-value.blade.php:
--------------------------------------------------------------------------------
1 |
26 |
27 |
28 | {{ $name }}
29 |
30 |
31 | {{$value}}
32 |
33 |
34 |
9 |
10 |
--------------------------------------------------------------------------------
/resources/views/cli/components/loader-spin.blade.php:
--------------------------------------------------------------------------------
1 | $intervals) {
7 | $i -= $intervals;
8 | $j++;
9 | if ($j > 3) {
10 | $j = 0;
11 | }
12 | }
13 |
14 | $show = $characters[$i];
15 | $textColor = $colors[$j];
16 | switch ($state) {
17 | case 'success':
18 | $textColor = "text-emerald-500";
19 | $show = "✔";
20 | break;
21 | case 'warning':
22 | $textColor = "text-amber-500";
23 | $show = "⚠";
24 | break;
25 | case 'failover':
26 | $textColor = "text-amber-500";
27 | $show = "◴";
28 | break;
29 | case 'error':
30 | $textColor = "text-rose-500";
31 | $show = "✘";
32 | break;
33 |
34 | }
35 | ?>
36 |
37 |
38 | {{ $show }}
39 |
40 | {{$message}}
41 | @if(!empty($details))
42 | {{$details}}
43 | @endif
44 |
--------------------------------------------------------------------------------
/resources/views/cli/components/title-row.blade.php:
--------------------------------------------------------------------------------
1 |
4 |
5 |
6 |
7 |
8 |
9 |
10 | {{$t}}
11 |
12 |
13 |
14 |
15 |
16 |
--------------------------------------------------------------------------------
/resources/views/cli/components/title.blade.php:
--------------------------------------------------------------------------------
1 |
4 |
5 | @include('elasticlens::cli.components.title-row')
6 | @include('elasticlens::cli.components.title-row',['t' => $title])
7 | @include('elasticlens::cli.components.title-row')
8 |
--------------------------------------------------------------------------------
/src/Builder/IndexBuilder.php:
--------------------------------------------------------------------------------
1 | for($model);
31 | $mapper->setBaseMap($mapper);
32 | if (is_callable($callback)) {
33 | $callback(new IndexField($mapper));
34 | }
35 |
36 | return $mapper;
37 | }
38 |
39 | public function getFieldMap()
40 | {
41 | return $this->fields;
42 | }
43 |
44 | public function getObservers()
45 | {
46 | return $this->baseMap->observers;
47 | }
48 |
49 | public function getRelationships()
50 | {
51 | return $this->baseMap->relationships;
52 | }
53 |
54 | public function for($model): static
55 | {
56 | $this->model = $model;
57 |
58 | return $this;
59 | }
60 |
61 | public function addField($field, $type): static
62 | {
63 | $this->fields[$field] = $type;
64 |
65 | return $this;
66 | }
67 |
68 | public function setEmbedField($field): static
69 | {
70 | $this->embedFieldName = $field;
71 |
72 | return $this;
73 | }
74 |
75 | public function setParentMap($parent): static
76 | {
77 | $this->parentMap = $parent;
78 |
79 | return $this;
80 | }
81 |
82 | public function setBaseMap($builder): static
83 | {
84 | $this->baseMap = $builder;
85 |
86 | return $this;
87 | }
88 |
89 | public function setRelationship($field, $relationship): static
90 | {
91 | $this->baseMap->relationships[$field] = $relationship;
92 |
93 | return $this;
94 | }
95 |
96 | public function getBaseMap()
97 | {
98 | return $this->baseMap;
99 | }
100 |
101 | // ----------------------------------------------------------------------
102 | // Embedded Methods
103 | // ----------------------------------------------------------------------
104 |
105 | public function isEmbedded(): bool
106 | {
107 | return ! empty($this->embedFieldName) && $this->parentMap;
108 | }
109 |
110 | // Add an embedded relationship to the map
111 | public function addEmbed($field, $relation, $type, $whereRelatedField = null, $equalsLocalField = null, $query = null): IndexBuilder
112 | {
113 | $current = $this;
114 |
115 | $mapper = new self;
116 | $mapper->for($relation);
117 | $mapper->setEmbedField($field);
118 | $mapper->setParentMap($current);
119 | $mapper->setBaseMap($current->getBaseMap());
120 | $relationship = $mapper->buildRelationship($mapper->parentMap->model, $relation, $type, $whereRelatedField, $equalsLocalField, $query);
121 | $mapper->attachEmbeddedObserver($relationship);
122 | $mapper->setRelationship($field, $relationship);
123 |
124 | return $mapper;
125 | }
126 |
127 | // Embed map for related models
128 | public function embedMap($callback): self
129 | {
130 | if (! $this->isEmbedded()) {
131 | throw new RuntimeException('Embedded Maps can only be called for embedded fields');
132 | }
133 |
134 | if (is_callable($callback)) {
135 | $callback(new IndexField($this));
136 | }
137 |
138 | $this->parentMap->addField($this->embedFieldName, $this->fields);
139 |
140 | return $this;
141 | }
142 |
143 | public function attachEmbeddedObserver($relationship): void
144 | {
145 | $this->baseMap->_attachObserver($relationship);
146 |
147 | }
148 |
149 | public function attachBaseObserver(): void
150 | {
151 | $relationship = $this->buildRelationship($this->model, $this->model, 'base');
152 | $this->baseMap->_attachObserver($relationship);
153 | }
154 |
155 | public function dontObserve(): self
156 | {
157 | if (! $this->isEmbedded()) {
158 | return $this;
159 | }
160 | $observers = $this->baseMap->observers;
161 | $observer = array_pop($observers);
162 | $observer['observe'] = false;
163 | $observers[] = $observer;
164 | $this->baseMap->observers = $observers;
165 |
166 | return $this;
167 | }
168 |
169 | public function _attachObserver($relationship): void
170 | {
171 | $observers = $this->baseMap->observers;
172 | $observers[] = $relationship;
173 |
174 | $this->baseMap->observers = $observers;
175 | }
176 |
177 | protected function buildRelationship($baseModel, $relation, $type, $whereRelatedField = null, $equalsLocalField = null, $query = null): array
178 | {
179 | if (! $whereRelatedField || $equalsLocalField) {
180 | [$whereRelatedField, $equalsLocalField] = $this->_inferKeys($baseModel, $relation, $type, $whereRelatedField, $equalsLocalField);
181 | }
182 |
183 | return [
184 | 'observe' => true,
185 | 'model' => $baseModel,
186 | 'type' => $type,
187 | 'relation' => $relation,
188 | 'whereRelatedField' => $whereRelatedField,
189 | 'equalsModelField' => $equalsLocalField,
190 | 'query' => $query,
191 | ];
192 | }
193 |
194 | private function _inferKeys($baseModel, $relation, $type, $foreignKey = null, $localKey = null): array
195 | {
196 |
197 | if ($type === 'base') {
198 | $base = (new $baseModel);
199 | $id = $base->getKeyName();
200 | $foreignKey = $id;
201 | $localKey = $id;
202 | }
203 |
204 | if ($type == 'belongsTo') {
205 | if (! $foreignKey) {
206 | $base = (new $baseModel);
207 | $foreignKey = $base->getKeyName();
208 | }
209 | if (! $localKey) {
210 | $rel = (new $relation);
211 | $table = $rel->getTable();
212 | $localKey = Str::singular($table).'_id';
213 | }
214 |
215 | }
216 |
217 | if (in_array($type, ['hasMany', 'hasOne'])) {
218 | if (! $foreignKey) {
219 | $base = (new $baseModel);
220 | $table = $base->getTable();
221 | $foreignKey = Str::singular($table).'_id';
222 |
223 | }
224 | if (! $localKey) {
225 | $rel = (new $relation);
226 | $localKey = $rel->getKeyName();
227 | }
228 |
229 | }
230 |
231 | return [$foreignKey, $localKey];
232 | }
233 | }
234 |
--------------------------------------------------------------------------------
/src/Builder/IndexField.php:
--------------------------------------------------------------------------------
1 | mapper = $mapper;
16 | }
17 |
18 | // ----------------------------------------------------------------------
19 | // Field sets
20 | // ----------------------------------------------------------------------
21 |
22 | public function text($field): IndexField|static
23 | {
24 | return $this->addField($field, 'string');
25 | }
26 |
27 | public function integer($field): IndexField|static
28 | {
29 | return $this->addField($field, 'integer');
30 | }
31 |
32 | public function array($field): IndexField|static
33 | {
34 | return $this->addField($field, 'array');
35 | }
36 |
37 | public function bool($field): IndexField|static
38 | {
39 | return $this->addField($field, 'bool');
40 | }
41 |
42 | public function carbon($field): IndexField|static
43 | {
44 | return $this->addField($field, Carbon::class);
45 | }
46 |
47 | public function type($field, $type): IndexField|static
48 | {
49 | return $this->addField($field, $type);
50 | }
51 |
52 | // ----------------------------------------------------------------------
53 | // Embeds
54 | // ----------------------------------------------------------------------
55 |
56 | public function embedsMany($field, $relation, $whereRelatedField = null, $equalsLocalField = null, $query = null)
57 | {
58 | return $this->addEmbed($field, $relation, 'hasMany', $whereRelatedField, $equalsLocalField, $query);
59 | }
60 |
61 | public function embedsOne($field, $relation, $whereRelatedField = null, $equalsLocalField = null, $query = null)
62 | {
63 | return $this->addEmbed($field, $relation, 'hasOne', $whereRelatedField, $equalsLocalField, $query);
64 | }
65 |
66 | public function embedsBelongTo($field, $relation, $whereRelatedField = null, $equalsLocalField = null, $query = null)
67 | {
68 | return $this->addEmbed($field, $relation, 'belongsTo', $whereRelatedField, $equalsLocalField, $query);
69 | }
70 |
71 | // ----------------------------------------------------------------------
72 | // Builder entry points
73 | // ----------------------------------------------------------------------
74 |
75 | protected function addField($field, $type): static
76 | {
77 | $this->mapper->addField($field, $type);
78 |
79 | return $this;
80 | }
81 |
82 | protected function addEmbed($field, $relation, $type, $whereRelatedField = null, $equalsLocalField = null, $query = null): IndexBuilder
83 | {
84 | return $this->mapper->addEmbed($field, $relation, $type, $whereRelatedField, $equalsLocalField, $query);
85 |
86 | }
87 | }
88 |
--------------------------------------------------------------------------------
/src/Commands/LensBuildCommand.php:
--------------------------------------------------------------------------------
1 | initOmni();
58 | $model = $this->argument('model');
59 | $modelCheck = QualifyModel::check($model);
60 | if (! $modelCheck['qualified']) {
61 | $this->omni->statusError('ERROR', 'Model not found', ['Model: '.$model]);
62 |
63 | return self::FAILURE;
64 | }
65 | $model = $modelCheck['qualified'];
66 | $this->model = $model;
67 | $this->indexModel = Lens::fetchIndexModelClass($this->model);
68 | $this->newLine();
69 | $name = Str::plural($this->model);
70 | render((string) view('elasticlens::cli.components.title', ['title' => 'Rebuild '.$name, 'color' => 'cyan']));
71 | $this->newLine();
72 | $health = new LensState($this->indexModel);
73 | $this->baseModel = $health->baseModel;
74 | if (! $health->indexExists) {
75 | $this->omni->statusError('ERROR', $health->indexModelTable.' index not found');
76 |
77 | $this->migrate = null;
78 | $this->migrationStep();
79 | }
80 | $health = new LensState($this->indexModel);
81 |
82 | if (! $health->indexExists) {
83 | $this->omni->statusError('ERROR', 'Index required', [
84 | 'Migrate to create the "'.$health->indexModelTable.'" index',
85 | ]);
86 |
87 | return self::FAILURE;
88 | }
89 | $this->setChunkRate($health);
90 | $built = $this->processAsyncBuild($health, $this->model);
91 |
92 | return $built ? self::SUCCESS : self::FAILURE;
93 | }
94 |
95 | /**
96 | * @throws Exception
97 | */
98 | private function processAsyncBuild($health, $model): bool
99 | {
100 | try {
101 | $recordsCount = $health->baseModel::count();
102 | } catch (Exception $e) {
103 | $this->omni->statusError('ERROR', 'Base Model not found', [
104 | $e->getMessage(),
105 | ]);
106 |
107 | return false;
108 | }
109 | if (! $recordsCount) {
110 | $this->omni->statusWarning('BUILD SKIPPED', 'No records found for '.$health->baseModel);
111 |
112 | return false;
113 | }
114 | $this->startTimer();
115 |
116 | $async = asyncFunction(function () {});
117 | $async->render((string) view('elasticlens::cli.bulk', [
118 | 'screenWidth' => $async->getScreenWidth(),
119 | 'model' => $this->model,
120 | 'i' => $async->getInterval(),
121 | 'created' => $this->created,
122 | 'skipped' => $this->skipped,
123 | 'updated' => $this->modified,
124 | 'failed' => $this->failed,
125 | 'completed' => false,
126 | 'took' => false,
127 | ]));
128 | $this->baseModel::chunk($this->chunkRate, function ($records) use ($async) {
129 | $result = $async->withTask(function () use ($records) {
130 | return $this->bulkInsertTask($records);
131 | })->run(function () use ($async) {
132 | $async->render((string) view('elasticlens::cli.bulk', [
133 | 'screenWidth' => $async->getScreenWidth(),
134 | 'model' => $this->model,
135 | 'i' => $async->getInterval(),
136 | 'created' => $this->created,
137 | 'skipped' => $this->skipped,
138 | 'updated' => $this->modified,
139 | 'failed' => $this->failed,
140 | 'completed' => false,
141 | 'took' => false,
142 | ]));
143 | });
144 | $this->created += $result['created'];
145 | $this->skipped += $result['skipped'];
146 | $this->modified += $result['modified'];
147 | $this->failed += $result['failed'];
148 | });
149 |
150 | $async->render((string) view('elasticlens::cli.bulk', [
151 | 'screenWidth' => $async->getScreenWidth(),
152 | 'model' => $model,
153 | 'i' => $async->getInterval(),
154 | 'created' => $this->created,
155 | 'skipped' => $this->skipped,
156 | 'updated' => $this->modified,
157 | 'failed' => $this->failed,
158 | 'completed' => true,
159 | ]));
160 |
161 | $this->newLine();
162 | $name = Str::plural($model);
163 | $total = $this->created + $this->modified;
164 | $time = $this->getTime();
165 | if ($total > 0) {
166 | $total = number_format($total);
167 | $this->omni->info('Indexed '.$total.' '.$name.' in '.$time['sec'].' seconds');
168 | } else {
169 | $this->omni->error('All indexes failed to build');
170 | }
171 |
172 | $this->newLine();
173 |
174 | return true;
175 | }
176 |
177 | /**
178 | * @throws Exception
179 | */
180 | public function bulkInsertTask($records): array
181 | {
182 | $bulk = new BulkIndexer($this->baseModel);
183 | $bulk->setRecords($records)->build();
184 | $result = $bulk->getResult();
185 |
186 | return [
187 | 'total' => $result['results']['total'],
188 | 'created' => $result['results']['created'],
189 | 'skipped' => $result['results']['skipped'],
190 | 'modified' => $result['results']['modified'],
191 | 'failed' => $result['results']['failed'],
192 | ];
193 | }
194 |
195 | public function setChunkRate(LensState $health): void
196 | {
197 | $chunk = $this->chunkRate;
198 | $relationships = count($health->indexModelInstance->getRelationships());
199 | if ($relationships > 3) {
200 | $chunk = 750;
201 | }
202 | if ($relationships > 6) {
203 | $chunk = 500;
204 | }
205 | if ($relationships > 9) {
206 | $chunk = 250;
207 | }
208 | $this->chunkRate = $chunk;
209 | }
210 | }
211 |
--------------------------------------------------------------------------------
/src/Commands/LensCommands.php:
--------------------------------------------------------------------------------
1 | newLine();
23 | $this->omni->status($loadError['status'], $loadError['name'], $loadError['title'], $loadError['help']);
24 |
25 | $this->newLine();
26 |
27 | return false;
28 | }
29 | try {
30 | $check = HealthCheck::check($model);
31 | if (! empty($check['configStatusHelp']['critical'])) {
32 | foreach ($check['configStatusHelp']['critical'] as $critical) {
33 | $this->newLine();
34 | $this->omni->statusError('ERROR', $critical['name'], $critical['help']);
35 | }
36 | $this->newLine();
37 |
38 | return false;
39 | }
40 | } catch (Exception $e) {
41 | $this->newLine();
42 | $this->omni->statusError('ERROR', $e->getMessage());
43 | $this->newLine();
44 |
45 | return false;
46 | }
47 |
48 | return true;
49 | }
50 |
51 | // ----------------------------------------------------------------------
52 | // Migrations
53 | // ----------------------------------------------------------------------
54 |
55 | public function migrationStep(): void
56 | {
57 | while (! in_array($this->migrate, ['yes', 'no', 'y', 'n'])) {
58 | $this->migrate = $this->omni->ask('Migrate index?', ['yes', 'no']);
59 | }
60 | if (in_array($this->migrate, ['yes', 'y'])) {
61 | $this->migrationPassed = $this->processMigration($this->indexModel);
62 | } else {
63 | $this->migrationPassed = true;
64 | $this->newLine();
65 | $this->omni->info('Index migration skipped');
66 | }
67 | }
68 |
69 | private function processMigration($indexModel): bool
70 | {
71 | $this->omni->newLoader();
72 | $result = $this->omni->runTask('Migrating Index', function () use ($indexModel) {
73 | try {
74 | $migration = new LensMigration($indexModel);
75 | $migration->runMigration();
76 |
77 | return [
78 | 'state' => 'success',
79 | 'message' => 'Migration Successful',
80 | 'details' => '',
81 | ];
82 | } catch (Exception $e) {
83 | return [
84 | 'state' => 'error',
85 | 'message' => 'Migration Failed',
86 | 'details' => $e->getMessage(),
87 | ];
88 | }
89 | });
90 | $this->newLine();
91 |
92 | return ! empty($result['state']) && $result['state'] === 'success';
93 |
94 | }
95 | }
96 |
--------------------------------------------------------------------------------
/src/Commands/LensHealthCommand.php:
--------------------------------------------------------------------------------
1 | initOmni();
28 | $model = $this->argument('model');
29 |
30 | $loadError = HealthCheck::loadErrorCheck($model);
31 | $this->newLine();
32 | if ($loadError) {
33 | $this->omni->status($loadError['status'], $loadError['name'], $loadError['title'], $loadError['help']);
34 | $this->newLine();
35 |
36 | return self::FAILURE;
37 | }
38 | $health = HealthCheck::check($model);
39 | render((string) view('elasticlens::cli.components.title', ['title' => $health['title'], 'color' => 'emerald']));
40 | $this->omni->status($health['indexStatus']['status'], $health['indexStatus']['title'], $health['indexStatus']['name'], $health['indexStatus']['help'] ?? []);
41 | $this->newLine();
42 | $this->omni->header('Index Model', 'Value');
43 | foreach ($health['indexData'] as $detail => $value) {
44 | $this->omni->row($detail, $value);
45 | }
46 | $this->newLine();
47 | $this->omni->header('Base Model', 'Value');
48 | foreach ($health['modelData'] as $detail => $value) {
49 | $this->omni->row($detail, $value);
50 | }
51 | $this->newLine();
52 | $this->omni->header('Build Data', 'Value');
53 | foreach ($health['buildData'] as $detail => $value) {
54 | $this->omni->row($detail, $value);
55 | }
56 | $this->omni->status($health['configStatus']['status'], $health['configStatus']['name'], $health['configStatus']['title'], $health['configStatus']['help'] ?? []);
57 | $this->newLine();
58 | $this->omni->header('Config', 'Value');
59 | foreach ($health['configData'] as $detail => $value) {
60 | $this->omni->row($detail, $value);
61 | }
62 | $this->newLine();
63 | if (! $health['observers']) {
64 | $this->omni->warning('No observers found');
65 | } else {
66 | $this->omni->header('Observed Model', 'Type');
67 | foreach ($health['observers'] as $observer) {
68 | $this->omni->row($observer['key'], $observer['value']);
69 | }
70 | }
71 | if ($health['configStatusHelp']['critical'] || $health['configStatusHelp']['warning']) {
72 | $this->newLine();
73 | $this->omni->info('Config Help');
74 | if ($health['configStatusHelp']['critical']) {
75 | foreach ($health['configStatusHelp']['critical'] as $critical) {
76 | $this->omni->statusError('Config Error', $critical['name'], $critical['help'] ?? []);
77 | $this->newLine();
78 | }
79 | }
80 | if ($health['configStatusHelp']['warning']) {
81 | foreach ($health['configStatusHelp']['warning'] as $warning) {
82 | $this->omni->statusWarning('Config Recommendation', $warning['name'], $warning['help'] ?? []);
83 | $this->newLine();
84 | }
85 | }
86 | }
87 |
88 | return self::SUCCESS;
89 | }
90 | }
91 |
--------------------------------------------------------------------------------
/src/Commands/LensMakeCommand.php:
--------------------------------------------------------------------------------
1 | initOmni();
27 |
28 | $this->newLine();
29 | $model = $this->argument('model');
30 | // ensure casing is correct
31 | $model = Str::studly($model);
32 |
33 | // Check if model exists
34 | $modelFound = null;
35 | $indexedModel = null;
36 | $namespaces = config('elasticlens.namespaces');
37 | $paths = config('elasticlens.index_paths');
38 |
39 | $config = [
40 | 'model' => [
41 | 'name' => '',
42 | 'namespace' => '',
43 | 'full' => '',
44 | ],
45 | 'index' => [
46 | 'name' => '',
47 | 'namespace' => '',
48 | 'full' => '',
49 | 'path' => '',
50 | ],
51 |
52 | ];
53 |
54 | $notFound = [];
55 | foreach ($namespaces as $modelNamespace => $indexNameSpace) {
56 | $modelCheck = $modelNamespace.'\\'.$model;
57 | if ($this->class_exists_case_sensitive($modelCheck)) {
58 | $modelFound = $modelCheck;
59 | $config['model']['name'] = $model;
60 | $config['model']['namespace'] = $modelNamespace;
61 | $config['model']['full'] = $modelCheck;
62 | $config['index']['name'] = 'Indexed'.$model;
63 | $config['index']['namespace'] = $indexNameSpace;
64 | $config['index']['full'] = $indexNameSpace.'\\'.$config['index']['name'];
65 |
66 | $path = array_search($indexNameSpace, $paths);
67 | if (! $path) {
68 | $this->omni->statusError('ERROR', 'Path for namespace '.$indexNameSpace.' not found', [
69 | 'Namespace found: '.$indexNameSpace,
70 | 'Check config("elasticlens.index_paths") for the correct {path} => \''.$indexNameSpace.'\'',
71 | ]);
72 | $this->newLine();
73 |
74 | return self::FAILURE;
75 | }
76 | $config['index']['path'] = $path;
77 | break;
78 | } else {
79 | $notFound[] = $modelCheck;
80 | }
81 | }
82 |
83 | if (! $modelFound) {
84 | foreach ($notFound as $modelCheck) {
85 | $this->omni->statusError('ERROR', 'Base Model ('.$model.') was not found at: '.$modelCheck);
86 | $this->newLine();
87 | }
88 |
89 | return self::FAILURE;
90 | }
91 | if ($this->class_exists_case_sensitive($config['index']['full'])) {
92 | $this->omni->statusError('ERROR', 'Indexed Model (for '.$model.' Model) already exists at: '.$config['index']['full']);
93 |
94 | return self::FAILURE;
95 | }
96 |
97 | $path = $config['index']['path'].$config['index']['name'];
98 |
99 | $finalPath = $this->getPath($path);
100 | // Make sure the directory exists
101 | $this->makeDirectory($finalPath);
102 |
103 | // Get the stub file contents
104 | $stub = $this->files->get($this->getStub());
105 | // Replace the stub variables
106 | $stub = $this->replaceNamespaceCustom($stub, $config['model']['namespace']);
107 | $stub = $this->replaceModel($stub, $config['model']['name']);
108 |
109 | // Write the file to disk
110 | $this->files->put($finalPath, $stub);
111 |
112 | $this->omni->statusSuccess('SUCCESS', 'Indexed Model (for '.$model.' Model) created at: '.$indexedModel);
113 | $this->omni->statusInfo('1', 'Add the Indexable trait to your '.$model.' model');
114 | render((string) view('elasticlens::cli.components.code-trait', ['model' => $model]));
115 | $this->newLine();
116 | $this->omni->statusInfo('2', 'Then run: "php artisan lens:build '.$model.'" to index your model');
117 |
118 | return self::SUCCESS;
119 | }
120 |
121 | protected $type = 'Model';
122 |
123 | protected function getStub(): string
124 | {
125 | $stubPath = __DIR__.'/../../resources/stubs/IndexedBase.php.stub';
126 |
127 | if (! file_exists($stubPath)) {
128 | throw new RuntimeException('Stub file not found: '.$stubPath);
129 | }
130 |
131 | return $stubPath;
132 | }
133 |
134 | protected function getPath($name)
135 | {
136 | $name = Str::replaceFirst($this->rootNamespace(), '', $name);
137 |
138 | return $this->laravel['path.base'].'/'.str_replace('\\', '/', $name).'.php';
139 | }
140 |
141 | public function replaceNamespaceCustom($stub, $namespace): string
142 | {
143 | return str_replace('{{ namespace }}', $namespace, $stub);
144 | }
145 |
146 | public function replaceModel($stub, $name): string
147 | {
148 | return str_replace('{{ model }}', $name, $stub);
149 | }
150 |
151 | public function class_exists_case_sensitive(string $class_name): bool
152 | {
153 | if (in_array($class_name, get_declared_classes(), true)) {
154 | return true;
155 | }
156 |
157 | try {
158 | $reflectionClass = new ReflectionClass($class_name);
159 |
160 | return $reflectionClass->getName() === $class_name;
161 | } catch (Exception $e) {
162 | // Class doesn't exist or couldn't be autoloaded
163 | return false;
164 | }
165 |
166 | }
167 | }
168 |
--------------------------------------------------------------------------------
/src/Commands/LensMigrateCommand.php:
--------------------------------------------------------------------------------
1 | false,
32 | 'processed' => 0,
33 | 'success' => 0,
34 | 'skipped' => 0,
35 | 'failed' => 0,
36 | 'total' => 0,
37 | 'state' => 'error',
38 | 'message' => '',
39 | ];
40 |
41 | protected bool $buildPassed = false;
42 |
43 | /**
44 | * @throws Exception
45 | */
46 | public function handle(): int
47 | {
48 | $this->initOmni();
49 | $model = $this->argument('model');
50 | $force = $this->option('force');
51 | $modelCheck = QualifyModel::check($model);
52 | if (! $modelCheck['qualified']) {
53 | $this->omni->statusError('ERROR', 'Model not found', ['Model: '.$model]);
54 |
55 | return self::FAILURE;
56 | }
57 | $model = $modelCheck['qualified'];
58 | $this->model = $model;
59 | $this->indexModel = Lens::fetchIndexModelClass($model);
60 | $this->newLine();
61 | render((string) view('elasticlens::cli.components.title', ['title' => 'Migrate and Build '.class_basename($this->indexModel), 'color' => 'sky']));
62 | $this->newLine();
63 | $this->migrate = $force ? 'yes' : null;
64 | $this->build = $force ? 'yes' : null;
65 | $this->migrationStep();
66 | $this->newLine();
67 | $this->buildStep();
68 | $this->newLine();
69 | $this->showStatus();
70 | $this->newLine();
71 |
72 | return $this->buildPassed ? self::SUCCESS : self::FAILURE;
73 | }
74 |
75 | // ----------------------------------------------------------------------
76 | // Builds
77 | // ----------------------------------------------------------------------
78 |
79 | public function buildStep(): void
80 | {
81 | $question = 'Rebuild Indexes?';
82 | if (! $this->migrationPassed) {
83 | $question = 'Migration Failed. Build Anyway?';
84 | }
85 | while (! in_array($this->build, ['yes', 'no', 'y', 'n'])) {
86 | $this->build = $this->omni->ask($question, ['yes', 'no']);
87 | }
88 | if (in_array($this->build, ['yes', 'y'])) {
89 | $this->buildPassed = $this->processBuild($this->indexModel);
90 | } else {
91 | $this->omni->info('Build Cancelled');
92 | $this->buildPassed = false;
93 | }
94 | }
95 |
96 | private function processBuild($indexModel): bool
97 | {
98 | try {
99 | $builder = new LensBuilder($indexModel);
100 | $recordsCount = $builder->baseModel::count();
101 | } catch (Exception $e) {
102 | $this->omni->statusError('ERROR', 'Base Model not found', [$e->getMessage()]);
103 |
104 | return false;
105 | }
106 | if (! $recordsCount) {
107 | $this->omni->statusWarning('BUILD SKIPPED', 'No records found for '.$builder->baseModel);
108 |
109 | return false;
110 | }
111 | $this->buildData['didRun'] = true;
112 | $this->buildData['total'] = $recordsCount;
113 | $this->omni->createSimpleProgressBar($this->buildData['total']);
114 | $migrationVersion = $builder->fetchCurrentMigrationVersion();
115 | $builder->baseModel::chunk(100, function ($records) use ($builder, $migrationVersion) {
116 | foreach ($records as $record) {
117 | $id = $record->{$builder->baseModelPrimaryKey};
118 | $build = $builder->buildIndex($id, 'Index Rebuild', $migrationVersion);
119 | $this->buildData['processed']++;
120 | if (! empty($build->success)) {
121 | $this->buildData['success']++;
122 | } elseif (! empty($build->skipped)) {
123 | $this->buildData['skipped']++;
124 | } else {
125 | $this->buildData['failed']++;
126 | }
127 | $this->omni->progressAdvance();
128 | }
129 | });
130 | $this->omni->progressFinish();
131 | $this->buildData['state'] = 'success';
132 | $this->buildData['message'] = 'Indexes Synced';
133 | if ($this->buildData['failed']) {
134 | $this->buildData['state'] = 'warning';
135 | $this->buildData['message'] = 'Some Build Errors';
136 | if ($this->buildData['failed'] === $this->buildData['processed']) {
137 | $this->buildData['state'] = 'error';
138 | $this->buildData['message'] = 'All Builds Failed';
139 | }
140 | }
141 |
142 | return true;
143 | }
144 |
145 | private function showStatus(): void
146 | {
147 | if ($this->buildData['didRun']) {
148 | $this->omni->header('Build Data', 'Value');
149 | $this->omni->row('Success', $this->buildData['success'], null, 'text-emerald-500');
150 | $this->omni->row('Skipped', $this->buildData['skipped'], null, 'text-amber-500');
151 | $this->omni->row('Failed', $this->buildData['failed'], null, 'text-rose-500');
152 | $this->omni->row('Total', $this->buildData['total'], null, 'text-emerald-500');
153 | $this->newLine();
154 | $this->omni->status($this->buildData['state'], 'Build Status', $this->buildData['message']);
155 | $this->newLine();
156 | }
157 | }
158 | }
159 |
--------------------------------------------------------------------------------
/src/Commands/LensStatusCommand.php:
--------------------------------------------------------------------------------
1 | initOmni();
25 |
26 | $this->newLine();
27 | render((string) view('elasticlens::cli.components.title', ['title' => 'ElasticLens Status', 'color' => 'teal']));
28 | $this->newLine();
29 | $checks = ConfigCheck::check();
30 | $indexes = IndexCheck::get();
31 |
32 | $this->omni->header('Config', 'Status', 'Value');
33 | foreach ($checks as $check) {
34 | $this->omni->rowAsStatus($check['label'], $check['status'], $check['extra'] ?? null, $check['help'] ?? []);
35 | }
36 | $this->newLine(2);
37 | if (! empty($indexes)) {
38 | foreach ($indexes as $index) {
39 | $this->omni->status($index['indexStatus']['status'], $index['name'], $index['indexStatus']['name'], $index['indexStatus']['help'] ?? []);
40 | foreach ($index['checks'] as $check) {
41 | $this->omni->rowAsStatus($check['label'], $check['status'], $check['extra'] ?? null, $check['help'] ?? []);
42 | }
43 | $this->newLine(2);
44 | }
45 | } else {
46 | $this->omni->statusError('ERROR', 'No indexes found');
47 | $this->newLine(2);
48 | }
49 |
50 | return self::SUCCESS;
51 | }
52 | }
53 |
--------------------------------------------------------------------------------
/src/Commands/Scripts/ConfigCheck.php:
--------------------------------------------------------------------------------
1 | [
20 | 'label' => 'Config File',
21 | 'extra' => 'ElasticLens.php',
22 | 'status' => 'error',
23 | 'help' => [
24 | 'Unexpected error: Config (ElasticLens.php) could not be loaded from anywhere',
25 | ],
26 | ],
27 | ];
28 | }
29 | $output = [];
30 | // ----------------------------------------------------------------------
31 | // Config File
32 | // ----------------------------------------------------------------------
33 | self::checkConfig($output);
34 | $connectionOK = self::checkConnection($output);
35 | self::checkQueue($output);
36 | if ($connectionOK) {
37 | self::buildIndexableBuildState($output);
38 | }
39 |
40 | return $output;
41 | }
42 |
43 | private static function checkConfig(&$output): void
44 | {
45 | $output['config_file'] = [
46 | 'label' => 'Config File',
47 | 'extra' => 'elasticlens.php',
48 | 'status' => 'ok',
49 | ];
50 | if (! file_exists(config_path('elasticlens.php'))) {
51 | $output['config_file']['status'] = 'warning';
52 | $output['config_file']['help'] = [
53 | 'Config file not published, using package default. Run: `php artisan lens:install` to publish',
54 | ];
55 | }
56 | }
57 |
58 | private static function checkConnection(&$output): bool
59 | {
60 | $connection = config('elasticlens.database');
61 | $output['connection'] = [
62 | 'label' => 'Connection',
63 | 'extra' => $connection,
64 | 'status' => 'ok',
65 | ];
66 | if (! $connection) {
67 | $output['connection']['status'] = 'error';
68 | $output['connection']['help'] = [
69 | "Connection empty in elasticlens.php config. Set 'database' => 'elasticsearch' in config file",
70 | ];
71 |
72 | return false;
73 | }
74 | try {
75 | Schema::on($connection)->getIndices();
76 | } catch (Exception $e) {
77 | $output['connection']['status'] = 'error';
78 | $output['connection']['help'] = [
79 | 'Connection error on ['.$connection.']: '.$e->getMessage(),
80 | ];
81 |
82 | // If we get here, return output
83 | return false;
84 |
85 | }
86 |
87 | return true;
88 | }
89 |
90 | private static function checkQueue(&$output): void
91 | {
92 | $output['queue'] = [
93 | 'label' => 'Queue Priority',
94 | 'extra' => config('elasticlens.queue') ?? 'default (not set)',
95 | 'status' => 'ok',
96 | ];
97 | }
98 |
99 | private static function buildIndexableBuildState(&$output): void
100 | {
101 |
102 | $enabled = config('elasticlens.index_build_state.enabled') ?? false;
103 | $trim = config('elasticlens.index_build_state.log_trim') ?? false;
104 | $output['indexable_state'] = [
105 | 'label' => 'IndexableBuildState Model',
106 | 'extra' => '',
107 | 'status' => 'ok',
108 | ];
109 | if (! $enabled) {
110 | $output['indexable_state']['status'] = 'disabled';
111 | $output['indexable_state']['help'] = [
112 | 'Indexable State Tracking is disabled. Build logs will not be tracked in the indexable_states index',
113 | ];
114 |
115 | return;
116 | }
117 | $output['indexable_state_connect'] = [
118 | 'label' => 'IndexableBuildState Connection',
119 | 'extra' => '',
120 | 'status' => 'ok',
121 | ];
122 | $hasIndex = IndexableBuild::checkHasIndex();
123 | if (! $hasIndex) {
124 | $output['indexable_state_connect']['status'] = 'error';
125 | $output['indexable_state_connect']['help'] = [
126 | 'Indexable State Tracking index not found. Run: php artisan lens:install',
127 | ];
128 | }
129 | $output['indexable_state_log_trim'] = [
130 | 'label' => 'Indexable States Log trim',
131 | 'extra' => $trim,
132 | 'status' => 'ok',
133 | ];
134 | if (! $trim) {
135 | $output['indexable_state_log_trim']['status'] = 'disabled';
136 | $output['indexable_state_log_trim']['help'] = [
137 | 'Logs will not be stored in the indexable_states index',
138 | ];
139 | }
140 |
141 | }
142 | }
143 |
--------------------------------------------------------------------------------
/src/Commands/Scripts/HealthCheck.php:
--------------------------------------------------------------------------------
1 | 'ERROR',
25 | 'status' => 'error',
26 | 'title' => 'Base Model not found',
27 | 'help' => $help,
28 | ];
29 | }
30 |
31 | $indexModel = Lens::fetchIndexModelClass($modelClass);
32 | if (! class_exists($indexModel)) {
33 | return [
34 | 'name' => 'ERROR',
35 | 'status' => 'error',
36 | 'title' => 'Index Model not found',
37 | 'help' => [
38 | $indexModel.' could not be found for base model: '.$model,
39 | ],
40 | ];
41 | }
42 | try {
43 | $index = new LensState($indexModel);
44 | $index->healthCheck();
45 | } catch (Exception $e) {
46 | return [
47 | 'name' => 'ERROR',
48 | 'status' => 'error',
49 | 'title' => 'Load Error',
50 | 'help' => [
51 | $e->getMessage(),
52 | ],
53 | ];
54 | }
55 |
56 | return false;
57 | }
58 |
59 | /**
60 | * @throws Exception
61 | */
62 | public static function check($model): array
63 | {
64 | $modelCheck = QualifyModel::check($model);
65 | $model = $modelCheck['qualified'];
66 | if (! $model) {
67 | return [
68 | 'name' => 'ERROR',
69 | 'status' => 'error',
70 | 'title' => 'Base Model not found',
71 | 'help' => $modelCheck['notFound'],
72 | ];
73 | }
74 | $indexModel = Lens::fetchIndexModelClass($model);
75 | $index = new LensState($indexModel);
76 | $index = $index->healthCheck();
77 |
78 | $health['title'] = $model.' → '.$index['indexModel'];
79 | $health['name'] = $index['name'];
80 | $health['indexStatus'] = $index['state']['status'];
81 | $health['indexStatus']['title'] = 'Index Status';
82 | $health['indexData'] = $index['state']['index'];
83 | $health['modelData'] = $index['state']['model'];
84 | $health['buildData'] = $index['state']['builds'];
85 | $health['configStatus'] = [
86 | 'name' => $index['config']['status']['name'],
87 | 'status' => $index['config']['status']['status'],
88 | 'title' => 'Config Status',
89 | ];
90 | self::setConfig($health, $index);
91 | self::setObservers($health, $index);
92 | $health['configStatusHelp'] = [
93 | 'critical' => $index['config']['status']['critical'],
94 | 'warning' => $index['config']['status']['warning'],
95 | ];
96 |
97 | return $health;
98 | }
99 |
100 | private static function setObservers(&$health, $index): void
101 | {
102 | $base = $index['config']['observers']['base'];
103 | $embedded = $index['config']['observers']['embedded'];
104 | $health['observers'] = [];
105 | if ($base) {
106 | $health['observers'][] = [
107 | 'key' => class_basename($base),
108 | 'value' => '(Base)',
109 | ];
110 | }
111 | if (! empty($embedded)) {
112 | foreach ($embedded as $embed) {
113 | if ($embed['observe']) {
114 | $chain = [];
115 | self::buildTriggerChain($embed, $chain);
116 |
117 | $chain = implode(' → ', $chain);
118 |
119 | $health['observers'][] = [
120 | 'key' => class_basename($embed['relation']),
121 | 'value' => 'Triggers → '.$chain,
122 | ];
123 | }
124 | }
125 | }
126 | }
127 |
128 | public static function buildTriggerChain($embed, &$chain): void
129 | {
130 | $chain[] = class_basename($embed['model']);
131 | if (! empty($embed['upstream'])) {
132 | self::buildTriggerChain($embed['upstream'], $chain);
133 | }
134 | }
135 |
136 | private static function setConfig(&$health, $index): void
137 | {
138 |
139 | $health['configData'] = [
140 | 'baseModelIndexable' => $index['config']['base_model_indexable'],
141 | 'baseModelSet' => $index['config']['base_model'],
142 | 'fieldMapSet' => $index['config']['field_map'],
143 | 'migrationMapSet' => $index['config']['migration']['has'],
144 | ];
145 | if ($index['config']['migration']['version']) {
146 | $health['configData']['migrationVersion'] = $index['config']['migration']['version'];
147 | }
148 | }
149 | }
150 |
--------------------------------------------------------------------------------
/src/Commands/Scripts/IndexCheck.php:
--------------------------------------------------------------------------------
1 | healthCheck();
22 | } catch (Exception $e) {
23 | // skip, I guess
24 | }
25 | }
26 | }
27 | $states = [];
28 | foreach ($indexes as $index) {
29 | $state['name'] = $index['name'];
30 | $state['indexStatus'] = $index['state']['status'];
31 | $state['checks'] = [];
32 | self::indexState($state['checks'], $index);
33 | self::modelState($state['checks'], $index);
34 | self::buildState($state['checks'], $index);
35 | self::configState($state['checks'], $index);
36 |
37 | $states[] = $state;
38 | }
39 |
40 | return $states;
41 | }
42 |
43 | public static function indexState(&$checks, $index): void
44 | {
45 | $modelName = $index['state']['index']['modelName'];
46 | $table = $index['state']['index']['table'];
47 | $accessible = $index['state']['index']['accessible'];
48 | $records = $index['state']['index']['records'];
49 | $checks['indexModel'] = [
50 | 'label' => $modelName,
51 | 'extra' => 'accessible',
52 | 'status' => 'ok',
53 | ];
54 | if (! $accessible) {
55 | $checks['indexModel']['status'] = 'error';
56 | $checks['indexModel']['help'] = [
57 | 'Index model not accessible',
58 | ];
59 | }
60 | $checks['indexModelRecords'] = [
61 | 'label' => $table.' records',
62 | 'extra' => $records,
63 | 'status' => 'ok',
64 | ];
65 | if (! $records) {
66 | $checks['indexModelRecords']['status'] = 'warning';
67 | }
68 | }
69 |
70 | public static function modelState(&$checks, $index): void
71 | {
72 | $defined = $index['state']['model']['defined'];
73 | $modelName = $index['state']['model']['modelName'];
74 | $table = $index['state']['model']['table'];
75 | $accessible = $index['state']['model']['accessible'];
76 | $records = $index['state']['model']['records'];
77 | if (! $defined) {
78 | $checks['model'] = [
79 | 'label' => $modelName,
80 | 'extra' => 'defined',
81 | 'status' => 'error',
82 | 'help' => [
83 | 'Model not defined',
84 | ],
85 | ];
86 |
87 | return;
88 | }
89 |
90 | $checks['model'] = [
91 | 'label' => $modelName,
92 | 'extra' => 'accessible',
93 | 'status' => 'ok',
94 | ];
95 | if (! $accessible) {
96 | $checks['model']['status'] = 'error';
97 | $checks['model']['help'] = [
98 | 'Index model not accessible',
99 | ];
100 | }
101 | $checks['modelRecords'] = [
102 | 'label' => $table.' records',
103 | 'extra' => $records,
104 | 'status' => 'ok',
105 | ];
106 | if (! $records) {
107 | $checks['modelRecords']['status'] = 'warning';
108 | }
109 | }
110 |
111 | public static function buildState(&$checks, $index): void
112 | {
113 | $builds = $index['state']['builds'];
114 | $total = $builds['total'];
115 | $errors = $builds['errors'];
116 | $success = $builds['success'];
117 | if (! $total) {
118 | $checks['builds'] = [
119 | 'label' => 'Builds (IndexedState)',
120 | 'extra' => 'No builds found',
121 | 'status' => 'warning',
122 | ];
123 |
124 | return;
125 | }
126 | if ($errors) {
127 | $checks['builds'] = [
128 | 'label' => 'Builds (IndexedState)',
129 | 'extra' => $total.' ('.$errors.')',
130 | 'status' => 'error',
131 | 'help' => [
132 | $errors.' index(es) could not be built, check logs',
133 | ],
134 | ];
135 |
136 | return;
137 | }
138 | $checks['builds'] = [
139 | 'label' => 'Builds (IndexedState)',
140 | 'extra' => $total,
141 | 'status' => 'ok',
142 | ];
143 |
144 | }
145 |
146 | public static function configState(&$checks, $index): void
147 | {
148 | $config = $index['config'];
149 | $status = $config['status'];
150 | $checks['config'] = [
151 | 'label' => 'Config State',
152 | 'extra' => $status['name'],
153 | 'status' => $status['status'],
154 | ];
155 | if ($status['status'] !== 'ok') {
156 | $checks['config']['help'] = [
157 | 'For details, run: php artisan lens:health '.$index['state']['model']['modelName'],
158 | ];
159 | }
160 | }
161 | }
162 |
--------------------------------------------------------------------------------
/src/Commands/Scripts/QualifyModel.php:
--------------------------------------------------------------------------------
1 | $indexNameSpace) {
13 | $modelPath = $modelNameSpace.'\\'.$model;
14 | if (class_exists($modelPath)) {
15 | $found = $modelPath;
16 | } else {
17 | $notFound[] = $modelPath;
18 | }
19 | }
20 |
21 | return [
22 | 'qualified' => $found,
23 | 'notFound' => $notFound,
24 | ];
25 | }
26 | }
27 |
--------------------------------------------------------------------------------
/src/ElasticLensServiceProvider.php:
--------------------------------------------------------------------------------
1 | name('elasticlens')
27 | ->hasConfigFile()
28 | ->hasViews('elasticlens')
29 | ->hasMigrations(['create_indexable_build_index', 'create_indexable_migration_logs_index'])
30 | ->runsMigrations()
31 | ->hasCommand(LensHealthCommand::class)
32 | ->hasCommand(LensStatusCommand::class)
33 | ->hasCommand(LensBuildCommand::class)
34 | ->hasCommand(LensMigrateCommand::class)
35 | ->hasCommand(LensMakeCommand::class)
36 | ->hasInstallCommand(function (InstallCommand $command) {
37 | $command
38 | ->setName('lens:install')
39 | ->publishConfigFile()
40 | ->publishMigrations()
41 | ->askToRunMigrations()
42 | ->copyAndRegisterServiceProviderInApp()
43 | ->askToStarRepoOnGitHub('pdphilip/elasticlens');
44 | });
45 | }
46 | }
47 |
--------------------------------------------------------------------------------
/src/Enums/IndexableBuildState.php:
--------------------------------------------------------------------------------
1 | 'slate',
18 | IndexableBuildState::SUCCESS => 'emerald',
19 | IndexableBuildState::SKIPPED => 'emerald',
20 | IndexableBuildState::FAILED => 'rose',
21 | };
22 | }
23 |
24 | public function label(): string
25 | {
26 | return match ($this) {
27 | IndexableBuildState::INIT => 'Build Initializing',
28 | IndexableBuildState::SUCCESS => 'Index Build Successful',
29 | IndexableBuildState::SKIPPED => 'Index Build Skipped',
30 | IndexableBuildState::FAILED => 'Index Build Failed',
31 | };
32 | }
33 | }
34 |
--------------------------------------------------------------------------------
/src/Enums/IndexableMigrationLogState.php:
--------------------------------------------------------------------------------
1 | 'slate',
17 | IndexableMigrationLogState::SUCCESS => 'emerald',
18 | IndexableMigrationLogState::FAILED => 'rose',
19 | };
20 | }
21 |
22 | public function label(): string
23 | {
24 | return match ($this) {
25 | IndexableMigrationLogState::UNDEFINED => 'No Blueprint defined',
26 | IndexableMigrationLogState::SUCCESS => 'Migration Successful',
27 | IndexableMigrationLogState::FAILED => 'Migration Failed',
28 | };
29 | }
30 | }
31 |
--------------------------------------------------------------------------------
/src/HasWatcher.php:
--------------------------------------------------------------------------------
1 | id = $id;
34 | $this->model = $model;
35 | $this->migration_version = $migrationVersion;
36 | $this->startTimer();
37 | }
38 |
39 | public function setMessage($msg, $details = ''): void
40 | {
41 | $this->msg = $msg;
42 | $this->details = $details;
43 | }
44 |
45 | public function setMap($map): void
46 | {
47 | $this->map = $map;
48 | }
49 |
50 | public function failed(): static
51 | {
52 | $this->success = false;
53 | $this->took = $this->getTime();
54 |
55 | return $this;
56 | }
57 |
58 | public function successful($details = ''): static
59 | {
60 | $this->details = $details;
61 | $this->success = true;
62 | $this->took = $this->getTime();
63 |
64 | return $this;
65 | }
66 |
67 | public function attachMigrationVersion($version): static
68 | {
69 | $this->migration_version = $version;
70 |
71 | return $this;
72 | }
73 |
74 | public function toArray(): array
75 | {
76 | return [
77 | 'id' => $this->id,
78 | 'model' => $this->model,
79 | 'success' => $this->success,
80 | 'msg' => $this->msg,
81 | 'details' => $this->details,
82 | 'map' => $this->map,
83 | 'migration_version' => $this->migration_version,
84 | 'took' => $this->took,
85 | ];
86 | }
87 | }
88 |
--------------------------------------------------------------------------------
/src/Index/BulkIndexer.php:
--------------------------------------------------------------------------------
1 | baseModel = $baseModel;
38 | $this->indexModel = Lens::fetchIndexModelClass($baseModel);
39 | $this->builder = new LensBuilder($this->indexModel);
40 | if (! $this->builder->baseModel) {
41 | throw new Exception('BulkIndexing not available for "'.$this->indexModel.'": BaseModel not set - Set property: `protected $baseModel = User::class;`');
42 | }
43 |
44 | if (! $this->builder->baseModelIndexable) {
45 | throw new Exception('BulkIndexing not available for "'.$this->indexModel.'": Base model not indexable - Add trait to base model: `use Indexable`');
46 | }
47 |
48 | $this->migrationVersion = $this->builder->fetchCurrentMigrationVersion();
49 | }
50 |
51 | public function setRecords($records): static
52 | {
53 | $this->clearActivity();
54 | $builds = [];
55 | if ($records) {
56 | foreach ($records as $record) {
57 | $id = $record->{$this->builder->baseModelPrimaryKey};
58 | $setup = $this->builder->prepareMap($id);
59 | $setup->attachMigrationVersion($this->migrationVersion);
60 | // Assume ok
61 | if (! $setup->skipped) {
62 | $setup->successful('Bulk Ok');
63 | }
64 |
65 | $builds[$id] = $setup;
66 | }
67 | }
68 |
69 | $this->buildMaps = $builds;
70 |
71 | return $this;
72 | }
73 |
74 | public function build(): static
75 | {
76 | $values = [];
77 | if ($this->buildMaps) {
78 | foreach ($this->buildMaps as $build) {
79 | if ($build->skipped) {
80 | $this->skipped++;
81 |
82 | continue;
83 | }
84 | $values[] = $build->map;
85 | }
86 | }
87 | if ($values) {
88 | $this->result = $this->indexModel::bulkInsert($values);
89 | }
90 | $this->updateAnyErrors();
91 | $this->result['skipped'] = $this->skipped;
92 | BulkBuildStateUpdateJob::dispatch($this->indexModel, $this->baseModel, $this->buildMaps);
93 | $this->took = $this->getTime();
94 |
95 | return $this;
96 | }
97 |
98 | public function updateAnyErrors(): void
99 | {
100 | if ($this->result) {
101 | if (! empty($this->result['error'])) {
102 | $builds = $this->buildMaps;
103 | foreach ($this->result['error'] as $error) {
104 | $id = $error['payload']['id'] ?? null;
105 | $build = $builds[$id];
106 | $build->setMessage($error['error']['type'], $error['error']['reason']);
107 | $build->failed();
108 | $builds[$id] = $build;
109 | }
110 | $this->buildMaps = $builds;
111 | }
112 | }
113 | }
114 |
115 | public function getResult(): array
116 | {
117 | return [
118 | 'took' => $this->took,
119 | 'results' => $this->result,
120 | ];
121 | }
122 |
123 | public function clearActivity(): void
124 | {
125 | $this->buildMaps = [];
126 | $this->startTimer();
127 |
128 | }
129 | }
130 |
--------------------------------------------------------------------------------
/src/Index/LensBuilder.php:
--------------------------------------------------------------------------------
1 | _buildInit($id);
23 | $passed = true;
24 | try {
25 | $callback();
26 | } catch (Exception $e) {
27 | $this->buildResult->setMessage('Custom Build Failed', 'Exception: '.$e->getMessage());
28 | $passed = false;
29 | }
30 | if ($passed) {
31 | $this->buildResult->successful('Custom Build Passed');
32 | }
33 | IndexableBuild::writeState($this->baseModelName, $id, $this->indexModelName, $this->buildResult, $source);
34 |
35 | return $this->buildResult;
36 | }
37 |
38 | public function prepareMap($id)
39 | {
40 | $this->_buildInit($id);
41 | $this->_buildMap();
42 |
43 | return $this->buildResult;
44 | }
45 |
46 | public function buildIndex($id, $source, $migrationVersion = null): BuildResult
47 | {
48 | if (! $migrationVersion) {
49 | $migrationVersion = $this->fetchCurrentMigrationVersion();
50 | }
51 | $this->buildProcess($id);
52 | $this->buildResult->attachMigrationVersion($migrationVersion);
53 | IndexableBuild::writeState($this->baseModelName, $id, $this->indexModelName, $this->buildResult, $source);
54 |
55 | return $this->buildResult;
56 | }
57 |
58 | public function dryRun($id): BuildResult
59 | {
60 | $this->_buildInit($id);
61 | $passedSetup = $this->checkSetup();
62 | if (! $passedSetup) {
63 | return $this->buildResult->failed();
64 | }
65 | $passedMapping = $this->_buildMap();
66 | if (! $passedMapping) {
67 | return $this->buildResult->failed();
68 | }
69 |
70 | return $this->buildResult->successful();
71 |
72 | }
73 |
74 | protected function buildProcess($id): BuildResult
75 | {
76 | $this->_buildInit($id);
77 | $passedSetup = $this->checkSetup();
78 | if (! $passedSetup) {
79 | return $this->buildResult->failed();
80 | }
81 | $passedMapping = $this->_buildMap();
82 | if (! $passedMapping) {
83 | return $this->buildResult->failed();
84 | }
85 |
86 | $createdIndex = $this->_createIndex();
87 |
88 | if (! $createdIndex) {
89 | return $this->buildResult->failed();
90 | }
91 |
92 | return $this->buildResult->successful('Indexed successfully');
93 | }
94 |
95 | // ----------------------------------------------------------------------
96 | // Init
97 | // ----------------------------------------------------------------------
98 |
99 | private function _buildInit($id): void
100 | {
101 | $this->buildResult = new BuildResult($id, $this->baseModel, 0);
102 | }
103 |
104 | // ----------------------------------------------------------------------
105 | // Setup Check
106 | // ----------------------------------------------------------------------
107 |
108 | public function checkSetup(): bool
109 | {
110 |
111 | if (! $this->baseModel) {
112 | $this->buildResult->setMessage('BaseModel not set', 'Set property: `protected $baseModel = User::class;`');
113 |
114 | return false;
115 | }
116 |
117 | if (! $this->baseModelIndexable) {
118 | $this->buildResult->setMessage('Base model not indexable', 'Add trait to base model: `use Indexable`');
119 |
120 | return false;
121 | }
122 |
123 | return true;
124 | }
125 |
126 | // ----------------------------------------------------------------------
127 | // Mapping Process
128 | // ----------------------------------------------------------------------
129 |
130 | private function _buildMap(): bool
131 | {
132 | $id = $this->buildResult->id;
133 | $fieldMap = $this->fieldMap;
134 |
135 | $model = $this->baseModelInstance->find($id);
136 | if (! $model) {
137 | $this->buildResult->setMessage('BaseModel not found', 'BaseModel '.$this->baseModel.' did not have a record for id: '.$id);
138 |
139 | return false;
140 | }
141 |
142 | if ($model->excludeIndex()) {
143 | $this->buildResult->skipped = true;
144 | $this->buildResult->setMessage('BaseModel excluded', 'BaseModel '.$this->baseModel.' has excludeIndex() set to true');
145 |
146 | return false;
147 | }
148 |
149 | $data = $this->mapId($model, $fieldMap);
150 | try {
151 | $dataMap = $this->mapRecordsToFields($fieldMap, $model);
152 | } catch (Exception $e) {
153 | $this->buildResult->setMessage('Record Mapping Error', 'Exception: '.$e->getMessage());
154 |
155 | return false;
156 | }
157 | $data = $data + $dataMap;
158 |
159 | $this->buildResult->setMap($data);
160 |
161 | return true;
162 | }
163 |
164 | private function mapId($model, $fieldMap): array
165 | {
166 | $data = [];
167 | $data['id'] = $model->{$model->getKeyName()};
168 | if (isset($fieldMap['id'])) {
169 | $data['id'] = $this->setType($data['id'], $fieldMap['id']);
170 | unset($fieldMap['id']);
171 | }
172 | if (isset($fieldMap['id'])) {
173 | $data['id'] = $this->setType($data['id'], $fieldMap['id']);
174 | unset($fieldMap['id']);
175 | }
176 |
177 | return $data;
178 | }
179 |
180 | private function mapRecordsToFields($fields, $modelData): array
181 | {
182 |
183 | $data = [];
184 | if ($fields) {
185 | foreach ($fields as $field => $type) {
186 | if (is_array($type)) {
187 | $embedFields = $type;
188 | $data[$field] = $this->buildEmbeddedRelationship($field, $embedFields, $modelData);
189 |
190 | continue;
191 | }
192 |
193 | $value = $modelData->{$field} ?? null;
194 | if ($value) {
195 | $value = $this->setType($value, $type);
196 | }
197 |
198 | $data[$field] = $value;
199 |
200 | }
201 |
202 | return $data;
203 | }
204 |
205 | // Else, take what you can get.
206 | // ....I Also Like to Live Dangerously
207 | $data = $modelData->toArray();
208 |
209 | // If this was the base model, kick the ID, we has it already
210 | if ($modelData instanceof $this->baseModel) {
211 | unset($data[$this->baseModelPrimaryKey]);
212 | }
213 |
214 | return $data;
215 | }
216 |
217 | private function buildEmbeddedRelationship($field, $embedFields, $parentData): array
218 | {
219 | $relationships = $this->relationships;
220 | $data = [];
221 | if (! empty($relationships[$field])) {
222 | $relationship = $relationships[$field];
223 | $type = $relationship['type'];
224 | $relation = $relationship['relation'];
225 | $whereRelatedField = $relationship['whereRelatedField'];
226 | $equalsModelField = $relationship['equalsModelField'];
227 | $modelFieldValue = $parentData->{$equalsModelField};
228 | $query = $relationship['query'];
229 |
230 | $records = $relation::where($whereRelatedField, $modelFieldValue);
231 | if ($query) {
232 | $records = $records->tap($query);
233 | }
234 | if ($type == 'hasMany') {
235 | $records = $records->get();
236 | if ($records->isNotEmpty()) {
237 | foreach ($records as $record) {
238 | $data[] = $this->mapRecordsToFields($embedFields, $record);
239 | }
240 | }
241 |
242 | return $data;
243 | }
244 | $record = $records->first();
245 | if ($record) {
246 | $data = $this->mapRecordsToFields($embedFields, $record);
247 | }
248 |
249 | return $data;
250 |
251 | }
252 |
253 | return $data;
254 | }
255 |
256 | // ----------------------------------------------------------------------
257 | // Create Index
258 | // ----------------------------------------------------------------------
259 |
260 | public function _createIndex(): bool
261 | {
262 | try {
263 | $modelId = $this->buildResult->id;
264 | $index = $this->indexModelInstance::find($modelId);
265 | if (! $index) {
266 | $index = new $this->indexModelInstance;
267 | }
268 | $index->id = $modelId;
269 |
270 | foreach ($this->buildResult->map as $field => $value) {
271 | $index->{$field} = $value;
272 | }
273 | $index->withoutRefresh()->save();
274 | } catch (Exception $e) {
275 | $this->buildResult->setMessage('Index build Error', $e->getMessage());
276 |
277 | return false;
278 | }
279 |
280 | return true;
281 | }
282 |
283 | // ----------------------------------------------------------------------
284 | // Delete Index (And Build)
285 | // ----------------------------------------------------------------------
286 |
287 | public function processDelete($id)
288 | {
289 | IndexableBuild::deleteState($this->baseModelName, $id, $this->indexModelName);
290 | $index = $this->indexModelInstance::find($id);
291 | $index->delete();
292 | }
293 |
294 | // ----------------------------------------------------------------------
295 | // Dispatchers
296 | // ----------------------------------------------------------------------
297 |
298 | public function dispatchBuild($modelId, $observedModel)
299 | {
300 | IndexBuildJob::dispatch($this->indexModel, $modelId, $observedModel);
301 | }
302 |
303 | public function dispatchDeleted($modelId)
304 | {
305 | IndexDeletedJob::dispatch($this->indexModel, $modelId);
306 | }
307 |
308 | // ----------------------------------------------------------------------
309 | // Helpers
310 | // ----------------------------------------------------------------------
311 |
312 | private function setType($value, $type)
313 | {
314 | if ($type == Carbon::class) {
315 | return Carbon::create($value);
316 | }
317 | if (enum_exists($type)) {
318 | $value = $value->value ?? $value;
319 | $type = 'string';
320 | }
321 | settype($value, $type);
322 |
323 | return $value;
324 | }
325 | }
326 |
--------------------------------------------------------------------------------
/src/Index/LensIndex.php:
--------------------------------------------------------------------------------
1 | indexModel = $indexModel;
59 | $this->indexModelInstance = $instance;
60 | $migrationSettings = $instance->getMigrationSettings();
61 | $this->indexModelName = class_basename($indexModel);
62 | $this->indexModelTable = $instance->getTable();
63 | $this->indexExists = $instance::indexExists();
64 | $this->fieldMap = $instance->getFieldSet();
65 | $this->observers = $instance->getObserverSet();
66 | $this->relationships = $instance->getRelationships();
67 | $this->indexMigration = $migrationSettings;
68 | $this->baseModelDefined = $instance->isBaseModelDefined();
69 | $baseModel = $instance->getBaseModel();
70 | if ($baseModel) {
71 | $this->baseModel = $baseModel;
72 | $this->baseModelInstance = (new $baseModel);
73 | $this->baseModelName = class_basename($baseModel);
74 | $this->baseModelTable = $this->baseModelInstance->getTable();
75 | $this->baseModelPrimaryKey = $this->baseModelInstance->getKeyName();
76 | try {
77 | $baseModel::indexModel();
78 | $this->baseModelIndexable = true;
79 | } catch (Exception) {
80 | $this->baseModelIndexable = false;
81 | }
82 | }
83 |
84 | }
85 |
86 | public function fetchCurrentMigrationVersion(): string
87 | {
88 | $this->indexMigrationVersion = $this->indexModelInstance->getCurrentMigrationVersion();
89 |
90 | return $this->indexMigrationVersion;
91 | }
92 | }
93 |
--------------------------------------------------------------------------------
/src/Index/LensMigration.php:
--------------------------------------------------------------------------------
1 | false,
21 | 'message' => '',
22 | 'state' => [
23 | 'index_model' => '',
24 | 'index_count' => 0,
25 | 'base_model' => false,
26 | 'base_count' => 0,
27 | 'has_blueprint' => false,
28 | 'migration_version' => false,
29 | ],
30 | ];
31 |
32 | public function migrationState(): array
33 | {
34 | $this->_state['state']['index_model'] = $this->indexModelName;
35 | try {
36 |
37 | $indexCount = 0;
38 | try {
39 | $indexCount = $this->indexModelInstance::count();
40 | } catch (Exception $e) {
41 |
42 | }
43 | $this->_state['state']['index_count'] = $indexCount;
44 |
45 | if ($this->baseModel) {
46 | $this->_state['state']['base_model'] = $this->baseModelName;
47 | $this->_state['state']['base_count'] = $this->baseModelInstance::count();
48 | }
49 |
50 | $this->_state['state']['has_blueprint'] = $this->indexModelInstance->migrationMap() !== null;
51 | $this->_state['state']['migration_version'] = $this->fetchCurrentMigrationVersion();
52 |
53 | } catch (Exception $e) {
54 | $this->_state['error'] = true;
55 | $this->_state['message'] = $e->getMessage();
56 | }
57 |
58 | return $this->_state;
59 | }
60 |
61 | public function runMigration(): bool
62 | {
63 | $validBlueprint = false;
64 | $tableName = $this->indexModelTable;
65 | $blueprint = $this->indexMigration['blueprint'] ?? null;
66 | $version = $this->indexMigration['majorVersion'] ?? null;
67 | if ($blueprint) {
68 | $validBlueprint = $this->validateMigration()['validated'];
69 | }
70 | try {
71 | IndexableBuild::deleteStateModel($this->indexModelName);
72 |
73 | Schema::deleteIfExists($tableName);
74 | if (! $validBlueprint) {
75 | Schema::create($tableName, function (Blueprint $index) {
76 | $index->date('created_at');
77 | });
78 | $map = Schema::getMappings($tableName, true);
79 | IndexableMigrationLog::saveMigrationLog($this->indexModelName, $version, IndexableMigrationLogState::UNDEFINED, $map);
80 | } else {
81 | Schema::create($tableName, $blueprint);
82 | $map = Schema::getMappings($tableName, true);
83 | IndexableMigrationLog::saveMigrationLog($this->indexModelName, $version, IndexableMigrationLogState::SUCCESS, $map);
84 | }
85 |
86 | return true;
87 | } catch (Exception $e) {
88 | $map = ['error' => $e->getMessage()];
89 | IndexableMigrationLog::saveMigrationLog($this->indexModelName, $version, IndexableMigrationLogState::FAILED, $map);
90 | }
91 |
92 | return false;
93 | }
94 |
95 | public function validateMigration(): array
96 | {
97 |
98 | $test = new MigrationValidator(
99 | $this->fetchCurrentMigrationVersion(),
100 | $this->indexModelInstance->migrationMap(),
101 | $this->indexModelTable
102 | );
103 |
104 | return $test->testMigration();
105 | }
106 | }
107 |
--------------------------------------------------------------------------------
/src/Index/LensState.php:
--------------------------------------------------------------------------------
1 | indexModelTable;
16 | $name = Str::title(str_replace('_', ' ', $indexTable));
17 |
18 | return [
19 | 'name' => $name,
20 | 'indexModel' => $this->indexModelName,
21 | 'state' => $this->_indexStatusCheck(),
22 | 'config' => $this->_configCheck(),
23 | ];
24 |
25 | }
26 |
27 | public function getObservedModelNames()
28 | {
29 | return $this->observers;
30 | }
31 |
32 | private function _indexStatusCheck(): array
33 | {
34 |
35 | $indexData['modelName'] = $this->indexModelName;
36 | $indexData['table'] = $this->indexModelTable;
37 | $indexData['accessible'] = false;
38 | $indexData['records'] = 0;
39 | try {
40 |
41 | $indexData['records'] = $this->indexModelInstance::count();
42 | $indexData['accessible'] = true;
43 | } catch (Exception $e) {
44 |
45 | }
46 | $modelData['modelName'] = $this->baseModelName;
47 | $modelData['defined'] = $this->baseModelDefined;
48 | $modelData['table'] = $this->baseModelTable;
49 | $modelData['accessible'] = false;
50 | $modelData['records'] = 0;
51 | if ($this->baseModel) {
52 | try {
53 | $modelData['records'] = $this->baseModel::count();
54 | $modelData['accessible'] = true;
55 | } catch (Exception $e) {
56 |
57 | }
58 | }
59 | $builds = [];
60 | $builds['success'] = 0;
61 | $builds['errors'] = 0;
62 | $builds['skipped'] = 0;
63 | $builds['total'] = IndexableBuild::countModelRecords($this->indexModelName);
64 | if ($builds['total'] > 0) {
65 | $builds['errors'] = IndexableBuild::countModelErrors($this->indexModelName);
66 | $builds['skipped'] = IndexableBuild::countModelSkips($this->indexModelName);
67 | $builds['success'] = $builds['total'] - $builds['errors'];
68 | }
69 | $status = $this->_buildStatus($indexData, $modelData, $builds);
70 |
71 | return [
72 | 'index' => $indexData,
73 | 'model' => $modelData,
74 | 'builds' => $builds,
75 | 'status' => $status,
76 | ];
77 | // Status calc
78 | }
79 |
80 | private function _configCheck(): array
81 | {
82 | $config = [
83 | 'base_model_indexable' => $this->baseModelIndexable,
84 | 'base_model' => $this->baseModelDefined,
85 | 'field_map' => ! empty($this->fieldMap),
86 | 'migration' => [
87 | 'has' => $this->indexMigration['blueprint'] !== null,
88 | 'version' => $this->fetchCurrentMigrationVersion(),
89 | ],
90 | 'observers' => $this->getObservedModelNames(),
91 | 'status' => [],
92 | ];
93 | $config['status'] = $this->_buildConfigStatus($config);
94 |
95 | return $config;
96 | }
97 |
98 | private function _buildStatus($indexData, $modelData, $builds): array
99 | {
100 | if (! $indexData['accessible']) {
101 | return [
102 | 'status' => 'error',
103 | 'name' => 'Index Not Accessible',
104 | 'help' => ['Check ES connection & index migration for ('.$indexData['table'].')'],
105 | ];
106 | }
107 | if (! $modelData['accessible']) {
108 | return [
109 | 'status' => 'error',
110 | 'name' => 'Model Not Accessible',
111 | 'help' => ['Base model ('.$modelData['modelName'].') could not be reached. Does it exist?'],
112 | ];
113 | }
114 | if ($builds['errors']) {
115 | return [
116 | 'status' => 'warning',
117 | 'name' => 'Index Build Errors',
118 | 'help' => ['Some indexed could not be built, check logs. Total: '.$builds['errors']],
119 | ];
120 | }
121 |
122 | if ($modelData['records'] !== $indexData['records'] + $builds['skipped']) {
123 | return [
124 | 'status' => 'warning',
125 | 'name' => 'Indexes out of sync',
126 | 'help' => ['Index count ('.$indexData['records'].') does not match model count ('.$modelData['records'].')'],
127 | ];
128 | }
129 | if (! $builds['total']) {
130 | return [
131 | 'status' => 'warning',
132 | 'name' => 'No indexes',
133 | 'help' => ['No indexes found for ('.$indexData['modelName'].')'],
134 | ];
135 | }
136 |
137 | return [
138 | 'status' => 'ok',
139 | 'name' => 'Index Synced',
140 | ];
141 | }
142 |
143 | private function _buildConfigStatus($config): array
144 | {
145 | $critical = [];
146 | $warning = [];
147 | if (! $config['base_model_indexable']) {
148 | $critical[] = [
149 | 'status' => 'error',
150 | 'name' => 'Base Model Not Indexable',
151 | 'help' => ['Add trait to base model: `use Indexable`'],
152 | ];
153 | }
154 | if (! $config['base_model']) {
155 |
156 | $baseModel = $this->baseModel;
157 | if (! $baseModel) {
158 | $baseModel = 'MyModel::class';
159 | } else {
160 | $baseModel = class_basename($this->baseModel).'::class';
161 | }
162 |
163 | $warning[] = [
164 | 'status' => 'warning',
165 | 'name' => 'Base Model Not Set',
166 | 'help' => [
167 | 'Base model will be guessed',
168 | 'Set property: `protected $baseModel = '.$baseModel.';`',
169 | ],
170 | ];
171 | }
172 | if (! $config['field_map']) {
173 | $warning[] = [
174 | 'status' => 'warning',
175 | 'name' => 'Field Map Recommended',
176 | 'help' => [
177 | 'Fields will taken as is from the base model when it is indexed',
178 | 'You can define your own in the Index Model with: `public function fieldMap(): IndexBuilder`',
179 | ],
180 | ];
181 | }
182 | if (! $config['observers']['base'] && ! $config['observers']['embedded']) {
183 | $warning[] = [
184 | 'status' => 'warning',
185 | 'name' => 'No Models Observed',
186 | 'help' => ['There are no events that will trigger indexing'],
187 | ];
188 | }
189 | if (! $config['migration']['has']) {
190 | $warning[] = [
191 | 'status' => 'warning',
192 | 'name' => 'Migration Recommended',
193 | 'help' => [
194 | 'The index will automatically infer the field types as data is indexed',
195 | 'You can define your own in the Index Model with: `public function migrationMap(): MigrationBuilder`',
196 | ],
197 | ];
198 | }
199 | $status = [
200 | 'status' => 'ok',
201 | 'name' => 'OK',
202 | 'critical' => $critical,
203 | 'warning' => $warning,
204 | ];
205 | if ($warning) {
206 | $warnings = count($warning);
207 | $status['status'] = 'warning';
208 | $status['name'] = '1 Config Warning';
209 | if ($warnings > 1) {
210 | $status['name'] = $warnings.' Config Warnings';
211 | }
212 |
213 | }
214 | if ($critical) {
215 | $status['status'] = 'error';
216 | $status['name'] = 'Critical Config Error';
217 |
218 | }
219 |
220 | return $status;
221 |
222 | }
223 | }
224 |
--------------------------------------------------------------------------------
/src/Index/MigrationValidator.php:
--------------------------------------------------------------------------------
1 | version = $version;
29 | $this->blueprint = $blueprint;
30 | $this->indexModelTable = $indexModelTable;
31 | }
32 |
33 | public function testMigration(): array
34 | {
35 | if (! $this->blueprint) {
36 | $this->state = 'No Blueprint';
37 | $this->message = '';
38 |
39 | return $this->asArray();
40 | }
41 | if (! is_callable($this->blueprint)) {
42 | $this->state = 'Blueprint Error';
43 | $this->message = 'Blueprint is not callable';
44 |
45 | return $this->asArray();
46 | }
47 |
48 | $tempIndex = 'elasticlens_test_index_for_'.$this->indexModelTable;
49 | Schema::deleteIfExists($tempIndex);
50 | try {
51 | Schema::create($tempIndex, $this->blueprint);
52 | $this->indexMap = Schema::getMappings($tempIndex);
53 | Schema::deleteIfExists($tempIndex);
54 | $this->validated = true;
55 | $this->state = 'Validated';
56 |
57 | return $this->asArray();
58 | } catch (Exception $e) {
59 | $this->message = $e->getMessage();
60 | $this->state = 'Failed';
61 |
62 | return $this->asArray();
63 | }
64 | }
65 |
66 | private function asArray(): array
67 | {
68 | return [
69 | 'version' => $this->version,
70 | 'blueprint' => $this->blueprint,
71 | 'validated' => $this->validated,
72 | 'state' => $this->state,
73 | 'message' => $this->message,
74 | 'indexMap' => $this->indexMap,
75 | ];
76 | }
77 | }
78 |
--------------------------------------------------------------------------------
/src/IndexModel.php:
--------------------------------------------------------------------------------
1 | baseModel) {
48 | $this->baseModel = $this->guessBaseModelName();
49 | }
50 | $this->setConnection(config('elasticlens.database') ?? 'elasticsearch');
51 | Builder::macro('paginateBase', function ($perPage = 15, $pageName = 'page', $page = null, $options = []) {
52 | $page = $page ?: LengthAwarePaginator::resolveCurrentPage($pageName);
53 | $path = LengthAwarePaginator::resolveCurrentPath();
54 | $items = $this->get()->forPage($page, $perPage);
55 | $items = $items->map(function ($value) {
56 | // @phpstan-ignore-next-line
57 | return $value->base;
58 | });
59 | $pagi = new LengthAwarePaginator(
60 | $items,
61 | $this->count(),
62 | $perPage,
63 | $page,
64 | $options
65 | );
66 | $pagi->setPath($path);
67 |
68 | return $pagi;
69 | });
70 | Builder::macro('getBase', function () {
71 | return $this->get()->map(function ($value) {
72 | // @phpstan-ignore-next-line
73 | return $value->base;
74 | });
75 | });
76 | Collection::macro('asBase', function () {
77 | return $this->map(function ($value) {
78 | return $value->base;
79 | });
80 | });
81 | }
82 |
83 | public function getBaseAttribute()
84 | {
85 | return $this->baseModel::find($this->id);
86 | }
87 |
88 | public function asBase()
89 | {
90 | return $this->base;
91 | }
92 |
93 | public static function lensHealth(): array
94 | {
95 | $lens = new LensState(static::class);
96 |
97 | return $lens->healthCheck();
98 | }
99 |
100 | public static function whereIndexBuilds($byLatest = false): Builder
101 | {
102 |
103 | $indexModel = strtolower(class_basename(static::class));
104 | $query = IndexableBuild::query()->where('index_model', $indexModel);
105 | if ($byLatest) {
106 | $query->orderByDesc('created_at');
107 | }
108 |
109 | return $query;
110 | }
111 |
112 | public static function whereFailedIndexBuilds($byLatest = false): Builder
113 | {
114 | $indexModel = strtolower(class_basename(static::class));
115 | $query = IndexableBuild::query()->where('index_model', $indexModel)->where('state', IndexableBuildState::FAILED);
116 | if ($byLatest) {
117 | $query->orderByDesc('created_at');
118 | }
119 |
120 | return $query;
121 | }
122 |
123 | public static function whereMigrations($byLatest = false): Builder
124 | {
125 | $indexModel = strtolower(class_basename(static::class));
126 | $query = IndexableMigrationLog::query()->where('index_model', $indexModel);
127 | if ($byLatest) {
128 | $query->orderByDesc('created_at');
129 | }
130 |
131 | return $query;
132 | }
133 |
134 | public static function whereMigrationErrors($byLatest = false): Builder
135 | {
136 | $indexModel = strtolower(class_basename(static::class));
137 | $query = IndexableMigrationLog::query()->where('index_model', $indexModel)->where('state', IndexableMigrationLogState::FAILED);
138 | if ($byLatest) {
139 | $query->orderByDesc('created_at');
140 | }
141 |
142 | return $query;
143 |
144 | }
145 | }
146 |
--------------------------------------------------------------------------------
/src/Indexable.php:
--------------------------------------------------------------------------------
1 | searchPhrasePrefix($phrase)->getBase();
30 | }
31 |
32 | public static function viaIndex(): IndexModel|Builder
33 | {
34 | $indexModel = Lens::fetchIndexModelClass((new static));
35 |
36 | return $indexModel::query();
37 |
38 | }
39 |
40 | /**
41 | * Fetch the fully qualified class name of the index model.
42 | *
43 | * @return class-string The fully qualified class name of the index model.
44 | */
45 | public static function indexModel(): string
46 | {
47 | return Lens::fetchIndexModelClass((new static));
48 | }
49 |
50 | public function returnIndex(): ?IndexModel
51 | {
52 | $modelId = $this->{$this->getKeyName()};
53 | $indexModel = Lens::fetchIndexModelClass($this);
54 |
55 | try {
56 | return $indexModel::where('id', $modelId)->first();
57 | } catch (Exception $e) {
58 |
59 | }
60 |
61 | return null;
62 | }
63 |
64 | public function buildIndex(): array
65 | {
66 | $modelId = $this->{$this->getKeyName()};
67 | $indexModel = Lens::fetchIndexModelClass($this);
68 |
69 | $build = $indexModel::indexBuild($modelId, 'Direct call from '.get_class($this).' trait');
70 |
71 | return $build->toArray();
72 |
73 | }
74 |
75 | public function excludeIndex(): bool
76 | {
77 | return false;
78 | }
79 |
80 | public function removeIndex(): bool
81 | {
82 | $modelId = $this->{$this->getKeyName()};
83 | $indexModel = Lens::fetchIndexModelClass($this);
84 |
85 | try {
86 | $deleted = $indexModel::destory($modelId);
87 | } catch (Exception $e) {
88 | return false;
89 | }
90 |
91 | return $deleted;
92 | }
93 | }
94 |
--------------------------------------------------------------------------------
/src/Jobs/BulkBuildStateUpdateJob.php:
--------------------------------------------------------------------------------
1 | buildStates)) {
23 | foreach ($this->buildStates as $modelId => $buildState) {
24 | IndexableBuild::writeState(class_basename($this->baseModel), $modelId, class_basename($this->indexModel), $buildState, 'Bulk Index');
25 | }
26 | }
27 | }
28 | }
29 |
--------------------------------------------------------------------------------
/src/Jobs/IndexBuildJob.php:
--------------------------------------------------------------------------------
1 | onQueue(config('elasticlens.queue'));
22 | }
23 | }
24 |
25 | /**
26 | * @throws Exception
27 | */
28 | public function handle(): void
29 | {
30 | $this->indexModelClass::indexBuild($this->modelId, 'Observed: '.$this->observedModel);
31 | }
32 | }
33 |
--------------------------------------------------------------------------------
/src/Jobs/IndexDeletedJob.php:
--------------------------------------------------------------------------------
1 | onQueue(config('elasticlens.queue'));
23 | }
24 | }
25 |
26 | /**
27 | * @throws Exception
28 | */
29 | public function handle(): void
30 | {
31 | $builder = new LensBuilder($this->indexModel);
32 | $builder->processDelete($this->modelId);
33 | }
34 | }
35 |
--------------------------------------------------------------------------------
/src/Lens.php:
--------------------------------------------------------------------------------
1 | $namespace) {
38 | foreach (glob(base_path($path.'*.php')) as $file) {
39 | $indexModel = $namespace.'\\'.basename($file, '.php');
40 | $indexes[] = (new $indexModel)::class;
41 | }
42 | }
43 |
44 | return $indexes;
45 | }
46 |
47 | public static function checkIfWatched($model, $indexModel): bool
48 | {
49 | $watchers = config('elasticlens.watchers');
50 | if (isset($watchers[$model])) {
51 | return in_array($indexModel, $watchers[$model]);
52 | }
53 |
54 | return false;
55 | }
56 | }
57 |
--------------------------------------------------------------------------------
/src/Models/IndexableBuild.php:
--------------------------------------------------------------------------------
1 | IndexableBuildState::class,
46 | ];
47 |
48 | public function __construct()
49 | {
50 | parent::__construct();
51 | $this->setConnection(self::getConnectionName());
52 | }
53 |
54 | public static function isEnabled()
55 | {
56 | return config('elasticlens.index_build_state.enabled', true);
57 | }
58 |
59 | public function getStateNameAttribute(): string
60 | {
61 | return $this->state->label();
62 | }
63 |
64 | public function getStateColorAttribute(): string
65 | {
66 | return $this->state->color();
67 | }
68 |
69 | public static function returnState($model, $modelId, $indexModel): mixed
70 | {
71 | return self::where('model', strtolower($model))->where('model_id', $modelId)->where('index_model', strtolower($indexModel))->first();
72 | }
73 |
74 | public static function writeState($model, $modelId, $indexModel, BuildResult $buildResult, $observerModel): ?IndexableBuild
75 | {
76 | if (! self::isEnabled()) {
77 | return null;
78 | }
79 | $model = strtolower($model);
80 | $indexModel = strtolower($indexModel);
81 | $stateData = $buildResult->toArray();
82 | unset($stateData['model']);
83 | $state = IndexableBuildState::FAILED;
84 | if ($buildResult->success) {
85 | $state = IndexableBuildState::SUCCESS;
86 | unset($stateData['msg']);
87 | unset($stateData['details']);
88 | unset($stateData['map']);
89 | }
90 | if ($buildResult->skipped) {
91 | $state = IndexableBuildState::SKIPPED;
92 | unset($stateData['msg']);
93 | unset($stateData['details']);
94 | unset($stateData['map']);
95 | }
96 |
97 | $source = $observerModel;
98 | $stateModel = self::returnState($model, $modelId, $indexModel);
99 | if (! $stateModel) {
100 | $stateModel = new IndexableBuild;
101 | $stateModel->model = $model;
102 | $stateModel->model_id = $modelId;
103 | $stateModel->index_model = $indexModel;
104 | }
105 | $stateModel->state = $state;
106 | $stateModel->state_data = $stateData;
107 | $stateModel->last_source = $source;
108 | $logs = $stateModel->_prepLogs($stateData, $source);
109 | $stateModel->logs = $logs;
110 | $stateModel->withoutRefresh()->save();
111 |
112 | return $stateModel;
113 | }
114 |
115 | public static function countModelErrors($indexModel): int
116 | {
117 | return IndexableBuild::where('index_model', strtolower($indexModel))->where('state', IndexableBuildState::FAILED)->count();
118 | }
119 |
120 | public static function countModelSkips($indexModel): int
121 | {
122 | return IndexableBuild::where('index_model', strtolower($indexModel))->where('state', IndexableBuildState::SKIPPED)->count();
123 | }
124 |
125 | public static function countModelRecords($indexModel): int
126 | {
127 | return IndexableBuild::where('index_model', strtolower($indexModel))->count();
128 | }
129 |
130 | public static function deleteState($model, $modelId, $indexModel): void
131 | {
132 | $stateModel = IndexableBuild::returnState(strtolower($model), $modelId, strtolower($indexModel));
133 | $stateModel?->delete();
134 | }
135 |
136 | public static function deleteStateModel($indexModel): void
137 | {
138 | IndexableBuild::where('index_model', strtolower($indexModel))->delete();
139 |
140 | }
141 |
142 | // ----------------------------------------------------------------------
143 | // Helpers
144 | // ----------------------------------------------------------------------
145 |
146 | public function _prepLogs($stateData, $source): array
147 | {
148 | $trim = config('elasticlens.index_build_state.log_trim', 2);
149 | if (! $trim) {
150 | return [];
151 | }
152 | $logs = $this->logs ?? [];
153 | unset($stateData['id']);
154 | $logs[] = [
155 | 'ts' => time(),
156 | 'success' => $stateData['success'],
157 | 'data' => $stateData,
158 | 'source' => $source,
159 | ];
160 | $collection = collect($logs);
161 |
162 | return $collection->sortByDesc('ts')->take($trim)->values()->all();
163 |
164 | }
165 |
166 | // ----------------------------------------------------------------------
167 | // Config
168 | // ----------------------------------------------------------------------
169 |
170 | public static function connectionName(): string
171 | {
172 | return config('elasticlens.database', 'elasticsearch');
173 | }
174 |
175 | public static function checkHasIndex(): bool
176 | {
177 | $connectionName = self::connectionName();
178 |
179 | return Schema::on($connectionName)->indexExists('indexable_builds');
180 | }
181 | }
182 |
--------------------------------------------------------------------------------
/src/Models/IndexableMigrationLog.php:
--------------------------------------------------------------------------------
1 | IndexableMigrationLogState::class,
39 | ];
40 |
41 | public static function isEnabled()
42 | {
43 | return config('elasticlens.index_migration_logs.enabled', true);
44 | }
45 |
46 | public function __construct()
47 | {
48 | parent::__construct();
49 | $this->setConnection(self::getConnectionName());
50 | }
51 |
52 | public function getVersionAttribute()
53 | {
54 | return 'v'.$this->version_major.'.'.$this->version_minor;
55 | }
56 |
57 | public static function getLatestVersion($indexModel): ?string
58 | {
59 | $indexModel = strtolower($indexModel);
60 | if (! self::isEnabled()) {
61 | return null;
62 | }
63 | $latest = self::getLatestMigration($indexModel);
64 |
65 | return $latest?->version;
66 | }
67 |
68 | public static function getLatestMigration($indexModel): mixed
69 | {
70 | $indexModel = strtolower($indexModel);
71 | $log = null;
72 | try {
73 | $log = self::where('index_model', $indexModel)->orderBy('version_major', 'desc')->orderBy('version_minor', 'desc')->first();
74 | } catch (Exception $e) {
75 |
76 | }
77 |
78 | return $log;
79 | }
80 |
81 | public static function saveMigrationLog($indexModel, $majorVersion, $state, $map)
82 | {
83 | $indexModel = strtolower($indexModel);
84 |
85 | $minor = self::calculateNextMinorVersion($indexModel, $majorVersion);
86 | $log = new self;
87 | $log->index_model = $indexModel;
88 | $log->version_major = $majorVersion;
89 | $log->version_minor = $minor;
90 | $log->state = $state;
91 | $log->map = $map;
92 | $log->save();
93 | }
94 |
95 | public static function calculateNextMinorVersion($indexModel, $majorVersion): int
96 | {
97 | $indexModel = strtolower($indexModel);
98 | $lastMigration = self::getLatestMigration($indexModel);
99 | if ($lastMigration) {
100 | if ($lastMigration->version_major == $majorVersion) {
101 | return $lastMigration->version_minor + 1;
102 | }
103 | }
104 |
105 | return 0;
106 | }
107 |
108 | // ----------------------------------------------------------------------
109 | // Config
110 | // ----------------------------------------------------------------------
111 |
112 | public static function connectionName(): string
113 | {
114 | return config('elasticlens.database', 'elasticsearch');
115 | }
116 |
117 | public static function checkHasIndex(): bool
118 | {
119 | $connectionName = self::connectionName();
120 |
121 | return Schema::on($connectionName)->indexExists('indexable_migration_logs');
122 | }
123 | }
124 |
--------------------------------------------------------------------------------
/src/Observers/BaseModelObserver.php:
--------------------------------------------------------------------------------
1 | dispatchBuild($model->{$builder->baseModelPrimaryKey}, ($builder->baseModel).' (saved)');
21 | } catch (Exception $e) {
22 |
23 | }
24 | }
25 |
26 | public function deleting($model): void
27 | {
28 | $modelId = $model->{$model->getKeyName()};
29 | $indexModel = Lens::fetchIndexModelClass($model);
30 | try {
31 | $index = new LensBuilder($indexModel);
32 | $index->dispatchDeleted($modelId);
33 | } catch (Exception $e) {
34 |
35 | }
36 |
37 | }
38 | }
39 |
--------------------------------------------------------------------------------
/src/Observers/ObserverRegistry.php:
--------------------------------------------------------------------------------
1 | getObserverSet();
25 |
26 | if (! empty($observers['base'])) {
27 | $baseModel::observe(new BaseModelObserver);
28 | }
29 | if (! empty($observers['embedded'])) {
30 | foreach ($observers['embedded'] as $settings) {
31 | if ($settings['observe']) {
32 | $embeddedModel = $settings['relation'];
33 | if (! Lens::checkIfWatched($embeddedModel, $indexModel)) {
34 | self::watchEmbedded($embeddedModel, $settings, $baseModel);
35 | }
36 | }
37 | }
38 | }
39 | }
40 |
41 | /**
42 | * @throws Exception
43 | */
44 | public static function registerWatcher($watchedModel, $indexModel): void
45 | {
46 | $indexModelInstance = new $indexModel;
47 | $observers = $indexModelInstance->getObserverSet();
48 | $baseModel = $indexModelInstance->getBaseModel();
49 | if (! empty($observers['embedded'])) {
50 | foreach ($observers['embedded'] as $settings) {
51 | if ($watchedModel == $settings['relation'] && $settings['observe']) {
52 | self::watchEmbedded($watchedModel, $settings, $baseModel);
53 | }
54 |
55 | }
56 | }
57 | }
58 |
59 | /**
60 | * @throws Exception
61 | */
62 | private static function watchEmbedded($watchedModel, $settings, $baseModel): void
63 | {
64 | $watchedModel::saved(function ($model) use ($settings, $baseModel) {
65 | $watcher = new EmbeddedModelTrigger($model, $baseModel, $settings);
66 | $watcher->handle('saved');
67 | });
68 | $watchedModel::deleted(function ($model) use ($settings, $baseModel) {
69 | $watcher = new EmbeddedModelTrigger($model, $baseModel, $settings);
70 | $watcher->handle('deleted');
71 | });
72 | }
73 | }
74 |
--------------------------------------------------------------------------------
/src/Traits/IndexBaseModel.php:
--------------------------------------------------------------------------------
1 | baseModel) {
20 | return $this->guessBaseModelName();
21 | }
22 |
23 | return $this->baseModel;
24 | }
25 |
26 | public function isBaseModelDefined(): bool
27 | {
28 | return ! empty($this->baseModel);
29 | }
30 | }
31 |
--------------------------------------------------------------------------------
/src/Traits/IndexFieldMap.php:
--------------------------------------------------------------------------------
1 | baseModel);
21 | }
22 |
23 | public function getFieldSet()
24 | {
25 | return $this->fieldMap()->getFieldMap();
26 | }
27 |
28 | public function getRelationships()
29 | {
30 | return $this->fieldMap()->getRelationships();
31 | }
32 |
33 | public function getObserverSet()
34 | {
35 | $base = null;
36 | if (! empty($this->observeBase)) {
37 | $base = $this->getBaseModel();
38 | }
39 | $embedded = $this->fieldMap()->getObservers();
40 | if ($embedded) {
41 | $embedded = $this->mapUpstreamEmbeds($embedded);
42 | }
43 |
44 | return [
45 | 'base' => $base,
46 | 'embedded' => $embedded,
47 | ];
48 | }
49 |
50 | public function getObservedModels()
51 | {
52 | $set = $this->getObserverSet();
53 | $embedded = $set['embedded'];
54 | $embeddedModels = [];
55 | if ($embedded) {
56 | foreach ($embedded as $embed) {
57 | if ($embed['observe']) {
58 | $embeddedModels[] = $embed['model'];
59 | }
60 | }
61 | }
62 |
63 | return [
64 | 'base' => $set['base'],
65 | 'embedded' => $embeddedModels,
66 | ];
67 | }
68 |
69 | // ----------------------------------------------------------------------
70 | // Build Index
71 | // ----------------------------------------------------------------------
72 |
73 | /**
74 | * @throws Exception
75 | */
76 | public static function indexBuild($id, $source): BuildResult
77 | {
78 | return (new static)->getBuilder()->buildIndex($id, $source);
79 | }
80 |
81 | /**
82 | * @throws Exception
83 | */
84 | public function indexRebuild($source): BuildResult
85 | {
86 | return $this->getBuilder()->buildIndex($this->id, $source);
87 | }
88 |
89 | /**
90 | * @throws Exception
91 | */
92 | public function getBuilder(): LensBuilder
93 | {
94 | return new LensBuilder(get_class($this));
95 | }
96 |
97 | // ----------------------------------------------------------------------
98 | // Map any upstream embeds
99 | // ----------------------------------------------------------------------
100 |
101 | public function mapUpstreamEmbeds($embeds)
102 | {
103 | foreach ($embeds as $i => $embed) {
104 | $embeds[$i] = $this->_fetchUpstream($embed, $embeds);
105 | }
106 |
107 | return $embeds;
108 | }
109 |
110 | private function _fetchUpstream($embed, $embeds)
111 | {
112 | if ($embed['model'] !== $this->baseModel) {
113 | foreach ($embeds as $em) {
114 | if ($em['relation'] === $embed['model']) {
115 | $embed['upstream'] = $this->_fetchUpstream($em, $embeds);
116 | }
117 | }
118 | }
119 |
120 | return $embed;
121 | }
122 | }
123 |
--------------------------------------------------------------------------------
/src/Traits/IndexMigrationMap.php:
--------------------------------------------------------------------------------
1 | $this->migrationMajorVersion,
23 | 'blueprint' => $this->migrationMap(),
24 | ];
25 | }
26 |
27 | public function getCurrentMigrationVersion(): string
28 | {
29 | $version = IndexableMigrationLog::getLatestVersion(class_basename($this));
30 | if (! $version) {
31 | $version = 'v'.$this->migrationMajorVersion.'.0';
32 | }
33 |
34 | return $version;
35 | }
36 |
37 | public static function validateIndexMigrationBlueprint(): array
38 | {
39 | $indexModel = new static;
40 | $version = $indexModel->getCurrentMigrationVersion();
41 | $blueprint = $indexModel->migrationMap();
42 | $indexModelTable = $indexModel->getTable();
43 | $validator = new MigrationValidator($version, $blueprint, $indexModelTable);
44 |
45 | return $validator->testMigration();
46 | }
47 | }
48 |
--------------------------------------------------------------------------------
/src/Traits/Timer.php:
--------------------------------------------------------------------------------
1 | timer['start'] = microtime(true);
14 |
15 | return $this->timer;
16 | }
17 |
18 | private function _endTimer(): array
19 | {
20 | if (! empty($this->timer['start'])) {
21 | $this->timer['end'] = microtime(true);
22 | $this->timer['took'] = round(($this->timer['end'] - $this->timer['start']) * 1000, 0);
23 | $this->timer['time']['ms'] = $this->timer['took'];
24 | $this->timer['time']['sec'] = round($this->timer['took'] / 1000, 2);
25 | $this->timer['time']['min'] = round($this->timer['took'] / 60000, 2);
26 | } else {
27 | $this->timer['time'] = 'Time was not initialized';
28 | }
29 |
30 | return $this->timer;
31 | }
32 |
33 | public function getTime()
34 | {
35 | $this->_endTimer();
36 |
37 | return $this->timer['time'];
38 |
39 | }
40 | }
41 |
--------------------------------------------------------------------------------
/src/Utils/IndexHelper.php:
--------------------------------------------------------------------------------
1 | indexModelTable);
15 | $mappings = reset($mappings);
16 | $schema = [];
17 | if (! empty($mappings['mappings']['properties'])) {
18 |
19 | $mappings = $mappings['mappings']['properties'];
20 | if (! empty($mappings['_meta']['properties']['id'])) {
21 | $id = $mappings['_meta']['properties']['id'];
22 |
23 | $schema['id'] = $id;
24 |
25 | } else {
26 | $schema['id'] = [
27 | 'type' => 'text',
28 | ];
29 | }
30 |
31 | unset($mappings['_meta']);
32 |
33 | foreach ($mappings as $field => $mapping) {
34 | $schema[$field] = $mapping;
35 | }
36 |
37 | }
38 |
39 | return $schema;
40 | }
41 |
42 | public function indexFields($sorted = false, $excludeEmbedded = false): array
43 | {
44 | $schema = $this->indexMapping();
45 | if (! $schema) {
46 | return [];
47 | }
48 |
49 | $maps = $this->_mapProperties($schema);
50 | if ($sorted) {
51 | $maps = $this->_reasonableMapSort($maps, $excludeEmbedded);
52 | }
53 |
54 | return $maps;
55 | }
56 |
57 | private function _mapField($val, $field): array
58 | {
59 | $maps = [];
60 | if (isset($val['properties'])) {
61 | return $this->_mapProperties($val['properties'], $field);
62 |
63 | }
64 | $maps[$field] = $val['type'];
65 |
66 | return $maps;
67 | }
68 |
69 | private function _mapProperties($properties, $parentField = null): array
70 | {
71 | $propertyMaps = [];
72 | foreach ($properties as $field => $value) {
73 | if ($parentField) {
74 | $field = $parentField.'.'.$field;
75 | }
76 | $maps = $this->_mapField($value, $field);
77 |
78 | foreach ($maps as $f => $type) {
79 | $propertyMaps[$f] = $type;
80 | }
81 | }
82 |
83 | return $propertyMaps;
84 | }
85 |
86 | private function _reasonableMapSort($maps, $excludeEmbedded): array
87 | {
88 | $sorted = [];
89 | $commonFields = [
90 | 'id',
91 | 'name',
92 | 'email',
93 | 'first_name',
94 | 'last_name',
95 | 'type',
96 | 'state',
97 | 'status',
98 | 'created_at',
99 | 'updated_at',
100 | ];
101 | foreach ($commonFields as $field) {
102 | if (isset($maps[$field])) {
103 | $sorted[$field] = $maps[$field];
104 | unset($maps[$field]);
105 | }
106 | }
107 | $embedded = [];
108 | foreach ($maps as $field => $type) {
109 | if (str_contains($field, '.')) {
110 | $embedded[$field] = $type;
111 | unset($maps[$field]);
112 |
113 | continue;
114 | }
115 | $sorted[$field] = $type;
116 | }
117 | if (! $excludeEmbedded) {
118 | if (! empty($embedded)) {
119 | foreach ($embedded as $field => $type) {
120 | $sorted[$field] = $type;
121 | }
122 | }
123 | }
124 |
125 | return $sorted;
126 | }
127 | }
128 |
--------------------------------------------------------------------------------
/src/Watchers/EmbeddedModelTrigger.php:
--------------------------------------------------------------------------------
1 | model = $model;
24 | $this->baseModel = $baseModel;
25 | $this->settings = $settings;
26 | }
27 |
28 | /**
29 | * @throws Exception
30 | */
31 | public function handle($event): void
32 | {
33 | $this->modelChain[] = '['.class_basename($this->model).'::'.$event.']';
34 | $this->fetchRecords($this->model, $this->settings);
35 | }
36 |
37 | /**
38 | * @throws Exception
39 | */
40 | public function fetchRecords($modelInstance, $settings): void
41 | {
42 | $this->modelChain[] = class_basename($settings['relation']);
43 | if ($settings['model'] === $this->baseModel) {
44 | $this->modelChain[] = class_basename($this->baseModel);
45 | $this->_dispatchBuild($modelInstance, $settings);
46 | } else {
47 | $modelKey = $settings['whereRelatedField'];
48 | $modelValue = $modelInstance->{$modelKey};
49 | $parentKey = $settings['equalsModelField'];
50 | $parentModel = $settings['model'];
51 | $parentModel::where($parentKey, $modelValue)->chunk(100, function ($records) use ($settings) {
52 | foreach ($records as $record) {
53 | if (! empty($settings['upstream'])) {
54 | $this->fetchRecords($record, $settings['upstream']);
55 | } else {
56 | $this->fetchRecords($record, $settings);
57 | }
58 | }
59 | });
60 | }
61 |
62 | }
63 |
64 | /**
65 | * @throws Exception
66 | */
67 | public function _dispatchBuild($modelInstance, $settings): void
68 | {
69 | $modelInstanceKey = $settings['whereRelatedField'];
70 | $value = $modelInstance->{$modelInstanceKey};
71 | $baseKey = $settings['equalsModelField'];
72 | $observers = implode(' -> ', $this->modelChain);
73 | $indexModel = Lens::fetchIndexModelClass($this->baseModel);
74 | $builder = new LensBuilder($indexModel);
75 | (new $this->baseModel)::where($baseKey, $value)->chunk(100, function ($records) use ($builder, $observers) {
76 | foreach ($records as $record) {
77 | $builder->dispatchBuild($record->{$builder->baseModelPrimaryKey}, $observers);
78 | }
79 | });
80 | }
81 | }
82 |
--------------------------------------------------------------------------------