← Index
NYTProf Performance Profile   « line view »
For flows_to_es.pl
  Run on Mon May 9 23:27:59 2016
Reported on Mon May 9 23:28:08 2016

Filename/opt/flows/lib/lib/perl5/Search/Elasticsearch.pm
StatementsExecuted 41 statements in 1.04ms
Subroutines
Calls P F Exclusive
Time		 Inclusive
Time		 Subroutine
1112.05ms14.4msSearch::Elasticsearch::::BEGIN@3Search::Elasticsearch::BEGIN@3
1111.31ms7.18msSearch::Elasticsearch::::BEGIN@6Search::Elasticsearch::BEGIN@6
111739µs11.8msSearch::Elasticsearch::::BEGIN@5Search::Elasticsearch::BEGIN@5
111105µs155msSearch::Elasticsearch::::newSearch::Elasticsearch::new
Call graph for these subroutines as a Graphviz dot language file.
Line State
ments		 Time
on line		 Calls Time
in subs		 Code
1package Search::Elasticsearch;
2
33143µs214.4ms
# spent 14.4ms (2.05+12.3) within Search::Elasticsearch::BEGIN@3 which was called: # once (2.05ms+12.3ms) by main::BEGIN@147 at line 3
use Moo 1.003 ();
# spent 14.4ms making 1 call to Search::Elasticsearch::BEGIN@3 # spent 11µs making 1 call to UNIVERSAL::VERSION
4
52121µs212.0ms
# spent 11.8ms (739µs+11.0) within Search::Elasticsearch::BEGIN@5 which was called: # once (739µs+11.0ms) by main::BEGIN@147 at line 5
use Search::Elasticsearch::Util qw(parse_params load_plugin);
# spent 11.8ms making 1 call to Search::Elasticsearch::BEGIN@5 # spent 210µs making 1 call to Sub::Exporter::__ANON__[/opt/flows/lib/lib/perl5/Sub/Exporter.pm:337]
62685µs27.39ms
# spent 7.18ms (1.31+5.86) within Search::Elasticsearch::BEGIN@6 which was called: # once (1.31ms+5.86ms) by main::BEGIN@147 at line 6
use namespace::clean;
# spent 7.18ms making 1 call to Search::Elasticsearch::BEGIN@6 # spent 214µs making 1 call to namespace::clean::import
7
81500nsour $VERSION = '2.02';
9
1016µsmy %Default_Plugins = (
11 client => [ 'Search::Elasticsearch::Client', '2_0::Direct' ],
12 cxn_factory => [ 'Search::Elasticsearch::Cxn::Factory', '' ],
13 cxn_pool => [ 'Search::Elasticsearch::CxnPool', 'Static' ],
14 logger => [ 'Search::Elasticsearch::Logger', 'LogAny' ],
15 serializer => [ 'Search::Elasticsearch::Serializer', 'JSON' ],
16 transport => [ 'Search::Elasticsearch::Transport', '' ],
17);
18
1911µsmy @Load_Order = qw(
20 serializer
21 logger
22 cxn_factory
23 cxn_pool
24 transport
25 client
26);
27
28#===================================
29
# spent 155ms (105µs+155) within Search::Elasticsearch::new which was called: # once (105µs+155ms) by main::create_es_bulk at line 151 of flows_to_es.pl
sub new {
30#===================================
3117µs117µs my ( $class, $params ) = parse_params(@_);
# spent 17µs making 1 call to Search::Elasticsearch::Util::parse_params
32
3313µs $params->{cxn} ||= 'HTTPTiny';
3412µs my $plugins = delete $params->{plugins} || [];
35
3614µs for my $name (@Load_Order) {
3769µs my ( $base, $default ) = @{ $Default_Plugins{$name} };
3863µs my $sub_class = $params->{$name} || $default;
39612µs6109ms my $plugin_class = load_plugin( $base, $sub_class );
# spent 109ms making 6 calls to Search::Elasticsearch::Util::load_plugin, avg 18.1ms/call
40624µs66.37ms $params->{$name} = $plugin_class->new($params);
# spent 1.92ms making 1 call to Search::Elasticsearch::CxnPool::Static::new # spent 1.01ms making 1 call to Search::Elasticsearch::Logger::LogAny::new # spent 985µs making 1 call to Search::Elasticsearch::Client::2_0::Direct::new # spent 894µs making 1 call to Search::Elasticsearch::Transport::new # spent 798µs making 1 call to Search::Elasticsearch::Serializer::JSON::new # spent 754µs making 1 call to Search::Elasticsearch::Cxn::Factory::new
41 }
42
4311µs for my $name (@$plugins) {
44 my $plugin_class
45 = load_plugin( 'Search::Elasticsearch::Plugin', $name );
46 $plugin_class->_init_plugin($params);
47 }
48
4915µs return $params->{client};
50}
51
5216µs1;
53
54=pod
55
56=encoding UTF-8
57
58=head1 NAME
59
60Search::Elasticsearch - The official client for Elasticsearch
61
62=head1 VERSION
63
64version 2.02
65
66=head1 SYNOPSIS
67
68 use Search::Elasticsearch;
69
70 # Connect to localhost:9200:
71
72 my $e = Search::Elasticsearch->new();
73
74 # Round-robin between two nodes:
75
76 my $e = Search::Elasticsearch->new(
77 nodes => [
78 'search1:9200',
79 'search2:9200'
80 ]
81 );
82
83 # Connect to cluster at search1:9200, sniff all nodes and round-robin between them:
84
85 my $e = Search::Elasticsearch->new(
86 nodes => 'search1:9200',
87 cxn_pool => 'Sniff'
88 );
89
90 # Index a document:
91
92 $e->index(
93 index => 'my_app',
94 type => 'blog_post',
95 id => 1,
96 body => {
97 title => 'Elasticsearch clients',
98 content => 'Interesting content...',
99 date => '2013-09-24'
100 }
101 );
102
103 # Get the document:
104
105 my $doc = $e->get(
106 index => 'my_app',
107 type => 'blog_post',
108 id => 1
109 );
110
111 # Search:
112
113 my $results = $e->search(
114 index => 'my_app',
115 body => {
116 query => {
117 match => { title => 'elasticsearch' }
118 }
119 }
120 );
121
122 # Cluster requests:
123
124 $info = $e->cluster->info;
125 $health = $e->cluster->health;
126 $node_stats = $e->cluster->node_stats;
127
128 # Index requests:
129
130 $e->indices->create(index=>'my_index');
131 $e->indices->delete(index=>'my_index');
132
133=head1 DESCRIPTION
134
135L<Search::Elasticsearch> is the official Perl client for Elasticsearch,
136supported by L<elasticsearch.com|http://www.elasticsearch.com>. Elasticsearch
137itself is a flexible and powerful open source, distributed real-time
138search and analytics engine for the cloud. You can read more about it
139on L<elastic.co|http://www.elastic.co>.
140
141=head1 BACKWARDS COMPATIBILITY
142
143This version of the client supports the Elasticsearch 2.0 branch by
144default, see L</Client> section below for details of working with 1.0
145and 0.90 branches.
146
147=head2 Motivation
148
149=over
150
151I<The greatest deception men suffer is from their own opinions.>
152
153Leonardo da Vinci
154
155=back
156
157All of us have opinions, especially when it comes to designing APIs.
158Unfortunately, the opinions of programmers seldom coincide. The intention of
159this client, and of the officially supported clients available for other
160languages, is to provide robust support for the full native Elasticsearch API
161with as few opinions as possible: you should be able to read the
162L<Elasticsearch reference documentation|http://www.elastic.co/guide>
163and understand how to use this client, or any of the other official clients.
164
165Should you decide that you want to customize the API, then this client
166provides the basis for your code. It does the hard stuff for you,
167allowing you to build on top of it.
168
169=head2 Features
170
171This client provides:
172
173=over
174
175=item *
176
177Full support for all Elasticsearch APIs
178
179=item *
180
181HTTP backend (for an async backend using L<Promises>, see
182L<Search::Elasticsearch::Async>)
183
184=item *
185
186Robust networking support which handles load balancing, failure detection
187and failover
188
189=item *
190
191Good defaults
192
193=item *
194
195Helper utilities for more complex operations, such as
196L<bulk indexing|Search::Elasticsearch::Bulk>,
197L<scrolled searches|Search::Elasticsearch::Scroll> and
198L<reindexing|Search::Elasticsearch::Bulk/"reindex()">.
199
200=item *
201
202Logging support via L<Log::Any>
203
204=item *
205
206Compatibility with the official clients for Python, Ruby, PHP and Javascript
207
208=item *
209
210Easy extensibility
211
212=back
213
214=head1 INSTALLING ELASTICSEARCH
215
216You can download the latest version of Elasticsearch from
217L<http://www.elastic.co/download>. See the
218L<installation instructions|http://www.elastic.co/guide/reference/setup/installation/>
219for details. You will need to have a recent version of Java installed,
220preferably the Java v7 from Sun.
221
222=head1 CREATING A NEW INSTANCE
223
224The L</new()> method returns a new L<client|Search::Elasticsearch::Client::2_0::Direct>
225which can be used to run requests against the Elasticsearch cluster.
226
227 use Search::Elasticsearch;
228 my $e = Search::Elasticsearch->new( %params );
229
230The most important arguments to L</new()> are the following:
231
232=head2 C<nodes>
233
234The C<nodes> parameter tells the client which Elasticsearch nodes it should
235talk to. It can be a single node, multiples nodes or, if not
236specified, will default to C<localhost:9200>:
237
238 # default: localhost:9200
239 $e = Search::Elasticsearch->new();
240
241 # single
242 $e = Search::Elasticsearch->new( nodes => 'search_1:9200');
243
244 # multiple
245 $e = Search::Elasticsearch->new(
246 nodes => [
247 'search_1:9200',
248 'search_2:9200'
249 ]
250 );
251
252Each C<node> can be a URL including a scheme, host, port, path and userinfo
253(for authentication). For instance, this would be a valid node:
254
255 https://username:password@search.domain.com:443/prefix/path
256
257See L<Search::Elasticsearch::Role::Cxn::HTTP/node> for more on node specification.
258
259=head2 C<cxn_pool>
260
261The L<CxnPool|Search::Elasticsearch::Role::CxnPool> modules manage connections to
262nodes in the Elasticsearch cluster. They handle the load balancing between
263nodes and failover when nodes fail. Which C<CxnPool> you should use depends on
264where your cluster is. There are three choices:
265
266=over
267
268=item * C<Static>
269
270 $e = Search::Elasticsearch->new(
271 cxn_pool => 'Static' # default
272 nodes => [
273 'search1.domain.com:9200',
274 'search2.domain.com:9200'
275 ],
276 );
277
278The L<Static|Search::Elasticsearch::CxnPool::Static> connection pool, which is the
279default, should be used when you don't have direct access to the Elasticsearch
280cluster, eg when you are accessing the cluster through a proxy. See
281L<Search::Elasticsearch::CxnPool::Static> for more.
282
283=item * C<Sniff>
284
285 $e = Search::Elasticsearch->new(
286 cxn_pool => 'Sniff',
287 nodes => [
288 'search1:9200',
289 'search2:9200'
290 ],
291 );
292
293The L<Sniff|Search::Elasticsearch::CxnPool::Sniff> connection pool should be used
294when you B<do> have direct access to the Elasticsearch cluster, eg when
295your web servers and Elasticsearch servers are on the same network.
296The nodes that you specify are used to I<discover> the cluster, which is
297then I<sniffed> to find the current list of live nodes that the cluster
298knows about. See L<Search::Elasticsearch::CxnPool::Sniff>.
299
300=item * C<Static::NoPing>
301
302 $e = Search::Elasticsearch->new(
303 cxn_pool => 'Static::NoPing'
304 nodes => [
305 'proxy1.domain.com:80',
306 'proxy2.domain.com:80'
307 ],
308 );
309
310The L<Static::NoPing|Search::Elasticsearch::CxnPool::Static::NoPing> connection
311pool should be used when your access to a remote cluster is so limited
312that you cannot ping individual nodes with a C<HEAD /> request.
313
314See L<Search::Elasticsearch::CxnPool::Static::NoPing> for more.
315
316=back
317
318=head2 C<trace_to>
319
320For debugging purposes, it is useful to be able to dump the actual HTTP
321requests which are sent to the cluster, and the response that is received.
322This can be enabled with the C<trace_to> parameter, as follows:
323
324 # To STDERR
325 $e = Search::Elasticsearch->new(
326 trace_to => 'Stderr'
327 );
328
329 # To a file
330 $e = Search::Elasticsearch->new(
331 trace_to => ['File','/path/to/filename']
332 );
333
334Logging is handled by L<Log::Any>. See L<Search::Elasticsearch::Logger::LogAny>
335for more information.
336
337=head2 Other
338
339Other arguments are explained in the respective L<module docs|/MODULES>.
340
341=head1 RUNNING REQUESTS
342
343When you create a new instance of Search::Elasticsearch, it returns a
344L<client|Search::Elasticsearch::Client::2_0::Direct> object, which can be used for
345running requests.
346
347 use Search::Elasticsearch;
348 my $e = Search::Elasticsearch->new( %params );
349
350 # create an index
351 $e->indices->create( index => 'my_index' );
352
353 # index a document
354 $e->index(
355 index => 'my_index',
356 type => 'blog_post',
357 id => 1,
358 body => {
359 title => 'Elasticsearch clients',
360 content => 'Interesting content...',
361 date => '2013-09-24'
362 }
363 );
364
365See L<Search::Elasticsearch::Client::2_0::Direct> for more details about the requests that
366can be run.
367
368=head1 MODULES
369
370Each chunk of functionality is handled by a different module,
371which can be specified in the call to L<new()> as shown in L<cxn_pool> above.
372For instance, the following will use the L<Search::Elasticsearch::CxnPool::Sniff>
373module for the connection pool.
374
375 $e = Search::Elasticsearch->new(
376 cxn_pool => 'Sniff'
377 );
378
379Custom modules can be named with the appropriate prefix,
380eg C<Search::Elasticsearch::CxnPool::>, or by prefixing the full class name
381with C<+>:
382
383 $e = Search::Elasticsearch->new(
384 cxn_pool => '+My::Custom::CxnClass'
385 );
386
387The modules that you can override are specified with the following
388arguments to L</new()>:
389
390=head2 C<client>
391
392The class to use for the client functionality, which provides
393methods that can be called to execute requests, such as
394C<search()>, C<index()> or C<delete()>. The client parses the user's
395requests and passes them to the L</transport> class to be executed.
396
397The default version of the client is C<2_0::Direct>, which can
398be explicitly specified as follows:
399
400 $e = Search::Elasticsearch->new(
401 client => '2_0::Direct'
402 );
403
404See :
405
406=over
407
408=item * L<Search::Elasticsearch::Client::2_0::Direct> (default, for 2.0 branch)
409
410=item * L<Search::Elasticsearch::Client::1_0::Direct> (for 1.0 branch)
411
412=item * L<Search::Elasticsearch::Client::0_90::Direct> (for 0.90 branch)
413
414=item * L<Search::Elasticsearch::Client::Compat> (for migration from the old
415L<ElasticSearch> module)
416
417=back
418
419=head2 C<transport>
420
421The Transport class accepts a parsed request from the L</client> class,
422fetches a L</cxn> from its L</cxn_pool> and tries to execute the request,
423retrying after failure where appropriate. See:
424
425=over
426
427=item * L<Search::Elasticsearch::Transport>
428
429=back
430
431=head2 C<cxn>
432
433The class which handles raw requests to Elasticsearch nodes.
434See:
435
436=over
437
438=item * L<Search::Elasticsearch::Cxn::HTTPTiny> (default)
439
440=item * L<Search::Elasticsearch::Cxn::Hijk>
441
442=item * L<Search::Elasticsearch::Cxn::LWP>
443
444=item * L<Search::Elasticsearch::Cxn::NetCurl>
445
446=back
447
448=head2 C<cxn_factory>
449
450The class which the L</cxn_pool> uses to create new L</cxn> objects.
451See:
452
453=over
454
455=item * L<Search::Elasticsearch::Cxn::Factory>
456
457=back
458
459=head2 C<cxn_pool> (2)
460
461The class to use for the L<connection pool|/cxn_pool> functionality.
462It calls the L</cxn_factory> class to create new L</cxn> objects when
463appropriate. See:
464
465=over
466
467=item * L<Search::Elasticsearch::CxnPool::Static> (default)
468
469=item * L<Search::Elasticsearch::CxnPool::Sniff>
470
471=item * L<Search::Elasticsearch::CxnPool::Static::NoPing>
472
473=back
474
475=head2 C<logger>
476
477The class to use for logging events and tracing HTTP requests/responses. See:
478
479=over
480
481=item * L<Search::Elasticsearch::Logger::LogAny>
482
483=back
484
485=head2 C<serializer>
486
487The class to use for serializing request bodies and deserializing response
488bodies. See:
489
490=over
491
492=item * L<Search::Elasticsearch::Serializer::JSON> (default)
493
494=item * L<Search::Elasticsearch::Serializer::JSON::Cpanel>
495
496=item * L<Search::Elasticsearch::Serializer::JSON::XS>
497
498=item * L<Search::Elasticsearch::Serializer::JSON::PP>
499
500=back
501
502=head1 MIGRATING FROM ElasticSearch.pm
503
504See L<Search::Elasticsearch::Compat>, which allows you to run your old
505L<ElasticSearch> code with the new L<Search::Elasticsearch> module.
506
507The L<Search::Elasticsearch> API is pretty similar to the old L<ElasticSearch>
508API, but there are a few differences. The most notable are:
509
510=head2 C<hosts> vs C<servers>
511
512When instantiating a new Search::Elasticsearch instance, use C<nodes> instead
513of C<servers>:
514
515 $e = Search::Elasticsearch->new(
516 nodes => [ 'search1:9200', 'search2:9200' ]
517 );
518
519=head2 C<no_refresh>
520
521By default, the new client does not sniff the cluster to discover nodes.
522To enable sniffing, use:
523
524 $e = Search::Elasticsearch->new(
525 cxn_pool => 'Sniff',
526 nodes => [ 'search1:9200', 'search2:9200' ]
527 );
528
529To disable sniffing (the equivalent of setting C<no_refresh> to C<true>), do:
530
531 $e = Search::Elasticsearch->new(
532 nodes => [ 'search1:9200', 'search2:9200' ]
533 );
534
535=head2 Request parameters
536
537In the old client, you could specify query string and body parameters at
538the same level, eg:
539
540 $e->search(
541 search_type => 'count',
542 query => {
543 match_all => {}
544 }
545 );
546
547In the new client, body parameters should be passed in a C<body> element:
548
549 $e->search(
550 search_type => 'count',
551 body => {
552 query => {
553 match_all => {}
554 }
555 }
556 );
557
558=head2 C<trace_calls>
559
560The new client uses L<Log::Any> for event logging and request tracing.
561To trace requests/responses in C<curl> format, do:
562
563 # To STDERR
564 $e = Search::Elasticsearch->new (trace_to => 'Stderr');
565
566 # To a file
567 $e = Search::Elasticsearch->new (trace_to => ['File','/path/to/file.log']);
568
569=head2 SearchBuilder
570
571The old API integrated L<ElasticSearch::SearchBuilder> for an L<SQL::Abstract>
572style of writing queries and filters in Elasticsearch.
573This integration does not exist in the new client.
574
575=head2 Bulk methods and C<scrolled_search()>
576
577Bulk indexing has changed a lot in the new client. The helper methods, eg
578C<bulk_index()> and C<reindex()> have been removed from the main client,
579and the C<bulk()> method itself now simply returns the response from
580Elasticsearch. It doesn't interfere with processing at all.
581
582These helper methods have been replaced by the L<Search::Elasticsearch::Bulk>
583class. Similarly, C<scrolled_search()> has been replaced by the
584L<Search::Elasticsearch::Scroll>. These helper classes are accessible as:
585
586 $bulk = $e->bulk_helper( %args_to_new );
587 $scroll = $e->scroll_helper( %args_to_new );
588
589=head1 BUGS
590
591This is a stable API but this implementation is new. Watch this space
592for new releases.
593
594If you have any suggestions for improvements, or find any bugs, please report
595them to L<http://github.com/elasticsearch/elasticsearch-perl/issues>.
596I will be notified, and then you'll automatically be notified of progress on
597your bug as I make changes.
598
599=head1 SUPPORT
600
601You can find documentation for this module with the perldoc command.
602
603 perldoc Search::Elasticsearch
604
605You can also look for information at:
606
607=over 4
608
609=item * GitHub
610
611L<http://github.com/elasticsearch/elasticsearch-perl>
612
613=item * CPAN Ratings
614
615L<http://cpanratings.perl.org/d/Search::Elasticsearch>
616
617=item * Search MetaCPAN
618
619L<https://metacpan.org/module/Search::Elasticsearch>
620
621=item * IRC
622
623The L<#elasticsearch|irc://irc.freenode.net/elasticsearch> channel on
624C<irc.freenode.net>.
625
626=item * Mailing list
627
628The main L<Elasticsearch mailing list|http://www.elastic.co/community>.
629
630=back
631
632=head1 TEST SUITE
633
634The full test suite requires a live Elasticsearch node to run, and should
635be run as :
636
637 perl Makefile.PL
638 ES=localhost:9200 make test
639
640B<TESTS RUN IN THIS WAY ARE DESTRUCTIVE! DO NOT RUN AGAINST A CLUSTER WITH
641DATA YOU WANT TO KEEP!>
642
643You can change the Cxn class which is used by setting the C<ES_CXN>
644environment variable:
645
646 ES_CXN=Hijk ES=localhost:9200 make test
647
648=head1 AUTHOR
649
650Clinton Gormley <drtech@cpan.org>
651
652=head1 COPYRIGHT AND LICENSE
653
654This software is Copyright (c) 2016 by Elasticsearch BV.
655
656This is free software, licensed under:
657
658 The Apache License, Version 2.0, January 2004
659
660=cut
661
662113µs1126µs__END__
# spent 126µs making 1 call to B::Hooks::EndOfScope::XS::__ANON__[/opt/flows/lib/lib/perl5/B/Hooks/EndOfScope/XS.pm:17]
Report produced by the NYTProf 5.06 Perl profiler, developed by Tim Bunce and Adam Kaplan.