Posts

Showing posts with the label elasticsearch

ElaticSearch build geo get query is extremely slow

Image
Clash Royale CLAN TAG #URR8PPP ElaticSearch build geo get query is extremely slow I need to search for polygons inside given point and return the documents. I build a query as below. long start = System.currentTimeMillis(); GeoShapeQueryBuilder filter = QueryBuilders.geoShapeQuery("fence", new PointBuilder(111.868221, 10.906159)); filter.relation(ShapeRelation.INTERSECTS); long end = System.currentTimeMillis() - start; This query build takes around 400 milliseconds. Why this is so slow? Any way to improve elasticsearch java geo queries ? Thanks. By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

How to deploy Elasticsearch on Azure blob storage

Image
Clash Royale CLAN TAG #URR8PPP How to deploy Elasticsearch on Azure blob storage To deploy elastic search i followed steps in "Azure Marketplace Elasticsearch offering youtube video https://www.youtube.com/watch?v=6oRIpSXYrC4. Then i copied kibana IP with port no 5601 and entered same credentials which i gave during this steps. But my credentials is not working. I am not sure either my credentials is wrong or is there any prerequisite before deploying elastic search on Azure. If my credentials is wrong then how i can reset it. Thanks. Did you setup external load balancer during setup? – sramalingam24 just now By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continu...

Elastic Search high CPU Usage with More Like This query

Image
Clash Royale CLAN TAG #URR8PPP Elastic Search high CPU Usage with More Like This query We tried using a more_like_this query in ES on a text field with 500-1500 words in each post and about 500,000 posts. The CPU usage was very high (almost 100%), but it looks like JVM Memory Pressure was normal. more_like_this One optimization that I think will help is adding/saving term_vectors . Will that help with CPU utilization? Trying to understand why this happened in the first place. term_vectors By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

How to perform search query on two different data types?

Image
Clash Royale CLAN TAG #URR8PPP How to perform search query on two different data types? my query is very simple, for the sake of even making it simpler, lets say I only search on two fields, name(text) & age(long): GET person_db/person/_search { "query": { "bool": { "should": [ { "match_phrase_prefix": { "name": "hank" } }, { "match_phrase_prefix": { "age": "hank" } } ], "minimum_should_match": 1, "boost": 1.0 } } } if I search for "23", no problem, elastic knows how to change it to numeric and it won't fail, but if the search input is "john" I get error 400 "reason": "failed to create query: {n "bool...." . "reason": "failed to create query: {n "bool...." what should I...

kubernetes : expost service internally within VPN

Image
Clash Royale CLAN TAG #URR8PPP kubernetes : expost service internally within VPN I've setup kubernetes cluster using kops kops create cluster --dns-zone=vpc.pharmeasy.in --dns=private --master-zones=ap-southeast-1a,ap-southeast-1b,ap-southeast-1c --zones=ap-southeast-1a,ap-southeast-1b,ap-southeast-1c --node-count 3 --topology private --networking flannel-vxlan --node-size=t2.medium --master-size=t2.micro ${NAME} Also, we have used internal LoadBalancer for Kubernetes api server. Whenever i create a service of type=LoadBalancer it creates internet-facing load balancer and is available publically. I tried created service using ClusterIp And exposed using Kubectl proxy, but it is not accessible. We already have VPN setup. We want elasticsearch and kibana dashboard available only within vpn. How to expose service internally with VPN? By clicking "Post Your Answer", you...

Elasticsearch 6.3 - How to enable SSL in elasticsearch

Image
Clash Royale CLAN TAG #URR8PPP Elasticsearch 6.3 - How to enable SSL in elasticsearch Any idea how to enable https in elastic search. Give me the steps to understand by developer 1 Answer 1 You can specify SSL options for any output that supports SSL. You can also specify SSL options when you set up the Kibana endpoint. Example output config with SSL enabled: output.elasticsearch.hosts: ["192.168.1.42:9200"] output.elasticsearch.ssl.certificate_authorities: ["/etc/pki/root/ca.pem"] output.elasticsearch.ssl.certificate: "/etc/pki/client/cert.pem" output.elasticsearch.ssl.key: "/etc/pki/client/cert.key" Example Kibana endpoint config with SSL enabled: setup.kibana.host: "192.0.2.255:5601" setup.kibana.protocol: "https" setup.kibana.ssl.enabled: true setup.kibana.ssl.certificate_authorities: ["/etc/pki/root/ca.pem"] setup.kibana.ssl....

Which is the best way to index the data from relational database table of One to many relationship

Image
Clash Royale CLAN TAG #URR8PPP Which is the best way to index the data from relational database table of One to many relationship Can you please let me know which is the best way to index the records in elastic search for my scenario. My Scenario is : 1) Need to index around 40 million records from oracle table which has entries having one to many relationship records. And the uniqueness of the records is based on the composite key with 4 columns 2) After indexing , Search should support "full text search" on all the fields 3) Filters and sorting on selected fields needs to be supported. After going through the official documentation i found couple of options , but want to know which approach would be most useful among below 1) For each record in table create a entry in the elastic index 2) Create a nested json object based on the composite key and then add this elastic index 3)Parent child Relationship mechanism and application side joins are not suitable for my scenario Tha...

how to add to different field values and store it in another field in elastic search python (elastic search field operation)

Image
Clash Royale CLAN TAG #URR8PPP how to add to different field values and store it in another field in elastic search python (elastic search field operation) I have written this elastic search query: es.search(index=['ind1'],doc_type=['doc']) I am getting following result: {'_shards': {'failed': 0, 'skipped': 0, 'successful': 5, 'total': 5}, 'hits': {'hits': [{'_id': '1327', '_index': 'ind1', '_score': 1.0, '_source': {'val1': 1, 'val2': None, 'value1': 1327, 'value2': 1531, 'new_values': {'nv1': 1, 'nv2': 0}, {'_id': '1349', '_index': 'ind1', '_score': 1.0, '_source': {'val1': 2, 'val2': 3, 'value1': 1328, 'value2': 1539, 'new_values': {'nv1': 1, 'nv2': 3}},....... I want all the add val1, value1 and...

Elastic search error - was expecting double-quote to start field name

Image
Clash Royale CLAN TAG #URR8PPP Elastic search error - was expecting double-quote to start field name Am trying to create a mapping for my new index named as region . Please find my below mapping file. region PUT region { "mappings": { "doc": { "properties": { "catalog_product_id": { "type": "long", }, "id": { "type": "long" }, "region_id":{ "type": "text" }, "region_type":{ "type" : "text" } } } } } } While am trying to execute this mapping script am getting the below error was expecting double-quote to start field name try to check your json for validity of any online service and you will find an error – Oleg SH ...