An idiomatic Go query builder for ElasticSearch https://github.com/aquasecurity/esquery.git
Go to file
Ido Perlmuter 1dd88421a2 Refactor API, add aggregations and custom queries
This commit introduces a refactor of the codebase and the API, to make
it more user friendly. Queries can now directly be executed via the
`Run()` method. Internally, the library no longer uses JSON generation
as a major mechanism, instead all types need to implement a `Mappable`
interface which simply turns each type in a `map[string]interface{}`,
which is what the ElasticSearch client expects. This makes the code
easier to write, and makes writing tests less error prone, as JSON need
not be written directly.

Support for metrics aggregations is also added. However, aggregations of
type bucket, pipeline and matrix are not supported yet.

To make the library more useful in its current state, support is added
for running custom queries and aggregations, via the `CustomQuery()` and
`CustomAgg()` functions, which both accepts an arbitrary
`map[string]interface{}`.
2020-02-20 11:50:11 +02:00
.golangci.yml Add Search() function, README and fix some lint errors 2020-02-20 11:50:11 +02:00
LICENSE Create LICENSE 2020-02-20 11:46:34 +02:00
README.md Refactor API, add aggregations and custom queries 2020-02-20 11:50:11 +02:00
aggregations.go Refactor API, add aggregations and custom queries 2020-02-20 11:50:11 +02:00
aggregations_test.go Refactor API, add aggregations and custom queries 2020-02-20 11:50:11 +02:00
aggs_custom.go Refactor API, add aggregations and custom queries 2020-02-20 11:50:11 +02:00
aggs_custom_test.go Refactor API, add aggregations and custom queries 2020-02-20 11:50:11 +02:00
aggs_metric.go Refactor API, add aggregations and custom queries 2020-02-20 11:50:11 +02:00
aggs_metric_test.go Refactor API, add aggregations and custom queries 2020-02-20 11:50:11 +02:00
es.go Refactor API, add aggregations and custom queries 2020-02-20 11:50:11 +02:00
es_test.go Refactor API, add aggregations and custom queries 2020-02-20 11:50:11 +02:00
go.mod Refactor API, add aggregations and custom queries 2020-02-20 11:50:11 +02:00
go.sum Refactor API, add aggregations and custom queries 2020-02-20 11:50:11 +02:00
queries.go Refactor API, add aggregations and custom queries 2020-02-20 11:50:11 +02:00
queries_test.go Refactor API, add aggregations and custom queries 2020-02-20 11:50:11 +02:00
query_boolean.go Refactor API, add aggregations and custom queries 2020-02-20 11:50:11 +02:00
query_boolean_test.go Refactor API, add aggregations and custom queries 2020-02-20 11:50:11 +02:00
query_boosting.go Refactor API, add aggregations and custom queries 2020-02-20 11:50:11 +02:00
query_boosting_test.go Refactor API, add aggregations and custom queries 2020-02-20 11:50:11 +02:00
query_constant_score.go Refactor API, add aggregations and custom queries 2020-02-20 11:50:11 +02:00
query_constant_score_test.go Refactor API, add aggregations and custom queries 2020-02-20 11:50:11 +02:00
query_custom.go Refactor API, add aggregations and custom queries 2020-02-20 11:50:11 +02:00
query_custom_test.go Refactor API, add aggregations and custom queries 2020-02-20 11:50:11 +02:00
query_dis_max.go Refactor API, add aggregations and custom queries 2020-02-20 11:50:11 +02:00
query_dis_max_test.go Refactor API, add aggregations and custom queries 2020-02-20 11:50:11 +02:00
query_match.go Refactor API, add aggregations and custom queries 2020-02-20 11:50:11 +02:00
query_match_all.go Refactor API, add aggregations and custom queries 2020-02-20 11:50:11 +02:00
query_match_all_test.go Refactor API, add aggregations and custom queries 2020-02-20 11:50:11 +02:00
query_match_test.go Refactor API, add aggregations and custom queries 2020-02-20 11:50:11 +02:00
query_term_level.go Refactor API, add aggregations and custom queries 2020-02-20 11:50:11 +02:00
query_term_level_test.go Refactor API, add aggregations and custom queries 2020-02-20 11:50:11 +02:00

README.md

esquery

esquery is a non-obtrusive, idiomatic and easy-to-use query and aggregation builder for the official Go client for ElasticSearch. It alleviates the need to use extremely nested maps (map[string]interface{}) and serializing queries to JSON manually. It also helps eliminating common mistakes such as misspelling query types, as everything is statically typed.

Save yourself some joint aches and many lines of code by switching for maps to esquery. Wanna know how much code you'll save? just read this project's test.

Usage

esquery provides a method chaining-style API for building and executing queries and aggregations. It does not wrap the official Go client nor does it require you to change your existing code in order to integrate the library. Queries can be directly built with esquery, and executed by passing an *elasticsearch.Client instance (with optional search parameters). Results are returned as-is from the official client (e.g. *esapi.Response objects).

Getting started is extremely simple:

package main

import (
	"context"
	"log"

	"bitbucket.org/scalock/esquery"
	"github.com/elastic/go-elasticsearch/v7"
)

func main() {
    // connect to an ElasticSearch instance
	es, err := elasticsearch.NewDefaultClient()
	if err != nil {
		log.Fatalf("Failed creating client: %s", err)
	}

    // run a boolean search query
	qRes, err := esquery.Query(
		esquery.
			Bool().
			Must(esquery.Term("title", "Go and Stuff")).
			Filter(esquery.Term("tag", "tech")),
    ).Run(
        es, 
		es.Search.WithContext(context.TODO()),
		es.Search.WithIndex("test"),
	)
	if err != nil {
		log.Fatalf("Failed searching for stuff: %s", err)
	}

	defer qRes.Body.Close()

	// run an aggregation
	aRes, err := esquery.Aggregate(
		esquery.Avg("average_score", "score"),
		esquery.Max("max_score", "score"),
	).Run(
		es,
		es.Search.WithContext(context.TODO()),
		es.Search.WithIndex("test"),
	)
	if err != nil {
		log.Fatalf("Failed searching for stuff: %s", err)
	}

	defer aRes.Body.Close()

    // ...
}

Notes

  • esquery currently supports version 7 of the ElasticSearch Go client.
  • The library cannot currently generate "short queries". For example, whereas ElasticSearch can accept this:
{ "query": { "term": { "user": "Kimchy" } } }

The library will always generate this:

{ "query": { "term": { "user": { "value": "Kimchy" } } } }

This is also true for queries such as "bool", where fields like "must" can either receive one query object, or an array of query objects. esquery will generate an array even if there's only one query object.

Supported Queries

The following queries are currently supported:

ElasticSearch DSL esquery Function
"match" Match()
"match_bool_prefix" MatchBoolPrefix()
"match_phrase" MatchPhrase()
"match_phrase_prefix" MatchPhrasePrefix()
"match_all" MatchAll()
"match_none" MatchNone()
"exists" Exists()
"fuzzy" Fuzzy()
"ids" IDs()
"prefix" Prefix()
"range" Range()
"regexp" Regexp()
"term" Term()
"terms" Terms()
"terms_set" TermsSet()
"wildcard" Wildcard()
"bool" Bool()
"boosting" Boosting()
"constant_score" ConstantScore()
"dis_max" DisMax()

Custom Queries

To execute an arbitrary query, or any query that is not natively supported by the library yet, use the CustomQuery() function, which accepts any map[string]interface{} value.

Supported Aggregations

The following aggregations are currently supported:

ElasticSearch DSL esquery Function
"avg" Avg()
"weighted_avg" WeightedAvg()
"cardinality" Cardinality()
"max" Max()
"min" Min()
"sum" Sum()
"value_count" ValueCount()
"percentiles" Percentiles()
"stats" Stats()
"string_stats" StringStats()

Custom Aggregations

To execute an arbitrary aggregation, or any aggregation that is not natively supported by the library yet, use the CustomAgg() function, which accepts any map[string]interface{} value.