2020-02-27 14:19:07 +00:00
|
|
|
// Package esquery provides a non-obtrusive, idiomatic and easy-to-use query
|
|
|
|
// and aggregation builder for the official Go client
|
|
|
|
// (https://github.com/elastic/go-elasticsearch) for the ElasticSearch
|
|
|
|
// database (https://www.elastic.co/products/elasticsearch).
|
|
|
|
//
|
|
|
|
// esquery alleviates the need to use extremely nested maps
|
|
|
|
// (map[string]interface{}) and serializing queries to JSON manually. It also
|
|
|
|
// helps eliminating common mistakes such as misspelling query types, as
|
|
|
|
// everything is statically typed.
|
|
|
|
//
|
|
|
|
// Using `esquery` can make your code much easier to write, read and maintain,
|
|
|
|
// and significantly reduce the amount of code you write.
|
|
|
|
//
|
|
|
|
//
|
|
|
|
//
|
|
|
|
// Usage
|
|
|
|
//
|
|
|
|
//
|
|
|
|
//
|
|
|
|
// esquery provides a method chaining-style API for building and executing
|
|
|
|
// queries and aggregations. It does not wrap the official Go client nor does it
|
|
|
|
// require you to change your existing code in order to integrate the library.
|
|
|
|
// Queries can be directly built with `esquery`, and executed by passing an
|
|
|
|
// `*elasticsearch.Client` instance (with optional search parameters). Results
|
|
|
|
// are returned as-is from the official client (e.g. `*esapi.Response` objects).
|
|
|
|
//
|
|
|
|
// Getting started is extremely simple:
|
|
|
|
//
|
|
|
|
// package main
|
|
|
|
//
|
|
|
|
// import (
|
|
|
|
// "context"
|
|
|
|
// "log"
|
|
|
|
//
|
|
|
|
// "github.com/aquasecurity/esquery"
|
|
|
|
// "github.com/elastic/go-elasticsearch/v7"
|
|
|
|
// )
|
|
|
|
//
|
|
|
|
// func main() {
|
|
|
|
// // connect to an ElasticSearch instance
|
|
|
|
// es, err := elasticsearch.NewDefaultClient()
|
|
|
|
// if err != nil {
|
|
|
|
// log.Fatalf("Failed creating client: %s", err)
|
|
|
|
// }
|
|
|
|
//
|
|
|
|
// // run a boolean search query
|
|
|
|
// qRes, err := esquery.Query(
|
|
|
|
// esquery.
|
|
|
|
// Bool().
|
|
|
|
// Must(esquery.Term("title", "Go and Stuff")).
|
|
|
|
// Filter(esquery.Term("tag", "tech")),
|
|
|
|
// ).Run(
|
|
|
|
// es,
|
|
|
|
// es.Search.WithContext(context.TODO()),
|
|
|
|
// es.Search.WithIndex("test"),
|
|
|
|
// )
|
|
|
|
// if err != nil {
|
|
|
|
// log.Fatalf("Failed searching for stuff: %s", err)
|
|
|
|
// }
|
|
|
|
//
|
|
|
|
// defer qRes.Body.Close()
|
|
|
|
//
|
|
|
|
// // run an aggregation
|
|
|
|
// aRes, err := esquery.Aggregate(
|
|
|
|
// esquery.Avg("average_score", "score"),
|
|
|
|
// esquery.Max("max_score", "score"),
|
|
|
|
// ).Run(
|
|
|
|
// es,
|
|
|
|
// es.Search.WithContext(context.TODO()),
|
|
|
|
// es.Search.WithIndex("test"),
|
|
|
|
// )
|
|
|
|
// if err != nil {
|
|
|
|
// log.Fatalf("Failed searching for stuff: %s", err)
|
|
|
|
// }
|
|
|
|
//
|
|
|
|
// defer aRes.Body.Close()
|
|
|
|
//
|
|
|
|
// // ...
|
|
|
|
// }
|
|
|
|
//
|
|
|
|
//
|
|
|
|
//
|
|
|
|
// Notes
|
|
|
|
//
|
|
|
|
//
|
|
|
|
//
|
|
|
|
//* esquery currently supports version 7 of the ElasticSearch Go client.
|
|
|
|
//* The library cannot currently generate "short queries". For example,
|
|
|
|
// whereas ElasticSearch can accept this:
|
|
|
|
//
|
|
|
|
// { "query": { "term": { "user": "Kimchy" } } }
|
|
|
|
//
|
|
|
|
// The library will always generate this:
|
|
|
|
//
|
|
|
|
// { "query": { "term": { "user": { "value": "Kimchy" } } } }
|
|
|
|
//
|
|
|
|
// This is also true for queries such as "bool", where fields like "must" can
|
|
|
|
// either receive one query object, or an array of query objects. `esquery` will
|
|
|
|
// generate an array even if there's only one query object.
|
Initial commit
This commit is the initial commit for a Go library providing an
idiomatic, easy-to-use query builder for ElasticSearch. The library can
build queries and execute them using the structures from the official Go
SDK provided by the ES project (https://github.com/elastic/go-elasticsearch).
The library currently provides the capabilities to create and execute
simple ElasticSearch queries, specifically Match queries (match,
match_bool_prefix, match_phrase and match_phrase_prefix), Match All
queries (match_all, match_none), and all of the Term-level queries (e.g.
range, regexp, etc.).
Unit tests are included for each support query, and the code is linted
using golangci-lint (see enabled linters in .golangci-lint). The unit
tests currently only verify the builder creates valid JSON queries and
does not attempt to actually run queries against a (mock) ES instance.
2020-02-18 12:00:15 +00:00
|
|
|
package esquery
|
|
|
|
|
2020-02-27 14:19:07 +00:00
|
|
|
// Mappable is the interface implemented by the various query and aggregation
|
|
|
|
// types provided by the package. It allows the library to easily transform the
|
|
|
|
// different queries to "generic" maps that can be easily encoded to JSON.
|
Refactor API, add aggregations and custom queries
This commit introduces a refactor of the codebase and the API, to make
it more user friendly. Queries can now directly be executed via the
`Run()` method. Internally, the library no longer uses JSON generation
as a major mechanism, instead all types need to implement a `Mappable`
interface which simply turns each type in a `map[string]interface{}`,
which is what the ElasticSearch client expects. This makes the code
easier to write, and makes writing tests less error prone, as JSON need
not be written directly.
Support for metrics aggregations is also added. However, aggregations of
type bucket, pipeline and matrix are not supported yet.
To make the library more useful in its current state, support is added
for running custom queries and aggregations, via the `CustomQuery()` and
`CustomAgg()` functions, which both accepts an arbitrary
`map[string]interface{}`.
2020-02-19 11:35:21 +00:00
|
|
|
type Mappable interface {
|
|
|
|
Map() map[string]interface{}
|
Initial commit
This commit is the initial commit for a Go library providing an
idiomatic, easy-to-use query builder for ElasticSearch. The library can
build queries and execute them using the structures from the official Go
SDK provided by the ES project (https://github.com/elastic/go-elasticsearch).
The library currently provides the capabilities to create and execute
simple ElasticSearch queries, specifically Match queries (match,
match_bool_prefix, match_phrase and match_phrase_prefix), Match All
queries (match_all, match_none), and all of the Term-level queries (e.g.
range, regexp, etc.).
Unit tests are included for each support query, and the code is linted
using golangci-lint (see enabled linters in .golangci-lint). The unit
tests currently only verify the builder creates valid JSON queries and
does not attempt to actually run queries against a (mock) ES instance.
2020-02-18 12:00:15 +00:00
|
|
|
}
|