Elasticsearch (Database)¶

Overview¶

While primarily used for logging, Elasticsearch also serves as a search engine for application data.

Note

For logging-specific documentation, see Monitoring > Elasticsearch

Use Cases¶

Use Case	Description
Log aggregation	Centralized logging
Full-text search	Application search
Analytics	Data analysis
Threat intelligence	IOC storage

Deployment¶

Configuration¶

See Monitoring > Elasticsearch for deployment details.

Connection¶

Property	Value
Host	elasticsearch.monitoring.svc
Port	9200
Protocol	HTTP/HTTPS

Search Features¶

Full-Text Search¶

GET /documents/_search
{
  "query": {
    "match": {
      "content": "kubernetes deployment"
    }
  }
}

Fuzzy Search¶

GET /documents/_search
{
  "query": {
    "fuzzy": {
      "title": {
        "value": "kuberntes",
        "fuzziness": "AUTO"
      }
    }
  }
}

Highlighting¶

GET /documents/_search
{
  "query": {
    "match": { "content": "search term" }
  },
  "highlight": {
    "fields": { "content": {} }
  }
}

Mappings¶

Creating Index with Mapping¶

PUT /documents
{
  "mappings": {
    "properties": {
      "title": { "type": "text" },
      "content": { "type": "text" },
      "category": { "type": "keyword" },
      "created": { "type": "date" },
      "views": { "type": "integer" }
    }
  }
}

Field Types¶

Type	Use Case
text	Full-text search
keyword	Exact match, aggregations
date	Timestamps
integer/long	Numbers
boolean	True/false
object	Nested JSON

Aggregations¶

Terms Aggregation¶

GET /logs/_search
{
  "size": 0,
  "aggs": {
    "by_namespace": {
      "terms": { "field": "kubernetes.namespace" }
    }
  }
}

Date Histogram¶

GET /logs/_search
{
  "size": 0,
  "aggs": {
    "logs_over_time": {
      "date_histogram": {
        "field": "@timestamp",
        "calendar_interval": "hour"
      }
    }
  }
}

Application Integration¶

.NET Client¶

var settings = new ElasticsearchClientSettings(
    new Uri("http://elasticsearch:9200"))
    .DefaultIndex("documents");

var client = new ElasticsearchClient(settings);

// Search
var response = await client.SearchAsync<Document>(s => s
    .Query(q => q
        .Match(m => m
            .Field(f => f.Content)
            .Query("search term")
        )
    )
);

Index Document¶

var document = new Document {
    Title = "My Document",
    Content = "Document content here"
};

await client.IndexAsync(document);

Performance¶

Index Settings¶

PUT /my-index/_settings
{
  "index": {
    "refresh_interval": "30s",
    "number_of_replicas": 1
  }
}

Query Optimization¶

Use filters for non-scoring queries
Limit returned fields with _source
Use scroll/search_after for large results
Enable doc_values for aggregations

Comparison with Other DBs¶

Feature	Elasticsearch	MongoDB	MySQL
Full-text search	Excellent	Good	Basic
Aggregations	Excellent	Good	Good
ACID	No	Partial	Yes
Real-time	Near real-time	Real-time	Real-time
Schema	Flexible	Flexible	Strict

When to Use¶

Use Elasticsearch for:

Log analysis
Full-text search
Analytics dashboards
Threat intelligence

Don't use for:

Primary data store
Transactional data
Strong consistency requirements