Mailing List Archive

I won't speak to the rsyslog work or what can/can't be done from there, but
isn't this something that you would rather setup when you create your index
templates within ES itself?

I think you would be better off creating your index template which contains
your sort. That said, you still might run into trouble with nested fields,
depending on what your data looks like.

Could you not just do this before the index is populated?

curl -XPUT -H "Content Type: application/json"
127.0.0.1:9200/_template/squid -d 'squid.template

{
"template": "squid-*",
"settings": { "number_of_shards": 1 , "number_of_replicas":0,
"index.routing.allocation.include.zone":"ssd", "index.sort.field":"date",
"index.sort.order":"desc"},
"mappings": {
"properties" : {
"@timestamp" : {
"type" : "date"
}},
"dynamic_templates": [.
{
"strings": {
"match_mapping_type": "string",
"mapping": {
"type": "text",
"fields": {
"raw": {
"type": "keyword",
"ignore_above": 8192
}
}
}
}
}
]
}
}
}'

As a general rule of thumb, it's normally better off to have your ES
instance configured to receive/know about your data feeds ahead of time,
instead of using default configurations with data you're chucking into ES.
Just my 0.02.

Cheers,

JB

On Fri, Jan 8, 2021 at 10:37 AM kyparisisg via rsyslog <
rsyslog@lists.adiscon.com> wrote:

> Hello everyone,
>
> I have a question regarding the capabilities of an elasticsearch action.
>
> Elasticsearch allows you to ingest data and sort the data during the
> ingestion.
>
> Is there a way to specify and use that feature of Elasticsearch within the
> Rsyslog configuration file?
>
> Currently I am only using the action below, and I cannot find a way to
> specify and use the ingestion sorting feature of elastic.
>
> action(
> type="omelasticsearch"
> server="localhost"
> serverport="9200"
> template="my-template"
> searchIndex="my-index"
> dynSearchIndex="on"
> searchType="_doc"
> bulkmode="on" # use
> the Bulk
> API
> queue.dequeuebatchsize="1000" # of up to 1000
> queue.workerthreads="5"
> action.resumeretrycount="-1" # retry
> indefinitely
> if ES is unreachable
> errorfile="/var/log/omelasticsearch.log"
> )
>
> The ingestion sorting feature is here:
>
> https://www.elastic.co/guide/en/elasticsearch/reference/master/index-modules-index-sorting.html
>
> I understand that this should configured when the index is first created,
> but I am asking if there is a way that can be configured through Rsyslog.
>
> Thank you very much in advance!
>
>
>
>
> --
> Sent from: http://rsyslog-users.1305293.n2.nabble.com/
> _______________________________________________
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.