Missing Analytics when there's a bottleneck in one of our backends in pump setup

Support
,
1

Hello,

there’s an unknown issue with our elastic backend side which make our elastic slowdown, that seems make our pump not pumping to hybrid too, so basically our pump stuck pumping into elastic before pumping into hybrid because of elastic bottleneck that cause to much analytics lost.

is this suppose to be the expected behaviour ?
is there a workaround that make our pump still pumping to hybrid to even when our elastic is slowingdown vice versa ?

This is our pump configuration

Thanks in advance

{
  "analytics_storage_type": "redis",
  "analytics_storage_config": {
    "type": "redis",
    "hosts": null,
    "username": "(username)",
    "password": "(password)",
    "addrs": [
      "(redis connection)"
    ],
    "database": 0,
    "optimisation_max_idle": 100,
    "optimisation_max_active": 0,
    "redis_use_ssl": false,
    "enable_cluster": false
  },
  "purge_delay": 5,
  "pumps": {
    "elasticsearch": {
      "type": "elasticsearch",
      "meta": {
        "elasticsearch_url": "(my elastic url)",
        "use_sniffing": true,
        "version": "7",
        "extended_stats": true,
        "max_record_size": 50000,
        "decode_base64": true,
	"index_name": (my index_name)",
	"disable_bulk": true
      }
    },
    "hybrid": {
      "type": "hybrid",
      "meta": {
        "rpc_key": "(my rpc key)",
        "api_key": "(my api key)",
        "connection_string": "(my mdcb url)",
        "use_ssl": true,
        "ssl_insecure_skip_verify": false,
        "group_id": "(group_id)",
	"call_timeout": 30,
        "ping_timeout": 60,
        "rpc_pool_size": 5
      },
      "omit_detailed_recording": false

    }
  },
  "dont_purge_uptime_data": true,
  "raw_response_decoded": true,
  "raw_request_decoded": true
}
2

@Mar Welcome to the community :tada:

there’s an unknown issue with our elastic backend side which make our elastic slowdown, that seems make our pump not pumping to hybrid too, so basically our pump stuck pumping into elastic before pumping into hybrid because of elastic bottleneck that cause to much analytics lost.
is this suppose to be the expected behaviour ?

The issue sounds like a similar issue with Kafka on this thread.

is there a workaround that make our pump still pumping to hybrid to even when our elastic is slowingdown vice versa ?

You would need to set up a pump timeout. The timeout ensures there isn’t severe interruptions pump instance when purging records to to the elastic search data sink.