Skip to main content
Version: edge

chash

The chash module contains functions for consistent hashing of values. This can be used to achieve consistent routing over multiple outputs.

Functions​

jump(key, slot_count)​

Hashes an input key (string) and determine its placement in a slot list.

For example It can be used to pick a routing destination using an array of hosts:

use tremor::chash;
 let hosts = ["host1", "host2", "host3", "host4", "host5"];
{
 "key1": hosts[chash::jump("key1", array::len(hosts))],
 "key1_again": hosts[chash::jump("key1", array::len(hosts))],
 "key2": hosts[chash::jump("key2", array::len(hosts))],
 "key3": hosts[chash::jump("key3", array::len(hosts))],
 "key4": hosts[chash::jump("key4", array::len(hosts))],
}

Returns an integer.

jump_with_keys(k1, k2, key, slot_count)​

The same as chash::jump but uses the integers k1 and k2 to initialise the hashing instead of using default values.

use tremor::chash;
 let hosts = ["host1", "host2", "host3", "host4", "host5"];
{
 "key1": hosts[chash::jump_with_keys(7, 42, "key1", array::len(hosts))],
 "key1_again": hosts[chash::jump_with_keys(7, 42, "key1", array::len(hosts))],
 "key2": hosts[chash::jump_with_keys(7, 42, "key2", array::len(hosts))],
 "key3": hosts[chash::jump_with_keys(7, 42, "key3", array::len(hosts))],
 "key4": hosts[chash::jump_with_keys(7, 42, "key4", array::len(hosts))],
}

Returns an integer

sorted_serialize(any)​

Serialises the given data in a sorted and repeatable fashion no matter how data is internally stored. In comparison, the normal serialisation functions do not ensure order for performance reasons. Their behaviour is well suited for encoding data on the wire, but in the context of consistent hashing we need to guarantee that data is always encoded to the same serialisation on a byte level not only on the logical level

use tremor::chash;
{
  "s1": chash::sorted_serialize("snot"),
  # Arrays serialize preserving element order
  "s2": chash::sorted_serialize(["badger", "snot"]),
  "s3": chash::sorted_serialize(["snot", "badger"]),
  # Records serialize *normalizing* field element order
  "s4": chash::sorted_serialize({ "badger": "snot", "snot": "badger"}),
  "s5": chash::sorted_serialize({ "snot": "badger", "badger": "snot"}),
}

Returns an string