ComputeNonCryptHash

Compute a non-cryptographic hash

This function is intended to be fast, but without requiring a Java or mex implementation to do the actual hashing. It was not checked for any security flaws and is therefore probably vulnerable to most attacks.

Non-cryptographic hashes should only be used as a checksum. Don't use this to do things like storing passwords.

This function will transform most common data types to a uint16 vector to apply the hash in an array operation. Changing the data type should change the hash. The allowed data types are uint*, int*, char, cell, struct, double, single, and string (which is cast to cell array of chars). The contents of the nested data types (i.e. cell and struct) must also be one of the mentioned data types.

Version 1.x of this algorithm attempts to cast string to char, instead of a cell array of chars. Version 1.x also has many hash collisions for scalar doubles. Version 2 will transcode the UTF-8 chars on Octave to UTF-16 (the Matlab standard), which ensures that the same Unicode code points as input will return the same hash.

Performance was tested with a 216553 items long English word list in both upper and lower case (this list with the two duplicates removed) and the numbers 0-1e6 as char and double. An additional test was performed with the images from the Stanford Dog Dataset containing 20580 images (the 89 duplicates were removed from this tar file before running this test). Timings below were determined on R2020b on Windows 10. For a comparison with other hash functions, see this SE thread. Note that these tests are different from the relative performance comparison.

<markdown-accessiblity-table><table class="readme_table"> <thead> <tr> <th class="readme_th">Hash length</th> <th class="readme_th">English words</th> <th class="readme_th">Numbers (in char)</th> <th class="readme_th">Numbers (in double)</th> <th class="readme_th">Images</th> </tr> </thead> <tbody> <tr> <td class="readme_td">16 bits</td> <td class="readme_td">50 μs/hash<br> 391 630 collisions</td> <td class="readme_td">50 μs/hash<br> 958 488 collisions</td> <td class="readme_td">56 μs/hash<br> 958 487 collisions</td> <td class="readme_td">106 129 μs/hash<br>4 976 collisions</td> </tr> <tr> <td class="readme_td">32 bits</td> <td class="readme_td">58 μs/hash<br>305 collisions</td> <td class="readme_td">55 μs/hash<br>31 056 collisions</td> <td class="readme_td">61 μs/hash<br>120 collisions</td> <td class="readme_td">105 571 μs/hash<br>0 collisions</td> </tr> <tr> <td class="readme_td">48 bits</td> <td class="readme_td">64 μs/hash<br>144 collisions</td> <td class="readme_td">63 μs/hash<br>0 collisions</td> <td class="readme_td">67 μs/hash<br>16 033 collisions</td> <td class="readme_td">105 288 μs/hash<br>0 collisions</td> </tr> <tr> <td class="readme_td">64 bits</td> <td class="readme_td">60 μs/hash<br>1 collisions</td> <td class="readme_td">56 μs/hash<br>0 collisions</td> <td class="readme_td">62 μs/hash<br>0 collisions</td> <td class="readme_td">105 366 μs/hash<br>0 collisions</td> </tr> <tr> <td class="readme_td">128 bits</td> <td class="readme_td">65 μs/hash<br>0 collisions</td> <td class="readme_td">58 μs/hash<br>0 collisions</td> <td class="readme_td">69 μs/hash<br>0 collisions</td> <td class="readme_td">106 289 μs/hash<br>0 collisions</td> </tr> <tr> <td class="readme_td">192 bits</td> <td class="readme_td">70 μs/hash<br>0 collisions</td> <td class="readme_td">68 μs/hash<br>0 collisions</td> <td class="readme_td">72 μs/hash<br>0 collisions</td> <td class="readme_td">105 932 μs/hash<br>0 collisions</td> </tr> <tr> <td class="readme_td">256 bits</td> <td class="readme_td">83 μs/hash<br>0 collisions</td> <td class="readme_td">85 μs/hash<br>0 collisions</td> <td class="readme_td">92 μs/hash<br>0 collisions</td> <td class="readme_td">106 187 μs/hash<br>0 collisions</td> </tr> </tbody> </table></markdown-accessiblity-table>

Licence: CC by-nc-sa 4.0